Large-N comparative studies have helped common pool resource scholars gain general insights into the factors that influence collective action and governance outcomes. However, these studies are often limited by missing data, and suffer from the methodological limitation that important information is lost when we reduce textual information to quantitative data. This study was motivated by nine case studies that appeared to be inconsistent with the expectation that the presence of Ostrom’s Design Principles increases the likelihood of successful common pool resource governance. These cases highlight the limitations of coding and analyzing Large-N case studies.

We examine two issues: 1) the challenge of missing data and 2) potential approaches that rely on context (which is often lost in the coding process) to address inconsistencies between empirical observations theoretical predictions. For the latter, we conduct a post-hoc qualitative analysis of a large-N comparative study to explore 2 types of inconsistencies: 1) cases where evidence for nearly all design principles was found, but available evidence led to the assessment that the CPR system was unsuccessful and 2) cases where the CPR system was deemed successful despite finding limited or no evidence for design principles. We describe inherent challenges to large-N comparative analysis to coding complex and dynamically changing common pool resource systems for the presence or absence of design principles and the determination of “success”. Finally, we illustrate how, in some cases, our qualitative analysis revealed that the identity of absent design principles explained inconsistencies hence de-facto reconciling such apparent inconsistencies with theoretical predictions. This analysis demonstrates the value of combining quantitative and qualitative analysis, and using mixed-methods approaches iteratively to build comprehensive methodological and theoretical approaches to understanding common pool resource governance in a dynamically changing context.

Social roles are thought to play an important role in determining the capacity for collective action in a community regarding the use of shared resources. Here we report on the results of a study using a behavioral experimental approach regarding the relationship between social roles and the performance of social-ecological systems. The computer-based irrigation experiment that was the basis of this study mimics the decisions faced by farmers in small-scale irrigation systems. In each of 20 rounds, which are analogous to growing seasons, participants face a two-stage commons dilemma. First they must decide how much to invest in the public infrastructure, e.g., canals and water diversion structures. Second, they must decide how much to extract from the water made available by that public infrastructure. Each round begins with a 60-second communication period before the players make their investment and extraction decisions. By analyzing the chat messages exchanged among participants during the communication stage of the experiment, we coded up to three roles per participant using the scheme of seven roles known to be important in the literature: leader, knowledge generator, connector, follower, moralist, enforcer, and observer. Our study supports the importance of certain social roles (e.g., connector) previously highlighted by several case study analyses. However, using qualitative comparative analysis we found that none of the individual roles was sufficient for groups to succeed, i.e., to reach a certain level of group production. Instead, we found that a combination of at least five roles was necessary for success. In addition, in the context of upstream-downstream asymmetry, we observed a pattern in which social roles assumed by participants tended to differ by their positions. Although our work generated some interesting insights, further research is needed to determine how robust our findings are to different action situations, such as biophysical context, social network, and resource uncertainty.