Methodological Issues in Socio-Legal Research on the Rule of Law
- Authors: Allalyev R.M.1,2
-
Affiliations:
- RUDN University
- Russian State University of Oil and Gas (National Research University) named after I.M. Gubkin
- Issue: Vol 30, No 2 (2026)
- Pages: 274-288
- Section: LEGAL RESEARCH METHODOLOGY
- URL: https://journals.rudn.ru/law/article/view/51004
- DOI: https://doi.org/10.22363/2313-2337-2026-30-2-274-288
- EDN: https://elibrary.ru/DEXDFD
- ID: 51004
Cite item
Abstract
The global debate on the rule of law is undergoing a methodological crisis in empirical research, manifested in problems of operationalization, cultural bias in indices, and a persistent gap between formal institutions and actual practices. This article aims to trace the evolution of approaches to the rule of law in legal sociology, identify the methodological limitations of three successive “generations” of studies, and outline prospects for synthetic methodology. The analysis draws on key works in socio-legal studies (R. Abel, M. Galanter, M. Hertogh, N. Cheesman, T. Ginsburg) and on a critical reading of global index methodologies (World Justice Project, Worldwide Governance Indicators). Methodologically, it employs historical-sociological and comparative-legal approaches. The first generation of empirical research - classic “gap studies” - demonstrated the systemic failure of legal reforms that ignore the social context, revealing the divergence between official legal norms and everyday normative orders. The second generation, centered on quantitative indices, made the rule of law measurable across countries but produced new difficulties, including expert dependence, data endogeneity, and the risk that governments use indicators to simulate reform rather than constrain power. The third generation, associated with ethnography and studies of legal consciousness, demonstrated the cultural embeddedness and plurality of legal meanings, while also exposing the challenges of aggregating and generalizing qualitative findings for comparative assessment. Overcoming the current methodological impasse requires a multi-dimensional research design that integrates quantitative assessment of institutions, qualitative analysis of legal consciousness, and everyday legality, and sustained theoretical reflections on law’s relationship to power. Such an interdisciplinary program should focus not on abstract rankings, but on the concrete ways in which arbitrary power is constrained (or legitimated) within specific socio-political contexts.
Full Text
Introduction
For a long time, the concept of the rule of law was examined within scholarly discourse primarily through the lens of philosophy and general legal theory, remaining on the periphery of sociological inquiry. This was largely because conceptions of the rule of law crystallized well before the social sciences developed a methodological toolkit capable of empirically testing them. While legal-philosophical thought seeks to provide a normative justification of the rule of law and enhance its status in theoretical debates, a sociological perspective concentrates on its actual role and functions in social practice. Moreover, the implementation of the rule of law involves the realization of an ideal construct that, by definition, cannot exist in a completely “pure” form. Accordingly, within the sociology of law, a simple dichotomous question about the presence or absence of the rule of law is methodologically misleading, since empirical research is oriented not toward binary answers but toward assessing the degree and modalities of its realization in specific social contexts.
Where traditional approaches often treat the rule of law as a kind of “checklist,” the sociological approach replaces the question “What is the rule of law?” with “Does law in fact rule?” It turns to the social foundations of legality: how law is manifested in everyday lives, whether people regard it as authoritative, and whether they actually comply with legal norms. In this perspective, the central issue becomes the methodology used to measure this dimension of legality and to capture the complex ways in which law structures social relations.
First Generation: Why Do Rule-of-Law Reforms Fail?
The first generation of empirical research addressed the problem of the “gap” between law “in the books” and law “in action” (Hertogh, 2024:37). The central concern of these “gap studies” was legislative compliance and the effectiveness of legal reforms, particularly in post-conflict and transition societies. The number of such studies initially remained limited, largely because of the fundamental difficulty of operationalizing the concept of the “rule of law,” whose very meaning is contested.
Some of the earliest works in this field (Abel, 1980; Feeley, 1976) used empirical data to compare “legal assumptions” with “legal reality” and rested on “the belief that the gap (between norm and implementation), once identified, can and should be closed” (Abel, 1973). One of the most influential examples is Marc Galanter’s landmark article published while he was at the University of Wisconsin-Madison (Galanter, 1974), which effectively refuted the idea that all people enjoy equal access to the justice system. Subsequent research confirmed this insight, and “access to justice” developed into a distinct field of inquiry (Albiston & Sandefur, 2013; Rhode, 2004; Sandefur, 2021).
However, in most of these studies, the connection to the rule of law remained implied, and their scope tended to be relatively narrow. The empirical focus was often placed on the liberalization of criminal legislation or corporate bankruptcy law (Halliday & Carruthers, 2007), changes in governmental structures (Kim, Jang & Hwang, 2002), or the transformation of public security institutions (Deflem, 2002; Suchman, & Eyre, 1992; Eyre & Suchman, 1996; Hironaka, 2009; Swiss, 2011; Shor, 2017; Jepperson, Wendt & Katzenstein, 1996). Yet the underlying premise of this body of research was the existence of a general model of the rule of law that had already been realized, at least in part, in some countries and served as a template of “expected institutional transformations” in others.
Scholars from Stanford University, Erik Jensen and Thomas Heller, were among the first to place empirical research on the rule of law at the center of a monograph (Jensen & Heller, 2003). Throughout the book, they criticize the strong normative bias of rule-of-law reforms and the paucity of empirical knowledge about the actual effects of these programs. The leading American analyst of democracy and the rule of law, Thomas Carothers, reinforced this point by formulating a similar argument about a pervasive “knowledge problem.” In his view, proponents of initiatives such as Law & Development frequently assert that the rule of law will strengthen the economy and that courts, for example, play a crucial role in this process, yet such “rule-of-law advocates... actually know little about how and why the rule of law emerges or fails to emerge in particular societies...” (Carothers, 2003:18).
The principal conclusion of this first generation of socio-legal research was that ambitious legal reforms often fail because they do not take local context into consideration. After several decades of concentrating on the “gap” between the theory and practice of the rule of law, this line of inquiry stopped producing major new insights. Some sociologists came to view donor assistance as largely ineffective in promoting the rule of law. Moreover, a number of studies found that aid flows are significantly negatively correlated with various rule-of-law reforms, suggesting that external assistance can be counterproductive and even harmful (Rajan & Subramanian, 2007; Knack, 2001; Young & Sheehan, 2014).
Commenting on the futility of further examining legal effectiveness within this framework, Richard Abel, Distinguished Professor of Law at the University of California, Los Angeles, remarked that, even at the stage of formulating a hypothesis “we know in advance that law will be largely ineffective”; the real question, he suggested, is how long can we maintain a posture of naive surprise? (Abel, 1980:827).
Second Generation: The Era of Indices and Quantitative Measurement
The second generation of research sought to capture the rule of law “in numbers” (Jakab & Kirchmair, 2024), relying on quantitative indicators to measure it (Botero & Ponce, 2011). This innovative approach promised to show not only the nature but also the magnitude of the “gap” between law “on paper” and law “in action.” Constructing a rule-of-law index required, first, the adoption of a clear conceptual definition; second, its operationalization; third, the selection of measurable indicators for each component; and finally, the classification of countries along these indicators. Producing indicators of legal-system performance is therefore a complex, multi-stage process typically carried out by networks of actors, including academics, NGOs, international organization and private firms.
In 2002, World Bank General Counsel Ko-Yung Tung argued that neither “human dignity” nor “private sector growth” is possible without the rule of law1. Three years later, the World Bank introduced the concept of the “Rule of Law Index,” defining it as an indicator of a society’s willingness to comply with laws and, accordingly, of the level of trust in the country’s legal system (Wang et al., 2019). This emphasis on the rule of law as a principle for institutional design stimulated the emergence of an entire industry of public and private indices devoted to measuring its quality. Today, among the most widely used are the World Bank’s Worldwide Governance Indicators, the World Justice Project’s Rule of Law Index, the Heritage Foundation’s measures, Freedom House’s Freedom in the World, and the Bertelsmann Transformation Index[2].
The "thin" (formal) conceptions of the rule of law proved to be the most compatible with empirical research on the state of the rule of law, since they set out clearer requirements for the object of inquiry, similar to those proposed by L. Fuller or J. Raz3. Although these lists primarily focus on the qualities a legal system should possess, they rest on assumptions concerning the relationship between the legal system and its addressees (Taekema, 2021). For example, L. Fuller’s story of King Rex’s attempt to regulate people’s behavior is entirely premised on their ability to obey legal norms, and J. Raz argued that law must above all be “capable of guiding its subjects” (Raz, 1979:213).
Nevertheless, there is a very weak connection between the methodologies used to construct indices and the most influential scholarly conceptions of the rule of law. This helps to explain why indicators designed to measure different aspects of the rule of law often correlate poorly both with one another and with contextual variables (Haggard, Macintyre & Tiede, 2008).
For instance, the WJP Global Rule of Law Index represents an attempt to balance formal and substantive understandings of the rule of law so that the index can be applied across diverse social and political systems (Gutiérrez et al., 2020). According to WJP staff, this approach is intended to ensure that the index reflects the variety of legal traditions and cultures and thus avoids Western ethnocentrism (Dougherty, Gryskiewicz & Ponce, 2018).
The index relies on two types of surveys as data sources:
- The General Population Poll consists of nationally representative surveys that include 127 perception-based and 213 experience-based questions to capture the views of ordinary citizens. The questionnaires are translated into local languages and administered by national polling companies every few years to a representative sample of 1,000 respondents in each country.
- The Qualified Respondents’ Questionnaire is conducted annually among local practitioners and experts (such as lawyers and academics) who routinely interact with state institutions. It contains closed-ended questions on perceptions and hypothetical scenarios designed to complement the population poll data and is administered in six languages, with an average of 26 experts participating per country.
Professor René Urueña of the Faculty of Law at the University of Los Andes (Venezuela) argues that the WJP Rule of Law Index can be understood as a technology of global governance. In his view, such indicators are not merely neoliberal tools but entrenched elements of the law-and-development landscape. They help constitute a “community of practitioners” that sets normative criteria for assessing adherence to the rule of law and creates spaces for discussion and policy formulation. Urueña’s analysis presents the WJP Index as a double-edged sword: it reduces complex legal ideals to data, yet also democratizes debate by enabling claims for transnational accountability (Urueña, 2015). Following this logic, the power of the rule-of-law index lies not in its neutrality but in its capacity to reshape how societies conceptualize or contest the rule of law.
By contrast, WJP representatives, reflecting on the difficulties they encountered in designing indicators to assess the rule of law, emphasize that the process begins with a robust underlying theory that defines the conceptual framework for measurement, since this framework determines the choice of methodology. At this stage, they highlight two key challenges: specifying the conceptual depth of the phenomenon to be measured and formulating a definition that adequately captures its multidimensional character4. The “depth” of the definition ultimately reflects a value choice made by the researcher, whereas the level of multidimensionality functions largely as a practical constraint, dictated by the desired degree of detail in measurement.
A key methodological problem for many indices is their heavy reliance on expert opinions, which are often influenced by media and narratives and entrenched stereotypes (for example, this may be reflected in downward adjustments of Russia’s ratings due to the broader political context). As shown in the review by Professor Svend-Erik Skaaning of Aarhus University (Denmark), devoted to quantitative rule-of-law measurement tools, almost all such indices either rely entirely on surveys of legal experts or assign their responses substantially greater weight in the aggregation of data (Skaaning, 2009:454). Professor Mila Versteeg of the University of Virginia and Professor Tom Ginsburg of the University of Chicago have found that, despite seemingly different conceptual approaches, the WGI, Heritage, and WJP indices are extremely strongly highly correlated (r > 0.95)5. Although the WJP Rule of Law Index is the only one of these that also incorporates public-opinion data, the assessments of ordinary citizens “ultimately have little impact on the overall index” (Versteeg & Ginsburg, 2017:120). On this basis, Versteeg and Ginsburg conclude that “the measurement strategy [placing greater emphasis on expert opinions than public opinion], rather than conceptualization, appears to be the dominant factor shaping rule of law indicators” (Versteeg & Ginsburg, 2017:125).
In fact, the overall similarity of rule-of-law indicators is explained less by their conceptual accuracy than by shared methodological constraints: reliance on expert perceptions, use of overlapping sources, and similar aggregation procedures (Versteeg & Ginsburg, 2017). This casts doubt on their validity as instruments for measuring the rule of law in a strict sense. It is therefore unsurprising that the WJP leaders caution against treating indices as ready-made answers rather than as tools for analysis, stressing that their value emerges only when used together with qualitative research. They function as proxies that demand contextual interpretation. For instance, the number of homicides per capita reflects not only levels of criminality but also the quality and accessibility of healthcare6. In other words, the WJP Index provides a standardized framework for cross-national comparison, but its data must be read in light of contextual constraints (Botero & Ponce, 2011).
Another problem concerns endogeneity and data unreliability. Legal development indicators strongly influence how donors, investors, and the media perceive countries, affecting resource allocation in the form of aid and investment as well as the design of political reforms (Davis, 2014). On the one hand, this influence is often attributed to the simplicity, accessibility, and lack of alternatives to such ratings; on the other, the centrality of quantitative scores for decision-making increases incentives to manipulate them. This is a textbook illustration of “Campbell’s Law”: tools created to strengthen the rule of law end up provoking strategic behavior that undermines it (Campbell, 1979). In practice, this results in the simulation of reforms: governments create “pocket” courts or adopt highly targeted legislation that improves formal indicators. Countries with “medium” ratings may thus exhibit a widening gap between formal institutions and actual practices, concealing the absence of genuine progress (Reátegui & Albujar, 2020).
Aggregating information undoubtedly reduces error, but only when the sources are independent – an assumption that is often not met in practice. Survey data are systematically biased: countries with weak institutions tend to overrate themselves, whereas highly developed states tend to underrate. For this reason, Princeton professor Kim Lane Scheppele and her Oxford colleague Kalypso Nicolaïdis criticize contemporary rule-of-law assessment through checklists and indicators, arguing that such tools frequently overlook a crucial dimension: the interaction among elements of the legal system, which can turn ostensibly legitimate mechanisms into instruments of authoritarian governance, as illustrated by many so-called “liberal democracies” (Scheppele, 2013).
A number of scholars have used data from the World Justice Project (WJP) to develop their own methodological approaches to studying the rule of law. For instance, P. Gowder relies on WJP data to propose an alternative quantitative methodology in this field. He criticizes existing multidimensional metrics, describing them as a “strange composite idea” that bundles together factors loosely associated with the rule of law (Gowder, 2016:179). In contrast, he views the rule of law as a unidimensional concept and uses WJP data to test the “workability” of his own scale.
Associate Professor of Constitutional Law at Charles University, Filip Horák, and Research Fellow in the Sociology of Law at Charles University, David Lacko, argue that the WJP Index measures primarily de jure parameters (the existence of laws) while neglecting de facto implementation (Horák, & Lacko, 2023). Unlike P. Gowder, F. Horák and D. Lacko employ elements of the WJP project to construct a more comprehensive framework. Responding to S. Taekema’s call to combine different methodological approaches (Taekema, 2021:), they integrate three distinct “dimensions” of the rule of law: conceptualizations in rule-of-law theory (Bedner, 2018), quantitative assessment tools from the WJP project, and an “empirically measurable construct of legal consciousness” (Horák & Lacko, 2023:192). The result is a detailed list of survey items that can be used for the empirical measurement of the actual level of the rule of law among various population groups (Horák & Lacko, 2023:213).
An empirical study of the economic and institutional determinants of the rule of law in a global sample of 103 countries (1970–2015), using machine learning methods, was conducted by Polish scholars (Lewkowicz, Byler & Litwiniuk, 2021:88). However, this study also does not rely on primary data7, and thus, despite employing an innovative method for this field, its findings are broadly comparable to those of other similar works.
Since most indicators function as a kind of “snapshot” at the moment of measurement and thus ignore the stability of legal institutions over time, a group of scholars at the University of Illinois developed a new conceptual framework and corresponding indicators for studying the rule of law dynamically. In their view, existing measures conflate the core of the rule of law (instrumental/procedural aspects), the means of its implementation (judicial resources), presumed outcomes (law-abidingness, low violence), and normative values (human rights). They therefore propose distinguishing between two key concepts: “legal order,” capturing the instrumental, procedural, and formal aspects of the rule of law, and “legal regime,” referring to the normative values and goals embedded within that order (for example, liberal human rights, socialist equality, Islamic principles). In other words, legal order concerns how law works, whereas legal regime concerns what values the system promotes. This distinction allows for a clearer delineation of the object of measurement and helps to avoid conceptual confusion. The indicators derived from this framework have already been used to analyze links between legal institutions, economic growth, and civil unrest following climate-related disasters, demonstrating their utility (Nardulli, Peyton & Bajjalieh, 2013).
Professor Sandra Joireman of the University of Richmond criticizes the major indices on the ground that they fail to capture what she sees as the most important factor for stable institutional development and economic growth: the predictability of law (Joireman, 2004:64). In this regard, Professor Ryan Carlin of Georgia State University proposes applying hierarchical agglomerative cluster analysis instead of constructing a single linear index. This method allows the rule of law to be conceived as a set of distinct configurations (profiles). According to R. Carlin, countries may exhibit strengths on one dimension (for example, stability) while remaining extremely weak on another (such as human rights or judicial independence) (Carlin, 2010). This approach is intended to provide a more nuanced analytical tool, permitting the study not only of the “level” of the rule of law but also its type8. His findings also indicate that there is no simple linear relationship between democracy, wealth, and the rule of law9.
Ultimately, criticism of proxy indicators laden with measurement error, the excessive focus on business-related legal norms, and mounting methodological problems concerns cooled the initial enthusiasm for this type of empirical research. The uncritical use of indices risks substituting genuine change with its simulation (Reátegui & Albujar, 2020), which in turn has prompted the search for more sophisticated and manipulation-resistant assessment methods that concentrate not on “improving numbers” but on capturing deep institutional transformations.
At the same time, calls to abandon global rankings in favor of sectoral-specific measurements appear well-founded (Ginsburg, 2018). T. Ginsburg, for instance, suggests focusing on particular institutions and measuring their concrete manifestations (judicial independence, contract enforcement, and so forth) instead of an abstract “rule of law,” as well as tracing sequential changes within a single country (for example, reforms of the Georgian judiciary between 2004-2012), since this approach is more reliable than cross-national league tables. He also argues for prioritizing objective metrics, such as the number of lawsuits against the state or case processing times.
Third Generation: Legal Consciousness and Ethnography as a Way Out of the Impasse
One can hardly dispute that, given differences in time and place, understandings of the phenomenon of the “rule of law” is culturally conditioned (Hadfield & Weingast, 2014). Professor at the University of California, Los Angeles, and an active critic of “index-mania,” Randall Peerenboom argues that ignoring the cultural context turns the rule of law into a tool of neocolonialism, hindering dialogue between legal traditions – the real path to effective reforms (Peerenboom, 2003). Marc Hertogh echoes this view, identifying eurocentrism as the central problem of sociological research in this field (Hertogh, 2016). Indeed, indices generally reflect Western standards, thereby marginalizing models of the rule of law in Asian countries or within Islamic law.
At the same time, it has become apparent that indices reduce complex phenomena to simplified metrics that depend on subjective perceptions, while empirical measurement tools should, by contrast, focus on the general population (for example, the addressees) rather than on expert opinion when assessing the level of the rule of law (Hertogh, 2016). Such an approach could also help to address the difficulties of assessing “ambiguous” legal norms and the hidden nature of many practices, as the problem in sociological research arises less from gross violations (which are easy to measure) than from systematic deviations, abuses, and unequal application of the law, which are difficult to capture with standard index metrics.
Therefore, in line with the growing interest in public opinion and ethnography, a third generation of empirical rule-of-law research has emerged. These studies analyze the rule of law “from below” (Buyse et al., 2021), focusing on the “subjective dimensions of law” (Holzer, 2013:873) and the gap between the goals and assumptions of the legal system and the reality experienced and desired by citizens (Polzer, 2007:22).
Moving away from quantitative research, a growing number of third-generation studies can be described as new forms of “rule-of-law ethnography” (Cheesman, 2018), based on qualitative methods such as observation and semi-structured interviews. To facilitate the “measurement” of the rule of law, this third generation has focused on an interdisciplinary determinant grounded in empirical methodology. The most balanced approach is to begin with society, with how ordinary people understand law in their everyday lives. It is therefore logical that this interdisciplinary determinant should be captured by the term “legal consciousness.”
Unlike earlier generations, this type of research is not fixated on a predetermined definition of the rule of law and does not rely on a fixed set of indicators. Instead, it develops a kind of open model of the rule of law structured around the question: what do people themselves consider the most desirable relationship between the individual, society, and the state? What are their normative ideas about justice, power, and accountability? In other words, such research does not treat competing understandings of the rule of law as “inconveniences to be eliminated” (Cheesman, 2018:170). On the contrary, these different perspectives are seen as opportunities to better understand how ideas and activities likely falling under the category of the rule of law are shaped in struggles over meaning (Cheesman, 2018:170).
To some extent, “transitional” type of research is exemplified by the work of University of Houston Professor James Gibson and Ohio University Professor Gregory Caldeira (Gibson & Caldeira, 1996), devoted to European legal cultures. Still characteristic of the second generation, their study is based on mass surveys in all European Union member states, yet – unlike earlier work, and in a way more typical of third-generation research – it explicitly focuses on people’s legal consciousness. Their analysis examines popular views on various aspects of law, including attitudes toward the rule of law, perceptions of the neutrality of law, and the relative value placed on individual freedom.
In qualitative research, systematic errors in cross-cultural comparisons typically include problems arising from how respondents from different cultures interpret the same question, as well as the differing cultural salience of the construct being measured, since that construct may have different meanings or manifestations across cultures (Lacko et al., 2022). Therefore, the key conclusion reached by third-generation research is that people’s perceptions of law often diverge significantly from established norms and compete with other value systems. However, the results of studies on social values are almost always ambiguous, and claims about unconditional adherence to such values should be interpreted with considerable skepticism.
Professor at Arizona State University, Stefanie Lindquist, in her critique of Ronald A. Cass (Cass, 2001:214), argues that contemporary rule-of-law scholarship relies too heavily on the writings of judges and theorists themselves and not enough on systematic behavioral data (for example, studies of judicial motivations or the influence of institutional context). In her view, Cass’s discussion of the rule of law overlooks research on why judges follow the law, such as Larry Baum’s work on judicial goals and Melinda Gann Hall’s on the role of judicial elections (Lindquist, 2003).
According to Marc Hertogh, there is a need to move from assessing conformity to predetermined standards toward studying the “social definitions” of law and to focus on citizens as users of law rather than on officials or institutions. In methodological terms, he advocates prioritizing qualitative ethnographic research over indices and rankings (Hertogh, 2016). He is echoed by Paul Burgess, who contends that empirical, contextual, “bottom-up” approaches are essential for studying the rule of law and calls for “going into the field,” observing and talking to people in order to identify the actual rules and practices in place.
Conclusion
The sociology of law currently faces several pressing challenges. The first concerns the problem of neutrality and objectivity in rule-of-law research; in this respect, it may be necessary to combine personal engagement with a readiness to have one’s views overturned by empirical evidence. A second challenge involves defining the rule of law in sociological inquiry, since it is essential to examine not only how people comply with the law but also how they themselves define it. A third challenge lies in the relative lack of theoretical reflection despite the large volume of accumulated empirical data.
It is evident that, to develop a more effective research model – one in which the key indicator in law’s capacity to constrain arbitrariness in specific socio-political contexts – it is necessary to integrate theoretical developments, quantitative research on the effectiveness of legal institutions (such as statistics on judicial decisions, corruption, and access to justice), and qualitative research on citizens’ perceptions of law, including trust in institutions and willingness to obey legal norms (for example, in-depth interviews exploring legal consciousness). At a final stage, the results obtained should be cross-validated across different sources.
Ideally, the use of rule-of-law indices as objective analytic tool applicable across countries, regardless of political-legal cultures or ideology, would mark a significant step toward understanding global legal trends. Achieving this, however, requires an innovative bridge between theory and empirical research. The study of legal indicators should therefore be treated as a single field of inquiry that calls for an interdisciplinary approach. The global crisis of the rule of law cannot be resolved by political and legal measures alone; grasping this phenomenon requires the development of a coherent social science of the rule of law that integrates all three approaches – comparison, measurement, and interpretation.
1 Bank analysts drew heavily on 1990s research suggesting that a country's “legal origin” (especially links to English common law) shapes economic growth and foreign investment, which in practice led the Bank to prioritize changes to economic regulation, reduction of business regulation, and expansion of private markets. By the mid-1990s, the Bank was already conditioning lending on reforms in judicial selection (merit-based appointment, judicial ethics codes, and financial autonomy of courts).
2 At the same time, the design of indicators has been influenced by data availability, the resources of their creators, and their willingness either to avoid or to provoke controversy, since the motivations for constructing such indices are not purely economic but also deeply political.
3 Substantive conceptions, which address both the content of legal norms and the socio-cultural context of the legal order, introduce a broader range of requirements intended to distinguish “bad” from “good” laws. Unsurprisingly, such an approach pushes research further into the study of sociological preconditions – that is toward the concerns characteristic of the third generation of studies).
4 Camilo Gutiérrez worked at the World Justice Project (WJP) from 2016 to early 2019, overseeing data analysis for the Rule of Law Index, after holding academic positions at the University of the Andes (Bogotá, Colombia). Joel Martinez, now Director of Partner Engagement at the WJP, previously enforced anti-discrimination laws at the Massachusetts Commission Against Discrimination, focusing on low-income communities. Alejandro Ponce, Chief Research Officer at the WJP and one of the architects of the Rule of Law Index, holds a PhD in Economics from Stanford and has worked as a researcher at Yale University, an economist at the World Bank, and the Mexican Securities and Banking Commission. Leslie Solís joined the WJP in 2016 after five years working as a criminal justice analyst at the think tank México Evalúa (Mexico City).
5 Freedom House differs somewhat in its correlation levels from the other indices, though not dramatically (r = 0.79 – 0.83).
6 At the time of publication, Juan Carlos Botero was Executive Director of the World Justice Project (WJP); Robert Nelson was Professor of Sociology and Law at Northwestern University (USA) and Director of the American Bar Foundation; and Christine Pratt was a research methods specialist at the WJP (Botero, Nelson & Pratt, 2011).
7 The authors draw on the publicly available “Varieties of Democracy” database, which contains variables characterizing the quality of democracy globally, as well as the “World Uncertainty Index,” an indicator derived from text mining of Economist Intelligence Unit country reports.
8 Ryan Carlin's analysis identifies five stable clusters, or “types” of rule of law: Full Rule of Law (high scores across all dimensions, e.g., most OECD states and some post-socialist countries such as Czech Republic and Poland; Incomplete Rule of Law (strong institutional and social dimensions but serious problems with equality in positive rights (discrimination), as in South Korea, Brazil, South Africa, Kuwait; Peaceful Unrule of Law (relative social peace coexisting with very weak institutions (controlled judiciary) and widespread rights violations, as in China, 1990s Mexico, and many African states. Unstable Lawlessness (very weak institutions and rights violations combined with minor armed conflicts, observed episodically in some CIS (Georgia, Kyrgyzstan) and African countries (Chad) in certain years; and Violent Unrule of Law (open armed conflict in which all other aspects of the rule of law collapse, as in Algeria, Peru, Sri Lanka, Rwanda in the 1990s.
9 For example, “incomplete rule of law” can be found in both wealthy authoritarian regimes and poor democracies.
About the authors
Ruslan M. Allalyev
RUDN University; Russian State University of Oil and Gas (National Research University) named after I.M. Gubkin
Author for correspondence.
Email: allalyev-rm@rudn.ru
ORCID iD: 0000-0002-0334-6862
SPIN-code: 4109-5931
Candidate of Legal Sciences, Associate Professor of the Department of Theory of Law and State, Law Institute, RUDN University; Associate Professor of the Department of Public and International Legal Support for National Security, Faculty of Fuel and Energy Complex Integrated Security, Russian State University of Oil and Gas (National Research University) named after I.M. Gubkin
6 Miklukho-Maklaya str., Moscow, 117198, Russian Federation; 65, Building 1, Leninsky Prospect, Moscow, 119991, Russian FederationReferences
- Abel, R. (1973) Law books and books about law. Stanford Law Review. 26, 188.
- Abel, R. (1980) Redirecting social studies of law. Law & Society Review. 14 (3), 805-829.
- Albiston, C.R. & Sandefur, R.L. (2013) Expanding the empirical study of access to justice. Wisconsin Law Review. (1), 101-120.
- Bedner, A. (2018) The promise of a thick view. In: May, C. & Winchester, A. (eds.) Handbook on the Rule of Law. Cheltenham, Edward Elgar Publishing, pp. 34-47. https://doi.org/10.4337/9781786432445.00008
- Botero, J.C., Nelson, R.L., & Pratt, C.S. (2011) Indices and Indicators of Justice, Governance, and the Rule of Law: An Overview. Hague Journal on the Rule of Law. 3 (2), 153-169. https://doi.org/10.1017/S1876404511200010 EDN: ZRYLPA.
- Botero, J. & Ponce, A. (2011) Measuring the rule of law. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.1966257
- Buyse, A., Fortin, K., McGonigle Leyh, B. & Fraser, J. (2021) The rule of law from below - a concept under development. Utrecht Law Review. 17 (2), 1-7. https://doi.org/10.36633/ulr.771 EDN: KVGBHU.
- Campbell, D.T. (1979) Assessing the impact of planned social change. Evaluation and Program Planning. 2 (1), 67-90.
- Carlin, R.E. (2010) Rule of law profiles in contemporary polities. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.1669493
- Carothers, T. (2003) The problem of knowledge. In: Carothers, T. (ed.) Promoting the Rule of Law Abroad: In Search of Knowledge. Washington, DC, Carnegie Endowment for International Peace Publ., pp. 15-28.
- Cass, R.A. (2001) The Rule of Law in America. Baltimore, Johns Hopkins University Press.
- Cheesman, N. (2018) Rule-of-law ethnography. Annual Review of Law and Social Science. 14, 167-184. https://doi.org/10.1146/annurev-lawsocsci-101317-030900
- Davis, K.E. (2014) Legal indicators: The power of quantitative measures of law. Annual Review of Law and Social Science. 10, 37-52. https://doi.org/10.1146/annurev-lawsocsci-110413-030857
- Deflem, M. (2002) Policing World Society: Historical Foundations of International Police Cooperation. Oxford, Oxford University Press.
- Dougherty, A., Gryskiewicz, A. & Ponce, A. (2018) Measuring the rule of law: The World Justice Project’s Rule of Law Index. In: Malito, D., Umbach, G. & Bhuta, N. (eds.) The Palgrave Handbook of Indicators in Global Governance. Cham, Palgrave Macmillan, pp. 255-276. https://doi.org/10.1007/978-3-319-62707-6_11
- Eyre, D.P. & Suchman, M.C. (1996) Status, norms, and the proliferation of conventional weapons: An institutional theory approach. In: Katzenstein, P.J. (ed.) The Culture of National Security: Norms and Identity in World Politics. New York, Columbia University Press, pp. 79-113.
- Feeley, M. (1976) The concept of laws in social science: A critique and notes on an expanded view. Law & Society Review. 10, 497-523.
- Galanter, M. (1974) Why the haves come out ahead: Speculations on the limits of legal change. Law & Society Review. 9 (1), 95-160.
- Gibson, J.L. & Caldeira, G.A. (1996) The legal cultures of Europe. Law & Society Review. 30 (1), 55-86.
- Ginsburg, T. (2018) Difficulties with measuring the rule of law. In: May, C. & Winchester, A. (eds.) Handbook on the Rule of Law. Cheltenham, Edward Elgar Publishing, pp. 48-56. https://doi.org/10.4337/9781786432445.00009
- Gowder, P. (2016) The Rule of Law in the Real World. Cambridge, Cambridge University Press.
- Gutiérrez, C., Martinez, J., Ponce, A. & Solís, L. (2020) Measuring the rule of law in Mexico. In: Le Clercq, J. & Abreu Sacramento, J. (eds.) Rebuilding the State Institutions. Cham, Springer, pp. 57-77. https://doi.org/10.1007/978-3-030-31314-2_4
- Hadfield, G.K. & Weingast, B.R. (2014) Microfoundations of the rule of law. Annual Review of Political Science. 17, 21-42. https://doi.org/10.1146/annurev-polisci-100711-135226
- Haggard, S., Macintyre, A.J. & Tiede, L.B. (2008) The rule of law and economic development. Annual Review of Political Science. 11, 205-234. https://doi.org/10.1146/annurev.polisci.10.081205.100244
- Halliday, T.C. & Carruthers, B.G. (2007) The recursivity of law: Global norm making and national lawmaking in the globalization of corporate insolvency regimes. American Journal of Sociology. 112 (4), 1135-1202.
- Hertogh, M. (2016) Your rule of law is not mine: Rethinking empirical approaches to EU rule of law promotion. Asia Europe Journal. 14, 43-59. https://doi.org/10.1007/s10308-015-0434-x EDN: TCQDXK.
- Hertogh, M. (2024) Empirical approaches to the rule of law: Contours and challenges of a social science that does not quite yet exist. Annual Review of Law and Social Science. 20, 35-51. https://doi.org/10.1146/annurev-lawsocsci-041822-044308 EDN: JUJECX.
- Hironaka, A. (2009) Neverending Wars: The International Community, Weak States, and the Perpetuation of Civil War. Cambridge, MA, Harvard University Press.
- Holzer, E. (2013) What happens to law in a refugee camp? Law & Society Review. 47 (4), 837-866. https://doi.org/10.1111/lasr.12041
- Horák, F. & Lacko, D. (2023) Triangulation of theoretical and empirical conceptualizations related to the rule of law. Hague Journal on the Rule of Law. 15, 191-216. https://doi.org/10.1007/s40803-022-00181-x EDN: KIMOIP.
- Jakab, A. & Kirchmair, L. (2024) The rule of law in numbers: How to use indices to measure adherence by the Member States and the EU itself. In: Wouters, J. et al. (eds.) Upholding the Rule of Law in the EU. Oxford, Oxford University Press. (Forthcoming).
- Jensen, E.G. & Heller, T.C. (eds.) (2003) Beyond Common Knowledge: Empirical Approaches to the Rule of Law. Stanford, CA, Stanford University Press.
- Jepperson, R.L., Wendt, A. & Katzenstein, P.J. (1996) Norms, identity, and culture in national security. In: Katzenstein, P.J. (ed.) The Culture of National Security: Norms and Identity in World Politics. New York, Columbia University Press, pp. 33-75.
- Joireman, S.F. (2004) Colonization and the rule of law: Comparing the effectiveness of common law and civil law countries. Constitutional Political Economy. 15 (4), 315-338. https://doi.org/10.1007/s10602-004-7766-7 EDN: VDKMXA.
- Kim, Y.S., Jang, Y.S. & Hwang, H. (2002) Structural expansion and the cost of global isomorphism: A cross-national study of ministerial structure, 1950-1990. International Sociology. 17 (4), 481-503. EDN: JMVVWT.
- Knack, S. (2001) Aid dependence and the quality of governance: Cross-country empirical tests. Southern Economic Journal. 68 (2), 310-329. https://doi.org/10.2307/1061596 EDN: DSNQSJ.
- Lacko, D. et al. (2022) The necessity of testing measurement invariance in cross-cultural research: Potential bias in cross-cultural comparisons with individualism-collectivism self-report scales. Cross-Cultural Research. 56 (2-3), 228-267. https://doi.org/10.1177/10693971211068971 EDN: NJLKEP.
- Lewkowicz, J., Byler, J. & Litwiniuk, P. (2021) Economic and institutional determinants of the rule of law: A new empirical exploration. Studia Iuridica. 88, 237-254. https://doi.org/10.31338/2544-3135.si.2021-88.12 EDN: KMRGGU.
- Lindquist, S.A. (2003) Review of The Rule of Law in America by Ronald A. Cass. The Journal of Politics. 65 (2), 615-617. https://doi.org/10.1111/1468-2508.00652_16
- Méndez Reátegui, R. & Sumar Albujar, O. (2020) Rule of law versus soft rule of law. Revista de Derecho Político. 1 (109), 373-400. https://doi.org/10.5944/rdp.109.2020.29065 EDN: FYOWYG.
- Nardulli, P.F., Peyton, B. & Bajjalieh, J.W. (2013) Conceptualizing and measuring rule of law constructs, 1850-2010. Journal of Law and Courts. 1 (1), 139-192. https://doi.org/10.1086/668543
- Peerenboom, R. (2003) Varieties of rule of law: an introduction and provisional conclusion. In: Peerenboom, R. (ed.) Asian Discourses of Rule of Law. London, Routledge, pp. 1-53. https://doi.org/10.2139/ssrn.445821
- Polzer, T. (2007) Adapting to changing legal frameworks: Mozambican refugees in South Africa. International Journal of Refugee Law. 19 (1), 22-50. https://doi.org/10.1093/ijrl/eem001 EDN: IYBCJZ.
- Rajan, R. & Subramanian, A. (2007) Does aid affect governance? American Economic Review. 97 (2), 322-327.
- Raz, J. (1979) The rule of law and its virtue. The Authority of Law. Oxford, Clarendon Press.
- Rhode, D.L. (2004) Access to Justice. Oxford, Oxford University Press.
- Sandefur, R.L. (2021) Access to justice. In: Talesh, S., Mertz, E. & Klug, H. (eds.) Research Handbook on Modern Legal Realism. Cheltenham, Edward Elgar Publ., pp. 324-335.
- Scheppele, K.L. (2013) The rule of law and the Frankenstate: Why governance checklists do not work. Governance. 26 (4), 559-562. https://doi.org/10.1111/gove.12049
- Shor, E. (2017) The spatial diffusion of counterterrorist legislation, 1970-2011. Social Problems. 64 (1), 106-132. https://doi.org/10.1093/socpro/spw028
- Skaaning, S.-E. (2010) Measuring the rule of law. Political Research Quarterly. 63 (2), 449-460. https://doi.org/10.1177/1065912909346745
- Suchman, M.C. & Eyre, D.P. (1992) Military procurement as rational myth: Notes on the social construction of weapons proliferation. Sociological Forum. 7 (1), 137-161. EDN: DIMVOT.
- Swiss, L. (2011) Security sector reform and development assistance: Explaining the diffusion of policy priorities among donor agencies. Qualitative Sociology. 34 (2), 371-393. https://doi.org/10.1007/s11133-011-9194-0
- Taekema, S. (2021) Methodologies of rule of law research: Why legal philosophy needs empirical and doctrinal scholarship. Law and Philosophy. 40, 33-66. https://doi.org/10.1007/s10982-020-09388-1 EDN: RHMHVS.
- Urueña, R. (2015) Indicators and the law: A case study of the Rule of Law Index. In: Merry, S., Davis, K. & Kingsbury, B. (eds.) The Quiet Power of Indicators. Cambridge, Cambridge University Press, pp. 75-102. https://doi.org/10.1007/9781139871532.003
- Versteeg, M. & Ginsburg, T. (2017) Measuring the rule of law: A comparison of indicators. Law & Social Inquiry. 42 (1), 100-137. https://doi.org/10.1111/lsi.12175
- Wang, H., Wang, Zh., Chen, J., Chen, J., Guan, N. & Li, G. (2019) Development of rule of law index for state land and resources in China. Land Use Policy. 81, 276-290. https://doi.org/10.1016/j.landusepol.2018.10.053
- Young, A.T. & Sheehan, K.M. (2014) Foreign aid, institutional quality, and growth. European Journal of Political Economy. 36, 195-208. https://doi.org/10.1016/j.ejpoleco.2014.08.003
Supplementary files










