Structural inequalities across groups defined by gender, religion, and ethnicity exist in almost all societies. Governments often try to remedy these inequalities through policies, such as anti-discrimination statutes or affirmative action, which must then be enforced by the legal system. A challenging problem is that the legal system itself may have unequal representation. It remains an open question whether legal systems in developing countries are effective at pushing back against structural inequality or whether they serve to entrench it.
This paper examines bias in India’s lower courts, asking whether judges deliver more favorable treatment to defendants who match their identities. Judicial bias along gender, religious, or ethnic lines appears to be nearly universal in richer countries, having been identified in a wide range of settings around the world. However, it has not been widely studied in the courts of lower-income countries. In-group bias of this form has been identified in other contexts in India, such as among loan officers and school-teachers. The judicial setting is of particular interest, given the premise that individuals who are discriminated against in informal settings should receive equal treatment under the law.
We focus on the dimensions of gender and religion in India’s lower courts, where unequal representation is a recognised issue. Women represent 48% of the Indian population but only 28% of district court judges. Similarly, India’s 200 million Muslims represent 14% of the population but only 7% of lower court judges. There is growing evidence that India’s Muslims and women do not enjoy equal access to economic or other opportunities. We examine whether unequal representation in the courts has a direct effect on the judicial outcomes of Muslims and women, in the form of judges delivering better outcomes to criminal defendants who match their gender or religion.
Our analysis draws upon a new dataset of 80 million court records covering 2010–2018 from https://ecourts.gov.in/, an online platform documenting the complete set of cases heard in India’s district courts. These cases cover the universe of India’s 7,000+ district and subordinate trial courts, staffed by over 80,000 judges. We are releasing anonymized data on these cases, opening the door to many new analyses of the judicial process in the world’s largest democracy and largest common-law legal system.
We enrich the dataset by classifying judges and defendants to gender and religious (Muslim and non-Muslim) identity groups based on their names. An automated process uses a deep neural network applied to the sequence of characters in names. The distinctive nature of female and Muslim names allows us to classify individuals with over 97% out-of-sample accuracy on both dimensions.
The main research question is whether judges tend to treat defendants differently when they share the same gender or religion. We focus on the subset of cases filed under India’s criminal codes (N = 5.5 million), where acquittal and conviction rates, as well as judicial delay, are readily interpretable as positive or negative outcomes. We implement two different identification strategies to generate causal estimates of how judge identity affects a defendant’s outcome.
First, we exploit the arbitrary rules by which cases are assigned to judges, generating as-good-as-random variation in judge identity. Our preferred specification includes court, charge, and month-year fixed effects. Effectively, we compare the outcomes of two defendants with the same identity classification, charged under the same criminal section, in the same court and in the same month, but who are assigned to judges with different identities.
Second, we exploit judicial turnover events that change the gender and religion balance of judges serving in a district court, exogenously changing the probability that a defendant matches identity with the judge overseeing their case. We use a regression discontinuity specification which measures the difference in judicial outcomes for defendants whose cases are heard immediately before and immediately after a transition that makes the bench more or less similar along identity dimensions.
In both of these specifications, we find a robust null estimate of in-group bias among Indian judges. Judges of different genders do not treat defendants differently according to their gender, nor do judges display favoritism on the basis of religion. This is true both in terms of outcomes (i.e. acquittals and convictions) and in terms of process (i.e. speed of decision). In a subset of specifications, we find a very small in-group gender bias, which is marginally positive and not robust. However, the size of this effect, even in the marginally significant specifications, is an order of magnitude smaller than nearly all prior estimates of in-group bias based on similar identification strategies in the literature. The upper end of our 95% confidence interval rejects a 0.7 percentage point effect size in the worst case; studies using the same identification strategies in other contexts have routinely found bias effects ranging from 5 to 20 percentage points.
Our estimates do not rule out bias in the Indian legal system entirely; we observe only a subset of the legal process and we measure only in-group bias by gender and religion. For example, it is possible that both Muslim and non-Muslim judges discriminate against Muslims (as found for Black defendants in Arnold et al (2017)). It is also possible that arrests and/or charges disproportionately target Muslims, or that judges exhibit bias based on defendant caste or income. However, the bias that we study has been widely reported in other studies with large effect sizes, and the public discussion of discrimination against Muslims and women in India in many ways parallels discussion of marginalized groups in other countries.
The most straightforward interpretation of these findings is that, unlike judges analyzed in the other papers, India’s district court judges do not exhibit in-group bias along the pertinent identity margins. Our research is consistent with judges taking their role seriously and working hard to provide justice on fair terms to all litigants, or with judicial institutions that constrain discretion and protect defendants from biased decision-making. It is also possible that the social distance between (normally) upper class judges and (normally) lower class criminal litigants may mitigate a sense of shared identity between judges and litigants. Yet it is also consistent with corruption that is blind to religious and gender identity. Rich as they are, our data do not allow us to differentiate between these very different mechanisms; exploring these possibilities is an important area for future work.
It is worth emphasising again that we have not ruled out bias in the Indian criminal justice system as a whole. We have focused on two kinds of bias, which have been widely documented in other countries, and we have focused on the singular contributions of judges to criminal-justice outcomes. The legal system could still be biased against Muslims and women overall, through geographic distribution of policing, discrimination in investigations, police/prosecutor decisions to file cases, the severity of charges applied, the severity of penalties imposed, the appeals process, and others. It is also possible that bias takes a more subtle form, such as discrimination conditional on the interaction between defendant, victim, and type of crime. More research, and in particular more data, are needed to study the entire justice process in India and other developing countries.
Elliott Ash is an assistant professor at ETH Zurich’s Center for Law & Economics, Switzerland.
Sam Asher is an assistant professor in the Department of International Economics at Johns Hopkins SAIS.
Aditi Bhowmick is a research associate at Development Data Lab, with a Masters in public administration from Princeton University.
Daniel Chen is a senior researcher at CNRS, France.
Tanaya Devi is a Ph.D. Candidate in Economics at Harvard University.
Christoph Goessmann is a PhD candidate in law, economics, and data science at ETH Zurich, Switzerland.
Paul Novosad is an associate professor of economics at Dartmouth College.
Bilal Siddiqi is director of research at UC Berkeley‘s Center for Effective Global Action.
This is an edited excerpt from the authors’ paper ‘Measuring Gender and Religious Bias in the Indian Judiciary’ and it has been published here with permission. Read the full paper here.