The Supreme Court in July set aside orders passed by the National Company Law Tribunal and the National Company Law Appellate Tribunal after finding that both relied on non-existent, AI-generated judicial precedents. The fabricated judgments were caught. But the more unsettling question is: how much isn’t?
This comes at a time when AI is already moving into the daily functioning of India’s judiciary. It is being used for vernacular translation of judgments, document scrutiny at the filing stage, and live transcription of proceedings. The High Courts of Kerala and Gujarat have introduced AI policies governing its use in the district judiciary, while the Supreme Court has circulated draft regulations for comment.
The pressure to adopt these technologies is understandable. India’s judiciary faces enormous backlogs, and AI offers the possibility of reducing administrative burdens and improving efficiency. But that argument can conflate two different claims: that the status quo is unacceptable, and that AI will fix it without introducing harms of its own. The first is true. The second is unproven.
AI is also not a silver bullet for problems that are fundamentally systemic. Consider AI-driven translation. Making judgments available in regional languages can expand access for litigants who do not read English. That is genuinely valuable. But the Supreme Court has itself criticised judges for impenetrable legal prose. Automatically translating jargon-heavy judgments will not, by itself, make judicial decisions easier to understand.
Technology often moves faster than regulation. In the justice system, that gap matters because errors are not merely technical failures. They can affect people’s rights and the outcomes of cases.
Also Read: India’s district judiciary isn’t filling SC-ST-OBC seats
The cost of getting it wrong
Hallucinations, where AI systems generate false information, are the most visible risk. But they are only part of the problem. Systems trained on biased data can reproduce historical inequities, including those linked to caste, class and gender. Errors in translation or transcription can distort the judicial record.
The gap between how a system performs in controlled conditions and how it works in a courtroom can widen these risks. An inaccurate output in a real case can contribute to a rights violation or an unfair outcome.
Not all technologies carry the same level of risk either. A decision tree used for case scheduling is fundamentally different from a large language model summarising witness testimony. The former performs a narrow task and has relatively predictable ways of failing. The latter is probabilistic and can generate errors that are harder to anticipate and detect. Courts cannot govern both in the same way.
Governance must come before deployment
The case for caution should not be mistaken for a case against AI in courts. The issue is one of accountability in the use of AI.
This balance is at the heart of the recent report, AI For Justice: Ethical, Fair and Robust Adoption in India’s Courts, written by DAKSH and the Digital Futures Lab with the support of UNDP India. The report offers practical tools to help courts assess their readiness for AI, identify risks, evaluate vendors of technical tools and services and monitor technologies once they are deployed.
AI is not a shortcut around process reform. If courts layer new technology over archaic processes designed for manual tasks, they risk simply automating existing inefficiencies.
Before deploying an AI system, courts should therefore be asking some basic questions. Who monitors it for accuracy? What level of error is tolerable, and for which tasks? What happens when it fails? Who is liable?
These are not simply technical questions. They are questions of governance, and they need answers before deployment, not after.
Courts need the capacity to govern AI
Most courts today do not have the institutional infrastructure needed to answer these questions. Building that capacity requires investment in both people and resources.
Courts need specialists who understand AI performance assessment, data governance and legal compliance, and who can translate technical findings into decisions that judges and court administrators can act on. The UNDP report therefore advocates dedicated technical and data cadres within High Courts and district courts.
Without such expertise inside the institution, courts risk becoming overly dependent on AI vendors whose interests and incentives may not always align with those of the judiciary.
Procurement also needs greater scrutiny. AI adoption is often driven by individual judges championing ad hoc pilots, rather than structured assessments of what courts actually need. Competitive bidding, disclosure of outcomes and continuous audits can help ensure that systems are chosen and evaluated against clear standards.
There is also a less visible challenge: the “shadow use” of unsanctioned AI tools. A global UNESCO survey found that 44 per cent of judicial operators had used AI for work-related tasks. Judges and court staff using such tools without formal guidance may not fully understand the risks around data privacy, confidentiality and hallucinations.
Clear guidelines and formal training are therefore essential. Courts need to know not only which technologies they are adopting institutionally, but also how AI is being used informally within the justice system.
Also Read: Bar exam to bench, how are Indian women lawyers doing?
Justice requires a higher standard
India’s judiciary is under enormous pressure. The temptation to reach for technology as relief is understandable. AI may yet prove transformative for the delivery of justice.
But courts are not logistics operations. They are institutions where a single error can mean wrongful imprisonment, loss of property, or a survivor being denied justice. The standard cannot simply be “good enough on average.” Courts have to be accountable every time.
Transformation without accountability is another way of failing people, only faster and at scale. Courts should therefore resist a race to adopt AI for its own sake. The promise of justice ultimately rests on human reasoning and notions of fairness. No algorithm can bear that burden.
Angela Lusigi is the Resident Representative of UNDP India, Surya Prakash B S is Fellow and Programme Director at DAKSH, and Urvashi Aneja is Founder and Director at Digital Futures Lab. Views are personal.
(Edited by Asavari Singh)