JSA Prism | Dispute Resolution

Please click here to download the Prism as PDF.

Supreme Court of India holds that a judicial decision based on fake or hallucinated material generated by Artificial Intelligence is unsustainable in law and must be set aside

The Supreme Court of India (“Supreme Court”), in Pooja Ramesh Singh vs. Jammu and Kashmir Bank Limited and Anr.[1] has inter alia held that a decision based on fake or hallucinated material generated by Artificial Intelligence (“AI”) is not a decision in the eyes of law and must be set aside for violating the sanctity of adjudication and subverting the rule of law.

Brief facts

Jammu and Kashmir Bank Limited (“Respondent No. 1”) had granted certain credit facilities to Pan India Utilities Distribution Company (“Borrower”). To secure these credit facilities, a corporate guarantee was executed by Essel Infraprojects Limited (“Respondent No. 2/Corporate Debtor”). Upon the Borrower’s continuous defaults, Respondent No. 1 sought recovery of its dues from the Corporate Debtor by filing an application under Section 7 of the Insolvency and Bankruptcy Code, 2016 before the National Company Law Tribunal (“NCLT”). The NCLT admitted Respondent No. 1’s application and inter alia relied on 6 (six) judgments in support of its decision.

Aggrieved by the NCLT’s admission order, the suspended director of the Corporate Debtor (“Appellant”) filed an appeal before the National Company Law Appellate Tribunal (“NCLAT”). In the appeal, the Appellant contended that the NCLT had failed to consider that the guarantee had relinquished on account of the transfer of the Borrower’s liabilities under a separate scheme of demerger and subsequent amalgamation. By relying upon the same judgments as the NCLT, the NCLAT upheld the NCLT’s admission order and dismissed the appeal (“Impugned Order”).

Aggrieved by the Impugned Order, the Appellant preferred a civil appeal before the Supreme Court inter alia contending that the judgments relied upon by the NCLT, and subsequently affirmed by NCLAT, were fake, non-existent, and likely AI-generated.

Issue

Whether judicial decisions based on fake or hallucinated AI-generated material are sustainable in law?

Findings and analysis

The Supreme Court allowed the civil appeal, set aside the orders passed by the NCLAT and NCLT and inter alia observed as follows:

courts must adopt a ‘zero-tolerance mode’ for producing, citing or using AI-generated precedents without verification. It amounts to misconduct on the part of an advocate to cite such judgments without verification and a serious lapse on the part of a judge to place reliance on fake or hallucinated AI-generated material as precedents in support of judicial determination;
a decision based on fake or hallucinated AI-generated material is not a decision in the eyes of the law, irrespective of whether such material had a direct or indirect bearing on the decision-making. Such decisions are to be set aside even if an iota of fake or hallucinated material enters the decision-making process, as this would violate the sanctity of adjudication and amount to subversion of the rule of law;
given that the judgments relied upon by NCLT were non-existent and some AI-generated paragraphs were wrongly attributed to genuine citations, the judicial process and the judgment under challenge stood tainted. Accordingly, the decision is unsustainable in law and liable to be set aside; and
the Bar Council of India must deliberate on the issue of members of the Bar submitting fake or hallucinated materials as if they are precedents of law and prescribe guiding principles to prevent such occurrences, along with disciplinary action for violation of such norms.

Conclusion

This decision has wide-reaching implications for the use of AI in legal proceedings. The Supreme Court has unequivocally held that the use of AI-generated fake or hallucinated material in the judicial decision-making process, even where such material has only a minimal bearing on the outcome vitiates the adjudication process and renders the resulting decision unsustainable in law. The judgment sends a clear message that AI cannot replace professional diligence in judicial proceedings and reinforces that, while AI may serve as a valuable aid in legal research, the responsibility for independently verifying the authenticity and accuracy of AI-generated material, including judicial precedents, rests squarely with advocates and courts.

This Prism is prepared by: