"Invisible, Insidious": Top Court Warns Against Use Of AI In Legal Process

The Supreme Court on Thursday expressed grave concern over a National Company Law Tribunal (NCLT) judgment which relied on fake precedents and cases generated using Artificial Intelligence (AI).

While setting aside an NCLT verdict on Essel Infraprojects insolvency, the top court observed that human control over adjudication must remain total and absolute at every stage and warned against AI making professionals dependent.

The court made these observations after finding that the NCLT had relied on non-existent, fake, and hallucinated judgments or past precedents generated through artificial intelligence tools.

"This byproduct, that is AI, that is the production of fake, non-existing, and hallucinated material, and its utilisation, as precedents in law, is like release of methyl isocyanide with the province of law and justice. Invisible, insidious, and catastrophic, by the time anyone notices, it is not only contaminant, but takes away the very lifeline of judicial determination," it added.

A bench of Justices PS Narasimha and Alok Aradhe held that such reliance on hallucinated material struck at the integrity of adjudication and its processes.

"This is yet again a case where the Tribunal relied on non-existent, fake and hallucinated material, generated through Artificial Intelligence (AI), as if it were a precedent in support of its judgment," the court noted.

The court emphasised the need to have a human in loop at every stage as the courts move towards adopting AI in adjudication.

"What is significant for our decision-making is our resolve to adopt artificial intelligence technology in the aid of adjudication, while at the same time asserting, and declaring total and absolute control over adjudications, to the human in the loop, at every stage," the judgment stated.

The court acknowledged that Artificial Intelligence has acquired the capability to better, if not fully substitute, human effort, both routine and intellectual.

"This extraordinary capability, amid increased workloads of modern life, is compelling professionals to adapt and employ AI for intelligent, efficient and swift functioning," the court said while also citing example of Garfield Law Limited (GLA)- first purely Artificial Intelligence-driven law firm approved in UK to provide regulated legal services.

The court said that result from AI can be gratifying, even inspiring; yet if left unregulated, Artificial Intelligence may infiltrate our intellectual work ethic and, before long, render us dependent on its vast capabilities.

It emphasised that while AI can be adopted as an aid, human control over adjudication must remain total and absolute at every stage.

The court further said that while technology has been seamlessly absorbed by courts and made an integral part of the judicial system, AI stands on a different footing.

"The story of AI, as it is unfolding, is, however, different. In fact, transformative as it is, not just an aid to assist us in our work, but it is an alternative to our own thinking, reasoning, and even decision making," the court said.

The court said judges need to be extra cautious, as unregulated use of AI will insidiously enter legal practice and process of justice, decision making itself.

The top court went on to clarify that it was not dealing with the cause of hallucinations or the technical process for resolving them, which it said was for engineers and scientists to deal with.

The Supreme Court also directed the Bar Council of India to constitute a committee and examine the issue in detail.

The case arose from a plea filed by suspended director Pooja Ramesh Singh against orders of NCLT admitting Essel Infraprojects to corporate insolvency resolution process on a Section 7 application filed by Jammu and Kashmir Bank Limited.

The NCLT Mumbai Bench had, on August 28, 2024, admitted the insolvency plea for a claimed debt of Rs 87.43 crore. The NCLAT upheld the order on September 11, 2025.

The debt arose from a Rs 200 crore facility granted by Jammu and Kashmir Bank to Pan India Utilities Distribution Company Limited.

The facility was secured by a corporate guarantee from Essel Infraprojects and a mortgage over land at Gorai, Borivali in Mumbai.

The NCLT Bench comprising Judicial Member Rita Kohli and Technical Member Madhu Sinha rejected this argument and held that the corporate guarantee continued to bind Essel Infraprojects.

However, the Supreme Court found that the tribunal had relied on non-existing and hallucinated material as precedents in support of its judgment.

The court set aside the judgement of the NCLT, as well as NCLAT, and asked the tribunals to decide them on the facts.