In the space of a few days, a senior minister appeared to say something inflammatory he never said. A prime minister appeared to make comments he never made. A protest leader appeared to call off a movement he never called off. All three videos were fake. All three were generated by AI, and all three spread on social media before anyone could stop them. That is the real story of the recent Gen Z-led protests: not just what people were angry about, but how easily that anger could be weaponised and set loose at scale.
AI misinformation is a live governance problem, and India needs a serious national conversation on regulating such synthetic content. Not to restrict innovation, but to ensure AI does not supercharge misinformation and hate speech capable of disturbing public order in a country as socially, culturally, and linguistically diverse as India.
Traditional misinformation required real effort to scale. AI collapses that cost, as anyone can now produce a realistic fake video, voice clip or image in minutes, publish it instantly, and watch algorithms push it to hundreds of millions before any fact-checker or law-enforcement agency notices. It is this combination of cheap creation, instant publication and algorithmic virality that makes generative AI different in kind, not degree. By the time authorities identify one fake, several variants may already be circulating across languages and regions, which is why this can no longer be treated as a content-moderation problem alone.
Once harmful AI content reaches millions, it becomes a law-and-order challenge, and sometimes it is already too late. India's law-enforcement machinery is structurally outmatched here: cybercrime units are stretched thin across a twenty-plus-language information space, forensic verification takes hours a viral clip will not wait for, and jurisdictional friction (platforms headquartered abroad, complaints filed in one state about content viewed nationwide) slows things further.
Police can register an FIR and courts can eventually establish the facts, but none of that stops the first few hours of unchecked spread, when the real damage is done. The goal must be to intervene before content goes viral, putting the burden on platforms that control amplification, not on police left cleaning up afterwards.
Accountability Must Sit with Big Tech Platforms
There are three points in the life cycle of AI content where responsibility could apply: creation, publication and viral amplification. Creation itself should not be heavily restricted. Generative AI has enormous legitimate value in education, creativity and business, and society should not stifle it because it can be misused.
Real accountability should begin at publication. Platforms already deploy sophisticated systems to catch child sexual abuse material and terrorist content, and can build comparable safeguards for high-risk synthetic content: deepfakes that impersonate public figures, fabricate riot footage or incite communal hatred.
Most important, though, is virality. Even after harmful content slips past initial moderation, platforms have rich signals once it starts spreading: engagement spikes, abnormal sharing, coordinated behaviour and user reports. AI systems are themselves capable of flagging synthetic media after the fact. If they can detect it post-virality, it is unacceptable that they lack the capability to catch much of it during its spread.
The trouble is that commercial incentives cut the other way. Emotionally charged, divisive content drives more engagement, a dynamic often called 'rage farming'. Platforms cannot claim to be neutral hosts while running algorithms that decide what millions see. Once a platform makes that editorial choice at scale, responsibility must extend across the content's entire life cycle, not end at hosting.
AI-generated misinformation does not manufacture division out of nothing. It exploits existing fault lines, lending fabricated narratives more credibility and speed, which is precisely why this has become a public-order issue, not merely a technology one.
Equal Safeguards for India
Big Tech and AI platforms do not apply uniform safeguards everywhere. During US elections, these companies invested heavily in election-integrity operations, fact-checker partnerships and rapid-response detection. Similar patterns have emerged across Europe as regulatory pressure has grown. In India, platforms have mostly taken the position of neutral intermediaries, responding only after content has already spread.
If companies can deploy stronger safeguards in the US and Europe, why should Indian users, in a country of far greater scale and linguistic complexity, receive weaker protection? Part of the answer is cost. Multilingual moderation is expensive, and platforms have been reluctant to fund it uniformly. A more uncomfortable possibility is structural. Many of the same companies that build AI tools capable of generating synthetic media also own the platforms that distribute it, complicating accountability when both functions sit within one ecosystem.
This should not be acceptable to India. With over 900 million internet users, India has the leverage and the legitimate national interest to demand the rigour it sees deployed elsewhere. Recommendation algorithms are built to maximise engagement. Virality is the objective, not a side effect, which may be commercially defensible in general. But once AI-generated content touches communal harmony, law and order, elections or national security, commercial incentives cannot override the safety of Indian citizens. Big Tech does not get to treat India as a lower-safeguard market simply because enforcement here has historically been slower to bite.
What India's New IT Rules Attempt
India began addressing this through the Information Technology (Amendment) Rules, 2026, marking a shift from reactive moderation to preventive compliance. AI-generated audio, video or images indistinguishable from real people or events now attract a higher standard of due diligence, and platforms must deploy automated detection to prevent uploads of unlawful synthetic content, such as impersonation of public figures, fabricated riot footage and fraudulent records.
The framework also emphasises transparency through watermarking, audio disclosures and provenance markers such as digital fingerprints or tools like SynthID, and it tightens takedown timelines, since every additional hour multiplies reach.
Critically, compliance is no longer voluntary, and intermediaries that repeatedly fail these obligations risk losing safe-harbour protection under Section 79 of the IT Act, exposing them to direct legal liability.
But labelling and provenance metadata mean little if platforms do not treat them as an operational priority. Detection must run at publication and, more importantly, at the point content begins to go viral, not days later, after a government notice.
In a country as vast and interconnected as India, unchecked AI-generated misinformation, deployed by extreme elements on every side of the political divide, can become a force multiplier for violence that no police force can contain after the fact. That burden has to sit primarily with Big Tech and AI companies, not an overstretched enforcement apparatus playing catch-up. These are the same companies that build the systems capable of generating synthetic media and the platforms that decide, algorithmically, how far it travels. They cannot profit from both ends of that chain while disclaiming responsibility for either.
(Vinay Sarawagi is Chief Content Officer at NDTV Network)
Disclaimer: These are the personal opinions of the author
