NDTV Exclusive - Inside Meta's Child Safety: Current Systems Miss Nuance, New AI Under Test

Weeks after NDTV's investigation exposed sexually suggestive videos involving minors, AI-generated exploitative content, coded hashtags and captions, and networks directing users to Telegram channels, Meta has acknowledged limitations, saying that its current automated enforcement systems 'often rely on straightforward patterns' and 'aren't as good at capturing nuance'.

The tech giant disclosed that it is developing a new generation of AI models to strengthen enforcement against child sexual exploitation on Facebook and Instagram.

The response, shared exclusively with NDTV, came after detailed answers were sought on how Meta's algorithms work, whether its child safety technology has been revamped following the findings, and how artificial intelligence and human moderators work together to police harmful content. 

The disclosures also come at a time when the social media giant is facing mounting scrutiny in India over its content moderation practices.

In recent weeks, Meta has been summoned by the Ministry of Electronics and Information Technology (MeitY) over multiple issues, including child sexual abuse material as well as temporary removal of a Facebook video posted by Prime Minister Narendra Modi during the Cockroach Janta Party protests. The company later apologised, attributing the takedown to a technical error.

The scrutiny intensified after Hyderabad police registered a case against Meta India's head over Facebook videos allegedly depicting the Prime Minister in an abusive manner.

On Wednesday, Meta's chief global affairs officer, Joel Kaplan, also apologised over the temporary restriction on PM Modi's post as the centre pressed the company to strengthen its safeguards and enforcement mechanisms.

Limits Of Current Systems

NDTV asked Meta whether the company's systems had been upgraded following the investigation and how it intended to address increasingly sophisticated forms of online child exploitation. In response, Meta acknowledged a limitation of its existing moderation technology in critical areas. 

"While our current automated systems often rely on straightforward patterns, they aren't as good at capturing nuance," the company said. Meta said that is precisely what its next generation of AI models is designed to address.

According to the company, unlike its current automated systems, which largely rely on probability scores and established patterns to determine whether content violates policy, the new models analyse the full context of a post, including intent, tone, syntax and language variations.

The objective, Meta said, is to enable the technology to make more informed decisions in complex cases where meaning is not immediately obvious. The company also said that it is already testing the upgraded models.

"Recently we've been testing significantly more advanced AI models. We're finding these enhanced models deliver more accurate and comprehensive enforcement than our current systems, reducing mistakes and more effectively finding and removing harmful content," a Meta spokesperson told NDTV.

Meta said the upgraded systems generate plain-text explanations for every moderation decision, making enforcement easier to audit and errors easier to diagnose. Unlike existing systems, which can take time to adapt as abuse patterns evolve, the company said the new AI can be updated and fine-tuned much more rapidly, allowing faster responses as online predators adopt new tactics, coded language and evasive techniques.

The company also described the models as representing a fundamental shift in how its moderation systems are trained. Rather than learning primarily from historical moderation decisions, which Meta acknowledged can perpetuate previous errors, the newer models are trained directly on its Community Standards, which it says improves consistency, fairness and transparency in enforcement.

Meta said it plans to gradually transition to the new systems over the coming years, with routine moderation increasingly automated while human reviewers focus on the most complex and sensitive cases.

Inside The Moderation Playbook

NDTV's investigation documented sexually suggestive content involving minors that remained accessible on Meta's platforms despite existing safeguards. While Meta permits certain content such as breastfeeding videos shared for awareness or educational purposes, the investigation also found that similar imagery was also being used in sexually suggestive contexts using minors. Several such videos amassed hundreds of thousands of views before they were removed after NDTV flagged them to the company.

Asked how it detects child sexual abuse material (CSAM), Meta said it deploys technologies including PhotoDNA, artificial intelligence and machine learning to identify both known and previously unseen CSAM. According to the company, its systems detect child nudity, suspicious interactions involving minors and accounts promoting exploitative content. 

The company also said it limits contact between suspicious adults and teenagers, automatically blurs explicit images sent to teens and displays warnings when conversations show signs of potential sextortion.

NDTV also asked how Meta distinguishes between legitimate content involving children and material that sexualises or exploits minors – a challenge highlighted by the investigation. Meta said its AI is designed to analyse multiple elements of a post before making an enforcement decision.

"Sometimes, the meaning of a piece of content is immediately obvious to a person but less clear to technology. To keep people safe, we train our Artificial Intelligence technology to first look at all the components of a post and only then determine the true meaning," the company said. 

Another key question put to Meta was how sexually exploitative content involving minors often disguised using coded captions, hashtags and AI-generated imagery was able to gain traction before enforcement action was taken. The company pointed to the very systems designed to help users discover content.

"We make recommendations in places like Reels and Instagram Explore to help people discover new things on our apps, and people use features like 'Search' and 'Hashtags' to find things they might be interested in," Meta said. The company maintained that while these recommendation systems are intended to help users discover relevant content, they are also backed by automated enforcement systems designed to proactively remove or stop recommending material that violates its rules.

Meta argued that online child exploitation presents a constantly evolving challenge because offenders continuously change the language, hashtags, emojis and spellings they use to evade detection. To counter this, the company said it combines artificial intelligence with user reports to identify violating hashtags, continuously expands its database of child safety-related keywords and coded terminology, and uses machine learning to identify new phrases, misspellings and spelling variations as they emerge.

Once identified, Meta said, these coded terms can be blocked from search results and trigger enforcement against related accounts, groups and pages across both Facebook and Instagram. "Continually expanding the list of child safety-related terms, phrases and emojis for our systems to find. We have many sources for these terms, including non-profits and experts in online safety, our specialist child safety teams who investigate predatory networks to understand the language they use, and our own technology which finds misspellings or spelling variations of these terms," the company said.

Meta also said that it is increasingly relying on machine learning to identify previously unknown code words before they become widespread. "We're using machine learning technology to find relationships between terms that we already know could be harmful or that break our rules and other terms used at the same time. These could be terms searched for in the same session as violating terms or other hashtags used in a caption that contains a violating hashtag," the company said.

NDTV also asked Meta what role human moderators continue to play alongside artificial intelligence, when content is escalated for review and how moderators are trained to identify nuanced child safety and sexual exploitation cases.

Meta said AI is only one layer of its enforcement system and that decisions involving the "highest-risk and most critical issues" continue to involve human reviewers.

According to the company, AI is primarily used to detect and remove large volumes of violating content and handle repetitive moderation tasks at scale. When its systems cannot confidently determine whether content violates policy or where contextual judgement is required, the material is escalated to reviewers with the appropriate language and subject-matter expertise.

Those cases, Meta said, can then be referred to specialist global operations and content policy teams for further review. "As potential content violations are routed to review teams, each reviewer is assigned a queue of posts to individually evaluate. Sometimes, this review means simply looking at a post to determine whether it goes against our policies, such as an image containing adult nudity, in instances when our technology didn't detect it first," the company said.