A 13-year-old Florida student has been arrested for allegedly asking OpenAI's ChatGPT "how to kill my friend" on a school device in the middle of class. The message was reportedly logged by Gaggle, a monitoring system employed on school computers, and was immediately flagged for review.
As per school security protocols, Gaggle's alert was forwarded to a campus police officer, who located and detained the student from Southwestern Middle School, according to a report by US news channel WFLA.
During police questioning, the teenager reportedly said that he was "just trolling" a friend. The officials, however, did not take the boy's claim lightly, given the United States' sensitive history with school violence.
Volusia County Sheriff's Office confirmed the arrest and subsequent booking of the boy into the county juvenile facility.
Following the incident, authorities issued a cautionary message to parents over minors' use of AI tools.
"Another 'joke' that created an emergency on campus. Parents, please talk to your kids so they don't make the same mistake," the sheriff's office said in a statement.
Not An Isolated Incident
In April, a 16-year-old California boy died by suicide after ChatGPT allegedly isolated him and helped plan his death.
His parents filed a lawsuit against OpenAI, saying that instead of helping him seek human aid, the chatbot supported his thoughts.
The family reportedly said he started using ChatGPT in the fall of 2024, mainly for homework, as other students did. He also used it to explore his hobbies, like music, Brazilian Jiu-Jitsu, and Japanese fantasy comics, and to ask about colleges and career paths.
Over the months, his conversations with the AI changed. Instead of just talking about school or hobbies, Adam began expressing more negative emotions and darker feelings, the family said.
Gaggle And Controversy
Gaggle has long been at the centre of controversy, having reportedly generated a few false alarms while also drawing criticism for creating a surveillance-like environment on school campuses.
According to AP, in the US, monitoring systems track nearly everything students write on school accounts and devices.
Thousands of schools rely on the software to flag potential risks, using artificial intelligence to scan conversations and immediately alert school officials or law enforcement.
