You match with someone, you chat for days, you start to like them... and it turns out three out of four times, "they" were never real. But how did you get it so wrong - when you had calls with them, perhaps even video calls, when you followed their social media profiles that seemed absolutely legit?
Anthropic has revealed its AI, Claude was hijacked to run fake dating profiles that fooled as many as 25,000 people.
These weren't some obscure apps, mind you. They were on Google Play Store and Apple's App Store, looking just like any other dating app.
One of them, Kira, is still live as of September 17 on Google Play Store.
Apps like Dora, Doni, Romi, and Kira looked like normal dating apps. But most of the "matches" chatting with users weren't real, they were AI, built to talk, flirt, and keep the conversation going. Real humans were mixed in occasionally (for video calls) and social media follows, just to make it feel believable.
On the request of Verge, an AI cybersecurity researcher, Matthew Gore-Kormanik dug deeper into the scam. He, in fact, got a video call from a match named "Jennifer" on one of these apps. No real video, just a shaky curtain moving in the background as if she was in a hurry or calling secretly. His microphone wasn't even on, and yet she "complimented his voice" after over messaging. That was Gore-Kormanik's Eureka moment.
How Big Is The Scam?
Anthropic revealed that it discovered over 4,700 fake AI profiles, that chatted with at least 25,000 real people, and roughly 3 out of every 4 profiles people matched with were fake
The Claude-maker found this after noticing one account sending an unusually huge number of AI requests a day. Claude ran the autonomous conversational personas, at roughly 2.36 million messages over the two-week window, the company disclosed.
Users were duped of real money in all of this. These apps used "coins" or "gems" and users had to keep paying to keep chatting. So people were literally paying money to talk to a chatbot, thinking it was a real person or worse their soulmate.
Most of these apps have now been taken down, but Kira is still live on the Google Play Store as of September 17. Which basically shows that even after being flagged, tech companies haven't acted fast enough. And more worryingly it is so easy to create apps with AI these days that more are likely to mushroom as takedowns happen. One doesn't even need a trained developer today, people can vibe code their way using AI to create an app.
Anthropic says it has banned the accounts responsible and alerted both Google and Apple. The Dario Amodei led company was able to trace the scam operation to a China-based group.
Among the Anthropic revelations, what's really telling is that even the AI itself sometimes noticed something was wrong, like when users shared they were going through a tough time, but it kept role-playing anyway instead of stopping.
"In a small number of sampled cases, the model's own reasoning surfaced the harm, including exchanges where users disclosed serious illness or acute distress, yet the model didn't refuse to complete and instead the output continued in persona," Anthropic revealed.
Also Read: AI Behaved Well Until Scientists Made It Think Nobody Was Watching
Stay Alert
The simple, direct takeaway from all this is that the next time you match with someone new, don't just assume they're real. Especially if they always reply instantly, avoid video calls, or ask you to keep spending on the app to talk longer. And like we highlighted earlier even video calls and social media follows may be part of the elaborate ruse, so keep you guard up and if money comes into the picture, you are almost certainly being catfished.
Track Latest News Live on NDTV.com and get news updates from India and around the world
