IIT Kharagpur Student Trials ChatGPT o3 On JEE Advanced Mock Test, Stunned By Result

Artificial intelligence (AI) has revolutionised numerous industries, from cutting-edge humanoid robots and self-driving cars to unexpected domains like relationship counselling. Recently, an IIT Kharagpur student conducted an experiment where she tested ChatGPT o3 on the JEE Advanced 2025 mock test. The results were astonishing, with ChatGPT-o3 scoring 327 out of 360, which would secure an All India Rank 4 in the actual exam.

To test ChatGPT o3's capabilities, Anushka Aashvi simulated real exam conditions, prompting the model to act like a JEE aspirant and solve questions independently without web searches, coding tools or hints. Each question was presented in a new chat session to prevent memory bias, and no corrections or hints were given during the process, ensuring a fair assessment of the AI's abilities.

"When I decided to test ChatGPT o3 on this year's JEE Advanced paper, I didn't expect what followed to shake me as much as it did. Giving away the result straightaway, ChatGPT o3 scored a whopping 327/360 in JEE Advanced 2025 Question Paper," Ms Aashvi wrote in a blog on Heltar.

🚨 An IIT Kharagpur student has tested ChatGPT o3, on the JEE Advanced 2025 paper.

The AI scored a staggering 327 out of 360, a score that would earn it All India Rank 4 in the real exam. 😉 pic.twitter.com/QL0V9oGI4J

— Indian Tech & Infra (@IndianTechGuide) June 8, 2025

Notably, the AI achieved perfect scores of 60 in both Chemistry and Mathematics in the second phase, with minor errors only in Physics and earlier sections.

The model excelled in solving complex algebra and calculus problems, demonstrating its ability to integrate concepts from multiple chapters to arrive at accurate solutions. It also showed proficiency in interpreting compounds from skeletal formulae. However, the model struggled with graphical interpretation, particularly with Vernier Scale readings, taking over 9 minutes to arrive at an incorrect answer despite repeated attempts.

"It was not able to understand the Vernier Scale readings. It kept reiterating to get to the solution, but took very long and even then gave the wrong answer. But an overall score of 327/360 is truly remarkable," Ms Aashvi added. 

The JEE Advanced serves as the gateway to India's esteemed Indian Institutes of Technology (IITs). Out of over 1.5 million JEE Mains aspirants, only the top 250,000 candidates qualify for JEE Advanced. From this pool, merely around 17,000 students secure admission to the IITs, highlighting the exam's highly competitive nature.