The National Testing Agency (NTA) uses a multi-stage system to prepare and secure examination papers, involving multiple sets of experts, independent vetting, translation checks, algorithm-based randomisation and sealed storage. But the process has come under scrutiny after the agency ordered a re-examination for UGC-NET English, Commerce and Sociology papers following multiple errors.
The three papers, conducted between June 22 and June 30, were found to have "multiple errors", prompting NTA to conclude that the defects were too extensive to be addressed simply by deleting individual questions after the challenge process.
NTA said the "defects" in the three papers were extensive and "cannot be cured merely by dropping questions after the challenge process". It said papers with such defects "do not meet the standards of fairness and error-free examination".
The three papers will now be conducted afresh on September 9. English will be held from 9 am to noon, while Commerce will take place from 3 pm to 6 pm.
The episode has put the spotlight on the elaborate process through which NTA prepares its question papers, and the safeguards intended to ensure that errors are detected before a paper reaches candidates.
Question Bank Is Created First
The process does not begin with the final examination paper. Subject experts from reputed academic institutions are first tasked with creating a large pool of questions and answer options. The experts are not told whether the questions they prepare will ultimately be selected for an examination.
The question bank is deliberately kept much larger than the actual requirement. According to the government's affidavit filed in the Supreme Court on August 4, it contains at least five times the number of questions required for an examination.
The questions then go through a separate moderation process. Another group of subject experts reviews them for syllabus coverage, correctness of the answers and broad consistency in difficulty before selecting questions for different paper sets.
The structure is intended to ensure that no individual has access to the complete set of questions that will eventually be presented to candidates.
A Separate Team Vets The Questions
The questions selected by moderators undergo another round of scrutiny. Independent vetters solve the question sets and their answers are compared with the answers proposed during moderation. Any disagreement is taken up with the examination director concerned, with the moderators and vetters brought together to resolve the issue.
The government's affidavit says different stages are ideally assigned to different experts. It also states that multiple question-paper sets are prepared "entirely separately", without overlap in either the questions or the experts involved.
Translation Goes Through Another Check
After the sets are finalised, the questions are translated into the languages in which the examination is conducted. The translated versions are then subjected to back-translation into English by another group of subject experts. This is intended to establish whether the original meaning and context have been retained.
Most NTA examinations are conducted in 13 languages, according to the government. NTA has also developed an AI-assisted translation system operating in an "air-gapped" environment. The system generates an initial translation, which is subsequently checked by human translators.
Importantly, the agency translates the entire question bank into 12 Indian languages rather than translating only the questions finally selected for an examination. This means translators work on a much larger pool and cannot know which questions will eventually be used.
The translation facility is isolated from the internet and does not permit personal devices, external storage media or cloud connectivity. Access for individual translators is also time-limited.
Question-setting Takes Place Under Tight Restrictions
People familiar with NTA's paper-setting process said separate groups of senior faculty members are brought to the agency's Delhi premises for around a week to prepare questions.
The experts hand over their mobile phones and other electronic devices before entering the premises. Reference material required for question-setting is made available inside.
Each panel comprises a convenor and four subject experts. The teams work independently and are not aware of the questions being prepared by other panels.
The panels create large question pools rather than the final examination paper. The final sets are generated later through algorithm-based randomisation.
"Even after the panels have completed their work, the final combination of questions is not known to any one person. The exact paper emerges only at the final generation stage," an NTA source familiar with the process said.
How Can Repeated Questions And Typos Still Occur?
Despite the multiple stages of scrutiny, the possibility of errors remains. A senior NTA official said questions from an earlier paper set could be reused and subsequently combined with questions from a newer set, potentially resulting in repetition.
"Repetition can happen when some questions from an earlier set are carried forward and then get mixed with questions generated for a subsequent set," the official said.
The official attributed spelling and wording errors largely to lapses during proofreading and translation. "Typos and similar errors are generally the result of oversight during proofreading or translation. In these stages, contractual personnel are also involved, which means the final checks become particularly important," the official said.
Multiple Sets Are Printed And Kept Sealed
Once the question sets have gone through translation and finalisation, they enter the next stage-printing, packing, transportation and secure storage. According to the government's affidavit, these processes are carried out in parallel for the different sets.
The system also provides for a replacement if a particular set is suspected to have been compromised before the examination. NTA can substitute another sealed set in such a situation. The final decision on which set will be administered is taken on the morning of the examination. Only one set is opened and used. The remaining sets continue to remain sealed.
UGC-NET Errors Put Quality Checks Under Scrutiny
The multiple layers are primarily designed to ensure two things: that the final paper remains secure and that errors are identified before candidates see it. The UGC-NET episode has now raised questions about the second part of that system.
While NTA has separate stages for question creation, moderation, independent vetting, translation, back-translation and final paper generation, errors across three papers were significant enough for the agency to scrap them rather than resolve individual questions through the usual challenge process.
The re-examination will therefore be closely watched not only for the security of the question papers but also for whether NTA's elaborate quality-control process succeeds in preventing the kinds of errors that triggered the re-test in the first place.
