Four ways to conduct a fair, top-quality NEET

When paper and pen tests are utilised, groups aiming to engineer paper leaks have access to the latest technology to coordinate their plans. Collusion from a few individuals at printing centres or test centres (of which there are hundreds) is enough. Successful paper conduction without leakages depends on hundreds of individuals remaining honest in the face of significant financial temptation. For these reasons, I would argue that conducting leak-proof paper-pen tests is near impossible – the financial rewards will always make the risk worthwhile.

It is, therefore, necessary to shift all testing to computer-based tests (CBTs) immediately. The JEE Mains, a high-stakes examination similar to the National Eligibility-cum-Entrance Test (NEET), and indeed, all modern national and international assessments, are conducted in this manner. While there have been instances of hackers attempting to compromise CBTs, this can be mitigated by monitoring for unauthorised software on computers used in the tests. Solutions include providing internet connections only for the first 10 minutes (to download encrypted questions) and 10 minutes after the test (to sync locally saved responses), or employing Artificial Intelligence (AI)-based proctoring, which is continuously improving.

Having addressed paper leaks and technology-driven cheating, let’s focus on setting high quality test papers that would not only reduce the incentives to go to extreme lengths to cheat but also ensure that not too many students score the top scores.

High quality papers of appropriate difficulty level: There is a notion in India, even repeated by senior leaders of institutions like the University Grants Commission (UGC), that more difficult examinations lead to greater coaching. However, the level of coaching is likely higher for higher stakes exams, regardless of difficulty. Making easier papers or basing them on a narrowly defined syllabus (like Class 12 NCERT textbooks) is likely to reduce the quality and difficulty, while increasing their coachability.

The difficulty level should be set based on the level at which the exam aims to discriminate. A board exam should discriminate at the average (or 50th percentile) student level. Exams like NEET need to discriminate at the 90-95th percentile and hence require difficult, yet valid, exams. A valid exam is one that top students or subject experts can answer correctly. When the correctness of a question hinges on a factual error in a textbook edition, as happened in this year’s NEET, it is not a sign of a high quality paper.

Multiple choice questions (MCQs) testing deep understanding with limited subjective questions: There is a common belief that MCQs can only test recall of facts and are prone to guesswork and cheating. These notions are factually incorrect. MCQs that test deep understanding and include common misconceptions as wrong choices (to tempt all but the best students) can be better for exams like NEET or JEE than short or long answers. While the manual correction of other question types introduces subjectivity, certain skills like making persuasive arguments or solving complex problems may not be measured by MCQs. Therefore, such tests should include one or a few subjective questions in a second paper, which would be manually scored only for, say, the top 20% of candidates if the goal is to eventually select the top 10%.

Transparent release of item performance data for a sample of questions: The level of transparency and anonymised performance data-sharing for exams like NEET must be increased dramatically. Even if all questions are not made public (to allow reusing some for anchoring purposes in future rounds), 20-40% of items, along with detailed performance characteristics (like the percentage of students choosing each option and the correlation between performance on the question and the overall paper), should be released. Two important benefits arise. First, it keeps test-makers focused on good quality questions, as poor ones stand out in such analysis; second, it boosts confidence in the test and testing process among candidates and the public.

Proper use of percentiles and scaled scores to determine ranks: The marks obtained by a student (raw score) are meaningless in a computer-based exam where different students get different papers. Even when students write the same paper, the actual questions two students with the same raw score may have answered correctly are different, rendering raw scores meaningless. Instead, well-known psychometric techniques should be used to calculate scaled scores (and percentiles based on the scaled scores), and only those should be shared, as is the common international practice.

Use of advanced psychometric techniques to detect copying and improve future papers: Statistical and psychometric analysis can detect phenomena like cheating or questions that did not work (possibly due to ambiguity). Such questions can be dropped, and candidates found to have used unfair means can be disqualified. This will increase the exam’s credibility and fairness.

While the National Testing Agency (NTA) depends on private agencies for conducting JEE Mains, it conducts NEET itself. Creating a high-quality yet fair paper is achievable, but only if it is treated as a highly technical endeavour driven by specialised agencies with expertise in subjects, assessment techniques, psychometrics, technology, and test operations. At stake is the sanctity of the test and the futures of not just candidates but society itself, as our future professionals are being selected this way.

Sridhar Rajagopalan is co-founder and chief learning officer of Educational Initiatives Pvt Ltd, a 20-year-old assessment and education technology company. The views expressed are personal