Exam glitch erroneously fails some medical students

The new clinical skills exam hit and recently fixed a snag in its testing: A computer bug incorrectly failed 38 of the 8,000 students who took the test between Jan. 12 and March 30.

Peter Scoles, MD, the National Board of Medical Examiners' senior vice president for assessment programs, said they discovered a software problem during a quality control check of the U.S. Medical Licensing Examination Step 2 Clinical Skills exam results. This is the first year that graduating medical school students are required to take the test, which organized medicine initially met with resistance but has since accepted.

Dr. Scoles said that he was personally calling those who the problem affected. The error is "something that the organization deeply regrets, but it [uncovering the error] shows the kind of process that we're committed to, to keep checking and rechecking," he said.

Of the 38 students affected, 14 were from U.S. medical schools and 24 were from international programs. Dr. Scoles said the students took the news well.

"I've talked to nine of the 14 [U.S. students] and with few exceptions things have turned out OK," he said, "Eight have residencies so far."

Those who took the test before Jan. 12 or after March 30 were not impacted, and their scores stand.

NBME officials also contacted the deans of the U.S. medical schools where each of the students is enrolled. A corrected score report is being sent to organizations that received the previous, incorrect results, and the board is establishing a personal liaison to help students who encountered problems resulting from the computer glitch.

How the error occurred

Students taking the Step 2 Clinical Skills test have an eight-hour period to evaluate 11 or 12 standardized patients who are actors trained to present with a specific set of symptoms. Students are scored based on their clinical encounter, which includes a patient history, physical exam and diagnosis, and initial work up plans, as well as communication and questioning skills and ability to speak English clearly.

The computer error occurred when student scores for note taking were assigned to the wrong individuals.

"This is not the first time we've had someone go from fail to pass at the USMLE," Dr. Scoles said. "When we first rolled out the computer-based testing, we had several episodes when individuals went from fail to pass, but this is probably the most we've had at one time."

Between mid-June 2004 and mid-March 2005, about 17,700 students took the Step 2 Clinical Skills exam. About half of these medical students were from the United States and Canada; the other half were from international medical schools. American and Canadian students passed 96% of the time, and international students passed 85% of the time.

Students must pass the three-part USMLEs to get a U.S. medical license. The clinical skills part had been required only of international medical graduates until this academic year.

Those who are concerned about their test scores can phone 866-504-8564 within the United States or Canada, 215-590-9260 for those outside the United States or Canada; or e-mail ([email protected]" target="_blank">link).