ECFMG: Quality controls ensure clinical skills exam is sound, reliable

Regarding "Evidence doesn't support push for clinical skills exam" (Column, Dec. 1, 2003): A recent commentary by AMA Board Chair William G. Plested III, MD, challenges the validity and reliability of the USMLE Step 2 CS examination to be implemented in 2004. Our experience provides a scientific basis for the clinical skills examination and quite a different interpretation than that of Dr. Plested. This assessment, a derivative of NBME prototypes and the ECFMG Clinical Skills Assessment, has been tested extensively with international and U.S. medical students and graduates.

Since 1998, ECFMG has performed some 40,000 assessments of international candidates in over 420,000 standardized patient encounters.

Unlike unstructured oral examinations, the CS examination assesses clinical skills under standardized conditions. Rigorous psychometric analysis of literally millions of data points, resulting in peer-reviewed publications, confirms the soundness, reliability and reproducibility of examination scores.

Based on equivalency studies done between the Philadelphia and Atlanta test sites, assessment decisions across test sites and sessions have been shown to be comparable.

Medical schools and other certifying bodies have used standardized patient examinations to assess clinical skills, and numerous publications support their use. Although assessment of history taking, physical examination and communication skills might seem to be potentially subjective, highly disciplined training of standardized patients, monitoring of their portrayals for consistency and rigorous quality control procedures applied to ECFMG testing over the past five years have ensured that examinee pass/fail decisions are accurate and reproducible. As a result, examinees are fairly and consistently evaluated regardless of where or when they take the examination.

Communication difficulties clearly lead to medical errors, patient dissatisfaction, negative health care outcomes, and litigation. Assessment of these skills as a condition of licensure will help to protect the American public.

James A. Hallock, MD, president

Gerald P. Whelan, MD, vice president, assessment services, Educational Commission for Foreign Medical Graduates, Philadelphia

Note: This item originally appeared at http://www.ama-assn.org/amednews/2004/01/12/edlt0112.htm.