Physician profiling found wrong 22% of the time

Researchers used a rating program similar to ones health insurers use. Only cost was measured, although many plans merge cost and quality into one score.

By Emily Berry — Posted April 12, 2010

Print  |   Email  |   Respond  |   Reprints  |   Like Facebook  |   Share Twitter  |   Tweet Linkedin

Physicians are at a strikingly high risk of being miscategorized when rated by cost-profiling programs, according to research mimicking the types of tiering programs health plans have used.

The study, funded in part by the U.S. Dept. of Labor and conducted by researchers at RAND Corp., outlined the outcome of a mock profiling program using claims from four Massachusetts health plans. The authors created a sample program similar to ones health plans have used to rate physicians' efficiency.

Using the insurers' claims from 2004 and 2005, researchers found that the model two-tier program misclassified as many as 22% of physicians, and that about two-thirds of physicians' designations would be less than "optimally reliable."

"Consumers, physicians and purchasers are all at risk of being misled by the results produced by these tools," concludes the study, which appeared in the March 18 New England Journal of Medicine.

John L. Adams, PhD, the RAND statistician who led the research team, said his aim was to add some hard evidence to the debate between health plans and physicians, which has been contentious and often ended in stalemate.

The study looked at cost profiling, not quality measurement, though many health plans merge the two into a single "efficiency" or "performance" score. Physicians have complained that those scores have been based more heavily on cost than quality.

The study's authors measured reliability by using a statistical analysis that accounted for a normal amount of variation or "noise" associated with each physician, then compared that with the variability that showed up in the tiering system. The model program produced a level of variability that meant many physicians ended up misclassified.

"What does it mean for a tool to be good enough?" Adams said. "No tool is going to be perfect ... I think there are people that look at a 25% misclassification rate and say, 'Great, better than what we had yesterday,' but almost no physician would find a 25% misclassification rate acceptable."

He said he didn't want to take a position on what percentage misclassification might be acceptable. But others did not hesitate to do so.

"It's critically important that patients and physicians get clear, accurate information about the cost and quality of health care. But this report, produced by an independent, renowned research firm, clearly demonstrates that these profiling programs fail to accomplish those goals," Mario Motta, MD, a cardiologist in Salem, Mass., and president of the Massachusetts Medical Society, said in a statement.

The Massachusetts Medical Society has sued the state's Group Insurance Commission, which purchases health coverage on behalf of state and some municipal employees, over what physicians believe are serious flaws in the GIC's physician profiling program.

The society owns the NEJM, and the authors disclosed past grant support from the organization, as well as grant support and consulting fees from the American Medical Association and other medical societies.

Dolores Mitchell, GIC executive director, who in the past has defended the profiling initiative as an imperfect but necessary program, did not respond to requests for comment.

The AMA pointed to the study's results as support for its long-held position that cost-profiling programs aren't designed well enough to give either patients or physicians good information.

"Inaccurate information can erode patient confidence and trust in caring physicians, and disrupt patients' longstanding relationships with physicians who have cared for them for years," AMA President J. James Rohack, MD, said in a statement.

He called on health insurance companies to "abandon flawed physician evaluation and ranking programs, and join with the AMA to create constructive programs that produce meaningful data for increasing the quality and efficiency of health care."

The AMA-led Physician Consortium for Performance Improvement is a project that focuses on outlining useful and reliable clinical measures of quality. The PCPI has introduced 266 model quality measures but has not published model cost-profiling methodology.

AMA policy calls for rating systems to be transparent and based on evidence-based quality measures rather than cost. The AMA also urges health plans to give physicians time to review the data behind such ratings and allow physicians to appeal if they disagree with the basis for their designation or score.

Robert Zirkelbach, spokesman for health insurance trade group America's Health Insurance Plans, said the field of physician ratings is "an evolving area."

"The report does highlight what health plans agree are two important areas that need to be addressed," he said. "First, there has to be an adequate sample size, and two, that you're attributing performance to the right physicians. Doing this in collaboration with physicians is key, so patients can have the peace of mind knowing the data is reliable and something they can trust."

Some health plans have been forced to scale back cost-profiling programs or reinvent them, most notably in New York. In that state, Attorney General Andrew Cuomo in 2007 sued insurers doing business there, demanding they abandon cost-based profiling programs he said were disguised as quality measurements.

UnitedHealth Group, Cigna, WellPoint, Aetna and several smaller insurers, while not admitting any wrongdoing, settled with the attorney general's office, agreeing to disclose the basis of physician tiering programs and tie them to recognized standards of quality, not solely cost. The largest plans also pledged to take those changes national rather than limiting them to the New York market.

In 2008, most of the country's largest insurers signed on to a charter agreement developed by the Consumer-Purchaser Disclosure Project, a group of businesses and consumers who demanded more transparent and meaningful quality measurement. The insurers agreed to have their tiering programs audited by the nationally recognized organization and to work toward using the same measures industrywide to gauge physician performance.

Back to top

External links

"Physician Cost Profiling -- Reliability and Risk of Misclassification," abstract, New England Journal of Medicine, March 18 (link)

Back to top



Read story

Confronting bias against obese patients

Medical educators are starting to raise awareness about how weight-related stigma can impair patient-physician communication and the treatment of obesity. Read story

Read story


American Medical News is ceasing publication after 55 years of serving physicians by keeping them informed of their rapidly changing profession. Read story

Read story

Policing medical practice employees after work

Doctors can try to regulate staff actions outside the office, but they must watch what they try to stamp out and how they do it. Read story

Read story

Diabetes prevention: Set on a course for lifestyle change

The YMCA's evidence-based program is helping prediabetic patients eat right, get active and lose weight. Read story

Read story

Medicaid's muddled preventive care picture

The health system reform law promises no-cost coverage of a lengthy list of screenings and other prevention services, but some beneficiaries still might miss out. Read story

Read story

How to get tax breaks for your medical practice

Federal, state and local governments offer doctors incentives because practices are recognized as economic engines. But physicians must know how and where to find them. Read story

Read story

Advance pay ACOs: A down payment on Medicare's future

Accountable care organizations that pay doctors up-front bring practice improvements, but it's unclear yet if program actuaries will see a return on investment. Read story

Read story

Physician liability: Your team, your legal risk

When health care team members drop the ball, it's often doctors who end up in court. How can physicians improve such care and avoid risks? Read story

  • Stay informed
  • Twitter
  • Facebook
  • RSS
  • LinkedIn