P4P found to have little impact on care quality

A new study finds that performance bonuses are not big enough to yield significant improvements. Some say more reform is necessary to reward recommended care.

By Kevin B. O’Reilly — Posted Aug. 4, 2008

Print  |   Email  |   Respond  |   Reprints  |   Like Facebook  |   Share Twitter  |   Tweet Linkedin

The idea of paying physicians more for providing guideline-based care has taken the American health system by storm in the last decade. Today, more than 150 pay-for-performance programs are centered on the notion that rewarding evidence-based care is key to improving health care quality.

But research shows there is a fundamental problem with the P4P programs: They have had little to no impact on quality. That is the conclusion of many studies, including a new analysis of quality incentives, published in the July/August Health Affairs. It compared 81 Massachusetts physician groups eligible for quality incentives with 73 that were not.

Unlike earlier studies, researchers had access to data from a cross section of doctors and payers -- five health plans that contract with more than 90% of the state's practicing primary care doctors. The study found that overall performance from 2001 to 2003 improved on 73% of preventive care measures such as diabetes hemoglobin A1c testing, breast cancer screening and well-child visits.

But the performance of the 5,350 physicians analyzed was statistically indistinguishable. Everyone's quality improved, regardless of whether the physician group stood to earn a bonus, which ranged from $200 to $2,500 per quality measure for an individual physician, depending on the health plan.

"It is easy to show that things were getting better from point-in-time A to point-in-time B," said Steven D. Pearson, MD, the study's lead author and president of Harvard Medical School's Institute for Clinical and Economic Review. "Unless you look at a control group of some kind, you may be misled about what's really happening."

Some P4P efforts have shown results. A Centers for Medicare & Medicaid Services evaluation of its Physician Group Practice Demonstration found that all 10 participating practices hit or exceeded targets on at least seven of 10 quality metrics of diabetes care. A Feb. 1, 2007, New England Journal of Medicine study found that Medicare's pay-for-performance demonstration project was associated with a modest improvement on quality metrics, compared with hospitals not in the project.

Another study that examined the CMS hospital P4P data wasn't as positive. A June 6, 2007, Journal of the American Medical Association study of heart attack care found no significant improvement for 54 hospitals in the CMS P4P demonstration, compared with 446 nonparticipating hospitals. A systematic review of 17 studies, published in the Aug. 15, 2006, Annals of Internal Medicine, found positive or partially positive effects of P4P programs, but the impact was usually small.

Meanwhile, an Oct. 12, 2005, JAMA study of PacifiCare Health Systems' pay-for-performance program found that California physician groups did better on only one measure -- cervical cancer screening -- than a comparison group of Oregon doctors.

"If you were around and read those opinion pieces that came out in 2001 and 2002, there was excessive exuberance about how pay-for-performance was going to solve everything," said Meredith B. Rosenthal, PhD, lead author of the California study and an associate professor of health economics and policy at the Harvard School of Public Health."We actually have remarkably few evaluations that have a comparison group of any kind, so the evidence on pay-for-performance is rather spotty. The programs we've evaluated over the last five years have been largely unimpressive in their results."

Rosenthal cautioned that studies finding little quality impact so far "have nothing to do with overturning the theory of pay-for-performance. It can be explained by the way in which these programs have been implemented, and at the top of the list is that the size of the bonus is too small."

Maximum quality incentives average 9% of plan payments, according to a Nov. 2, 2006, New England Journal of Medicine study by Rosenthal, but most physicians average less than a 5% bonus. By contrast, under a British P4P plan that was rolled out in 2004, physicians can more than double their income by achieving high scores on 149 quality indicators.

Francois de Brantes, CEO of Bridges to Excellence, an employer-funded P4P program, said BTE's effort in the last five years has paid $12 million in incentives to physicians in 18 states. He said the program has helped drive quality improvement but said success "takes a strong willingness of health plans to collaborate, and that, unfortunately, has not been a very popular notion."

Toward payment reform

The biggest nongovernmental pay-for-performance program is operated by California's Integrated Healthcare Assn., a collaboration of health plans, physician groups and hospitals.

Last year, 40,000 physicians participated, and IHA paid $65 million in bonuses -- about 2% of overall reimbursement. Unlike most bonus programs, IHA uses an integrated set of measures and aggregates data from all payers, meaning physicians have only one set of metrics to target for improvement.

IHA Executive Director Tom Williams said an analysis being prepared for peer-review publication shows "there appeared to be some effect" of the organization's bonus payments. "It wasn't strong."

He added that "pay-for-performance is not the end solution" but part of "the foundation for more payment reform, which is really what we're driving toward."

Along these lines, Congress has required the Health and Human Services secretary to deliver proposed payment reforms by May 2010. Pay-for-performance likely will be one element of the proposal, said Michael Rapp, MD, director of the CMS Quality Measurement and Health Assessment Group.

In July, the agency reported that in 2007 more than 109,000 physicians took part in Medicare's Physician Quality Reporting Initiative, which paid $36 million in bonuses to 56,700 doctors -- an average of $600 per doctor. Starting next year, physicians could earn up to 4% of Medicare reimbursements by reporting quality metrics and implementing electronic prescribing systems.

In 2005, the AMA adopted principles stating that pay-for-performance programs should be voluntary, not reduce reimbursement, and use physician-developed quality metrics, accurate data and scientifically valid analytical methodology. The AMA-convened Physician Consortium for Performance Improvement has developed 215 performance measures in clinical areas ranging from diabetes to substance abuse.

Back to top


Incentives and performance

Physician bonuses of $1,000 or more per performance measure produced statistically insignificant differences between Massachusetts medical groups eligible for the incentives and those that were not. Data show the percentage of appropriate patients for whom the care was provided.

2001 2003
Breast cancer screening
Incentivized groups 82% 82%
Comparison groups 83% 84%
Diabetes care -- eye exams
Incentivized groups 51% 54%
Comparison groups 52% 56%
Diabetes care -- HbA1c testing
Incentivized groups 81% 85%
Comparison groups 81% 87%
Diabetes care -- LDL-C screening
Incentivized groups 79% 88%
Comparison groups 80% 89%
Diabetic nephropathy monitoring
Incentivized groups 39% 48%
Comparison groups 40% 52%
Well-child visits, ages 3-6
Incentivized groups 81% 86%
Comparison groups 87% 90%

Source: "The Impact of Pay-for-Performance On Health Care Quality In Massachusetts, 2001-2003," Health Affairs, July/August

Back to top

External links

"The Impact Of Pay-for-Performance On Health Care Quality In Massachusetts, 2001-2003," Health Affairs, July/August (link)

AMA-convened Physician Consortium for Performance Improvement (link)

Back to top



Read story

Confronting bias against obese patients

Medical educators are starting to raise awareness about how weight-related stigma can impair patient-physician communication and the treatment of obesity. Read story

Read story


American Medical News is ceasing publication after 55 years of serving physicians by keeping them informed of their rapidly changing profession. Read story

Read story

Policing medical practice employees after work

Doctors can try to regulate staff actions outside the office, but they must watch what they try to stamp out and how they do it. Read story

Read story

Diabetes prevention: Set on a course for lifestyle change

The YMCA's evidence-based program is helping prediabetic patients eat right, get active and lose weight. Read story

Read story

Medicaid's muddled preventive care picture

The health system reform law promises no-cost coverage of a lengthy list of screenings and other prevention services, but some beneficiaries still might miss out. Read story

Read story

How to get tax breaks for your medical practice

Federal, state and local governments offer doctors incentives because practices are recognized as economic engines. But physicians must know how and where to find them. Read story

Read story

Advance pay ACOs: A down payment on Medicare's future

Accountable care organizations that pay doctors up-front bring practice improvements, but it's unclear yet if program actuaries will see a return on investment. Read story

Read story

Physician liability: Your team, your legal risk

When health care team members drop the ball, it's often doctors who end up in court. How can physicians improve such care and avoid risks? Read story

  • Stay informed
  • Twitter
  • Facebook
  • RSS
  • LinkedIn