P4P found to have little impact on care quality
■ A new study finds that performance bonuses are not big enough to yield significant improvements. Some say more reform is necessary to reward recommended care.
By Kevin B. O’Reilly — Posted Aug. 4, 2008
The idea of paying physicians more for providing guideline-based care has taken the American health system by storm in the last decade. Today, more than 150 pay-for-performance programs are centered on the notion that rewarding evidence-based care is key to improving health care quality.
But research shows there is a fundamental problem with the P4P programs: They have had little to no impact on quality. That is the conclusion of many studies, including a new analysis of quality incentives, published in the July/August Health Affairs. It compared 81 Massachusetts physician groups eligible for quality incentives with 73 that were not.
Unlike earlier studies, researchers had access to data from a cross section of doctors and payers -- five health plans that contract with more than 90% of the state's practicing primary care doctors. The study found that overall performance from 2001 to 2003 improved on 73% of preventive care measures such as diabetes hemoglobin A1c testing, breast cancer screening and well-child visits.
But the performance of the 5,350 physicians analyzed was statistically indistinguishable. Everyone's quality improved, regardless of whether the physician group stood to earn a bonus, which ranged from $200 to $2,500 per quality measure for an individual physician, depending on the health plan.
"It is easy to show that things were getting better from point-in-time A to point-in-time B," said Steven D. Pearson, MD, the study's lead author and president of Harvard Medical School's Institute for Clinical and Economic Review. "Unless you look at a control group of some kind, you may be misled about what's really happening."
Some P4P efforts have shown results. A Centers for Medicare & Medicaid Services evaluation of its Physician Group Practice Demonstration found that all 10 participating practices hit or exceeded targets on at least seven of 10 quality metrics of diabetes care. A Feb. 1, 2007, New England Journal of Medicine study found that Medicare's pay-for-performance demonstration project was associated with a modest improvement on quality metrics, compared with hospitals not in the project.
Another study that examined the CMS hospital P4P data wasn't as positive. A June 6, 2007, Journal of the American Medical Association study of heart attack care found no significant improvement for 54 hospitals in the CMS P4P demonstration, compared with 446 nonparticipating hospitals. A systematic review of 17 studies, published in the Aug. 15, 2006, Annals of Internal Medicine, found positive or partially positive effects of P4P programs, but the impact was usually small.
Meanwhile, an Oct. 12, 2005, JAMA study of PacifiCare Health Systems' pay-for-performance program found that California physician groups did better on only one measure -- cervical cancer screening -- than a comparison group of Oregon doctors.
"If you were around and read those opinion pieces that came out in 2001 and 2002, there was excessive exuberance about how pay-for-performance was going to solve everything," said Meredith B. Rosenthal, PhD, lead author of the California study and an associate professor of health economics and policy at the Harvard School of Public Health."We actually have remarkably few evaluations that have a comparison group of any kind, so the evidence on pay-for-performance is rather spotty. The programs we've evaluated over the last five years have been largely unimpressive in their results."
Rosenthal cautioned that studies finding little quality impact so far "have nothing to do with overturning the theory of pay-for-performance. It can be explained by the way in which these programs have been implemented, and at the top of the list is that the size of the bonus is too small."
Maximum quality incentives average 9% of plan payments, according to a Nov. 2, 2006, New England Journal of Medicine study by Rosenthal, but most physicians average less than a 5% bonus. By contrast, under a British P4P plan that was rolled out in 2004, physicians can more than double their income by achieving high scores on 149 quality indicators.
Francois de Brantes, CEO of Bridges to Excellence, an employer-funded P4P program, said BTE's effort in the last five years has paid $12 million in incentives to physicians in 18 states. He said the program has helped drive quality improvement but said success "takes a strong willingness of health plans to collaborate, and that, unfortunately, has not been a very popular notion."
Toward payment reform
The biggest nongovernmental pay-for-performance program is operated by California's Integrated Healthcare Assn., a collaboration of health plans, physician groups and hospitals.
Last year, 40,000 physicians participated, and IHA paid $65 million in bonuses -- about 2% of overall reimbursement. Unlike most bonus programs, IHA uses an integrated set of measures and aggregates data from all payers, meaning physicians have only one set of metrics to target for improvement.
IHA Executive Director Tom Williams said an analysis being prepared for peer-review publication shows "there appeared to be some effect" of the organization's bonus payments. "It wasn't strong."
He added that "pay-for-performance is not the end solution" but part of "the foundation for more payment reform, which is really what we're driving toward."
Along these lines, Congress has required the Health and Human Services secretary to deliver proposed payment reforms by May 2010. Pay-for-performance likely will be one element of the proposal, said Michael Rapp, MD, director of the CMS Quality Measurement and Health Assessment Group.
In July, the agency reported that in 2007 more than 109,000 physicians took part in Medicare's Physician Quality Reporting Initiative, which paid $36 million in bonuses to 56,700 doctors -- an average of $600 per doctor. Starting next year, physicians could earn up to 4% of Medicare reimbursements by reporting quality metrics and implementing electronic prescribing systems.
In 2005, the AMA adopted principles stating that pay-for-performance programs should be voluntary, not reduce reimbursement, and use physician-developed quality metrics, accurate data and scientifically valid analytical methodology. The AMA-convened Physician Consortium for Performance Improvement has developed 215 performance measures in clinical areas ranging from diabetes to substance abuse.