Despite warning, some hospitals get harsh patient safety grade

An updated scorecard from the Leapfrog Group has hospitals crying foul, with some big names getting poor marks.

By — Posted Dec. 31, 2012

Print  |   Email  |   Respond  |   Reprints  |   Like Facebook  |   Share Twitter  |   Tweet Linkedin

When the employer-backed Leapfrog Group handed out grades of A, B or C on patient safety to individual hospitals in June, 5% of hospitals scored low enough to receive D's or F's but were marked as “grade pending.” The hospitals had better shape up by the end of the year, Leapfrog said, or they will get a D or F.

Now the group has kept its promise, handing out the low marks to 146 U.S. hospitals, including highly regarded facilities such as Cleveland Clinic and the Ronald Reagan UCLA Medical Center in Los Angeles.

The criticism from hospital officials that greeted the initial round of Leapfrog grades has escalated now that D's and F's are on the report card. But Leapfrog's leaders are standing firm behind their scorecard's methodology, arguing that the grades are needed to help patients decide where to seek care and to spur dramatic hospital improvement on patient safety.

The Leapfrog grades, released Nov. 28, are based on 26 measures of patient safety and drawn from publicly available Medicare data for 2,618 general acute care hospitals. When applicable, grades also were based on hospitals' answers to the organization's patient safety survey or an American Hospital Assn. survey. Leapfrog's stated goal is to use their employer members' health purchasing power to influence quality and affordability.

Metrics cover areas such as intensive care unit physician staffing, medication reconciliation, use of computerized physician order entry, hand hygiene, antibiotic administration, catheter removal and a few “never events,” including air embolisms and serious pressure ulcers. A bad grade does not necessarily reflect the overall quality of care at a hospital, but does mean it is performing poorly on these safety metrics, Leapfrog officials said.

Hospitals without enough information available on at least 14 of the 26 measures do not receive a score and are not included in the report card. The formulation of the grading process was conducted by a nine-member panel that includes some of the most prominent names in the world of patient safety research, such as Harvard's Lucian Leape, MD.

A majority of hospitals got A's or B's, but 38% were given C's, nearly 5% got a D and fewer than 1% received a failing grade. The main Ohio campus of Cleveland Clinic — praised by President Obama during an October presidential debate as “one of the best health care systems in the world” and a model for how health care should be delivered — received a D. The Ronald Reagan UCLA Medical Center also has a sterling reputation, yet got an F from Leapfrog.

Hospitals question methodology

Hospital officials said Leapfrog's use of publicly available performance data in grading is problematic, because they capture only a slice of the care happening in hospitals, with grades derived from information collected as long ago as July 2009. Many also claim that the nature of the grading system gives preferential treatment to hospitals that answered Leapfrog's patient safety survey, compared with those that answered similar care questions on the AHA survey.

“Cleveland Clinic, like an increasing number of hospitals across the country, has declined to participate in the Leapfrog survey for the past two years,” the health system said in a statement. “We have not found this survey useful in prioritizing our patient safety efforts. Due to its methodology and limitations associated with the use of publicly available data, the Leapfrog report does not accurately reflect Cleveland Clinic's safety program or the many safety improvements throughout our system in recent years.”

In a statement, the Ronald Reagan UCLA Medical Center raised similar concerns, adding: “The problem with nearly all of the hospital report cards promoted by various organizations is that there is no consistency. The same hospital can rank high on one report card and low on another report card.”

Nancy Foster, the AHA's vice president of quality and patient safety policy, said improvements have been made in the Leapfrog methodology since the June report card, but there are still problems. It is possible that hospitals may look better or worse depending on the variability in their answers on the Leapfrog survey, as opposed to their actual safety records.

Research shows that hospital report cards are inconsistent in evaluations and fail to improve quality or safety.

Meanwhile, hospitals that did not use the Leapfrog survey seemed to get penalized. Those answering a Leapfrog survey question about implementation of CPOE were eligible for up to 100 points, but hospitals answering a similar question on the AHA survey could earn only 65 points, Foster said.

“We would prefer that it be an even playing field,” she said. That highly regarded hospitals such as Cleveland Clinic got poor grades should cause concern about Leapfrog's methodology, Foster added.

“It does make one question whether the data that's being used is in fact as accurate as it should be,” she said. “It's that sort of face-validity question.”

Previous research has found that hospital report cards are inconsistent in how they evaluate care, are hard for patients to understand and use, and fail to demonstrably improve quality or safety.

Grading will continue

Yet Leapfrog is not backing down, said Leah Binder, its president and CEO.

“The expert panel will reconvene in the new year, and they'll just continue on a regular basis to review the methodology for calculating the hospital safety score, and we will continue to be putting out grades for all the general hospitals,” she said. “Our board is 100% committed to this, and our expert panel is behind it, and we're full speed ahead.”

Hospitals unhappy with their grades should inspect their scores and implement quality improvements, Binder said.

“We've used publicly available, well-vetted, focused safety measures, and we've made it 100% transparent so that hospitals can go in themselves and see how did they get that C,” she said. “We're not selling magazines, and this is not a one-time deal. This is about an ongoing way of improving the performance and safety of our hospitals. I think that's caught their attention.”

Fifty-eight percent of hospitals got the same grade in November as they got in June, while 34% saw their grades change up or down by one level. An additional 8% saw them change by two or more grade levels. The Leapfrog grades will be handed out annually from now on, the organization said.

In December, Leapfrog announced a consulting arrangement with Johns Hopkins' Armstrong Institute for Patient Safety and Quality in Baltimore. The Leapfrog grades are not the final word on hospital safety, said Matt Austin, PhD, an instructor at the institute, who formerly directed Leapfrog's hospital survey.

“This should have a physician at least double-check him- or herself to say, 'Oh, am I admitting patients to a hospital that maybe doesn't have the best patient safety record? Should I be working with hospital administrators to see what they're doing to improve safety?' ” said Austin, an expert on quality measurement. “To me, these grades are a conversation starter about patient safety, a way for doctors to talk to hospitals where they have patients, and a way for patients to talk with doctors.”

Back to top


How one group rates hospitals on patient safety

The employer-backed Leapfrog Group recently released a report that gave 2,618 hospitals letter grades based on their performances on 26 patient safety measures. The low grades are meant to inspire hospitals to improve, and to caution patients about those facilities.

Grade No. of hospitals % of hospitals receiving grades
A 790 30.2%
B 678 25.9%
C 1,004 38.3%
D 121 4.6%
F 25 0.9%

Note: Percentages do not add up to 100 due to rounding.

Source: Update to the Hospital Safety Score by the Leapfrog Group, Nov. 28 (link)

Back to top

External links

The Leapfrog Group's searchable Hospital Safety Score website, updated Nov. 28 (link)

Explanation of methodology used in Leapfrog's Hospital Safety Score, November (link)

Update to the Hospital Safety Score by the Leapfrog Group, Nov. 28 (link)

Back to top



Read story

Confronting bias against obese patients

Medical educators are starting to raise awareness about how weight-related stigma can impair patient-physician communication and the treatment of obesity. Read story

Read story


American Medical News is ceasing publication after 55 years of serving physicians by keeping them informed of their rapidly changing profession. Read story

Read story

Policing medical practice employees after work

Doctors can try to regulate staff actions outside the office, but they must watch what they try to stamp out and how they do it. Read story

Read story

Diabetes prevention: Set on a course for lifestyle change

The YMCA's evidence-based program is helping prediabetic patients eat right, get active and lose weight. Read story

Read story

Medicaid's muddled preventive care picture

The health system reform law promises no-cost coverage of a lengthy list of screenings and other prevention services, but some beneficiaries still might miss out. Read story

Read story

How to get tax breaks for your medical practice

Federal, state and local governments offer doctors incentives because practices are recognized as economic engines. But physicians must know how and where to find them. Read story

Read story

Advance pay ACOs: A down payment on Medicare's future

Accountable care organizations that pay doctors up-front bring practice improvements, but it's unclear yet if program actuaries will see a return on investment. Read story

Read story

Physician liability: Your team, your legal risk

When health care team members drop the ball, it's often doctors who end up in court. How can physicians improve such care and avoid risks? Read story

  • Stay informed
  • Twitter
  • Facebook
  • RSS
  • LinkedIn