Researchers press for risk-stratified analysis of clinical trial data
■ Current practice determines the average effect for the population at large but often fails to take into account patient variability.
By Victoria Stagg Elliott — Posted Dec. 12, 2005
Rodney Hayward, MD, professor of medicine and public health at the University of Michigan Medical School, Ann Arbor, doesn't always feel like he has the information he needs to tailor prescribing to his patients, even when he can draw on the knowledge produced by large randomized trials.
He knows, for instance, that hormone therapy can increase the risk of heart disease in the population of older women as a whole, but what he wants to know is how it affects the risk of an individual patient based on her other cardiovascular characteristics.
"If I have a 50-year-old in my office and she has a lot of hot flashes, I can't tell what her risk for a [hormone therapy-related] adverse event is," said Dr. Hayward, who is also the director of the VA Center for Practice Management and Outcomes Research in Ann Arbor. "All we have are the average results, and they may or may not apply. The harm may be very, very small in some individuals."
It is this kind of frustration, coupled with recent controversies surrounding adverse events that emerge long after a drug hits the market, that has researchers looking for new ways to make clinical trial data more individually applicable. They are increasingly recognizing that risk and benefit are far from evenly distributed, and they hope to develop tools to determine who is more likely to be harmed or benefited by a particular drug.
"Clinical trials are basically done to determine whether a drug works in groups of people and the answer applies to the group," said Curt Furberg, MD, PhD, a former member of the Food and Drug Administration's Drug Safety & Risk Management Advisory Committee and professor of public health sciences at Wake Forest University School of Medicine in Winston-Salem, N.C. "But people are different, and their responses to drugs are different."
A recent paper authored by Dr. Hayward and published in the November/December Health Affairs urged researchers to put statistics through a risk-stratified analysis in an effort to gain this better understanding.
"If a small group of patients gets a very large amount of benefit, the treatment can have average benefit even when most people in the trial received no benefit and some are harmed," said Dr. Hayward. "The current way medical treatments are evaluated is inadequate to detect this problem; this problem is not rare, and mandating risk-stratified analysis has the potential to decrease the risk of promoting wasteful and unsafe treatments."
To a very limited extent this stratification is already done. Dr. Hayward's paper found that 4% of studies use risk-stratified analysis, and the FDA is increasingly asking for such information. The FDA required it for the 2001 approval of Xigris, or drotrecogin alfa (activated), a genetically engineered human protein used to treat sepsis. It has become a poster child for this type of approach. When it was originally studied, the treatment was shown to reduce overall mortality by 6%. As scientists dug into the data, though, it became clear that those at high risk of death got a lot of benefit from the drug while those who weren't quite as ill didn't get any.
"We are interested in this," said Robert Temple, MD, associate director for medical policy at the FDA's Center for Drug Evaluation and Research. "The question is how to do it right."
Those on the drug industry side also expressed an interest in getting to the heart of whom a drug might help versus hurt.
"Everybody is trying to find better ways to stratify patients, and understand risk and benefit," said Alan Goldhammer, PhD, associate vice president of regulatory affairs at Pharmaceutical Research and Manufacturers of America.
Experts said that while there is a lot of interest in statistical methods that may provide more individualized information, there's also a need for caution. Any kind of subgroup analysis can be full of pitfalls caused by a lack of statistical power, and such an approach can develop new directions for research but rarely answer a question definitively.
"The idea of doing subgroup analysis is not novel, but it has to be done very carefully," said Brian Strom, MD, MPH, professor and chair of Dept. of Biostatistics and Epidemiology at the University of Pennsylvania School of Medicine in Philadelphia. "There are a lot of opportunities for false signals."
Better information for physicians
Advocates of the risk-stratification approach say, though, that this can work, but it may require more statistical finesse than scientists are used to. They are particularly hopeful about the potential findings.
"Risk stratification of study results will provide better information to help doctors talk with their patients about whether or not a certain treatment is right for them, considering their individual circumstances," said Dr. Hayward.
Experts said this was a desirable goal but also expressed concern about how to communicate the even more complicated information to doctors.
A study published online in Pharmacoepidemiology and Drug Safety last month found that compliance with black-box warnings varied widely depending on the nature of the information.
"This is a fundamental tool for the FDA, but we're asking too much of the black-box warning," said Richard Platt, MD, study leader and professor and chair of the Dept. of Ambulatory Care and Prevention at Harvard Medical School/Harvard Pilgrim Healthcare in Boston. "For some kinds of information it's excellent, but for some kinds it clearly isn't working."