Just Words? You Say Efficacy, And I Say Effectiveness

As anyone will tell you, different words can mean different things to different people. And so it is with government healthcare outcomes experts, as well. Consider efficacy and effectiveness, which are often used interchangeably to the possible detriment of those who rely on reimbursement decisions, according to a new analysis.

To be clear, effectiveness is used to describe drugs in real-world circumstances where the patient population and other variables cannot be controlled. By contrast, efficacy is used to describe how a drug performs in an ideal or controlled setting, such as a clinical trial. Yet some confusion apparently exists when it comes to using the correct word.

Of 38 health technology assessments conducted between 2005 and 2011 by agencies in English-speaking countries, 20 indicated they were evaluating effectiveness. Of those, however, only one actually evaluated effectiveness. Conversely, 17 stated they were measuring efficacy and did so, according to the analysis by Context Matters, a data analytics firm.

The agencies stated they measure efficacy and effectiveness at similar rates - 45 percent and 55 percent, respectively. But the agencies measure effectiveness significantly less frequently - just 3 percent of the time, according to the analysis, which was presented this week at the International Society of Pharmacoeconomics and Outcomes Research conference in Berlin (see this).

Why should this matter? The firm notes that reimbursement decisions are based on HTAs, but the analysis demonstrated that effectiveness is often used when the evidence presented for consideration is, instead, based on clinical trial scenarios rather than real-world circumstances.

"This is an obvious barrier to clear communication, but the implications might be broader," the firm wrote in its poster presentation. "If this is an indication of a more widespread misuse of critical terminology, then researchers, policymakers, and other readers may be misunderstanding the implications of comparative effectiveness and HTA reviews" (see more here).

"The terms confer a different level of rigor. This may appear to be semantics, but there's something imprecise about it. If you interchange these two terms, which mean different things for making reimbursement decisions, it really doesn't make a lot of sense," says Context Matters ceo Yin Ho and co-author of the study, which was presented this week at the International Society of Pharmacoeconomics and Outcomes Research conference in Berlin.

"People are trying to justify spending dollars and regulators are trying to decide whether drugs should be on the market," she tells us. "If people are non-chalant about the use of these terms, it can create a misleading perspective. The determinations have large consequences and calls into question their entire methodology... It's partially out of ignorance and partially out of sloppiness and we're not sure why. But it's more than just a blip."

The firm analyzed reviews made by the Canadian Agency for Drugs and Technologies in Health; the UK's National Institute for Health and Clinical Excellence; the Agency for Healthcare Research and Quality in the US; Germany's IQWIG, the Institute for Quality and Efficiency in Healthcare; Healthcare Improvement Scotland, which is a division of the UK's National Health Service; and the Drug Effectiveness Review Project, which is based at Oregon Health & Science University.