In this edition of #Nephstats, we look at Number Needed to Treat (NNT), a controversial topic creates ripples and roars on social media amongst stat savvy physicians, epidemiologists and biostatisticians. This seems like such a simple and easy number to make sense of a trial’s importance - what could be controversial or wrong about it?
NNT was first described by Laupacis et al. in a 1988 paper on clinically useful measures on the effectiveness of treatments. It was created to help people understand and explain the efficacy of an intervention in concrete and relatable terms. The NNT is defined as the number of patients that need to be treated for one patient to benefit- i.e to get a positive outcome or to avoid an adverse outcome. Mathematically it is the inverse of Absolute risk reduction (ARR) and expressed as 1/ARR.
NNT is a measure to explain how much effort must be expended to prevent one event with the intervention or drug in question. However, over the years, NNT has been misused, misinterpreted and often times quoted as an isolated entity to prove efficacy of new drugs on the market.
NNT and the hidden figures
Let's take a look at the landmark clinical trial, CREDENCE. Study investigators estimated that among 1000 patients in the trial treated for 2.5 years, the primary composite outcome of ESKD, doubling of the serum creatinine level, or renal or cardiovascular death occured in 47 fewer patients in the canagliflozin group than in the placebo group (NNT 22; 95% CI, 15 to 38).
This means that if 22 patients were treated with canagliflozin for 2.5 years, one fewer patient would develop the primary end point than if those same patients were treated with placebo.
A common misinterpretation of NNT is that “1 out of every 22 patients treated with canagliflozin benefitted from the treatment and the other 21 had the adverse outcome” This is obviously not the case, so what happened to the other 21? A disadvantage of NNT is that it tells us nothing about the baseline risk in the population studied nor does it say anything about how much risk remains despite the intervention. An NNT of 22 implies that the 21 other patients either did not need the drug or did not benefit from the drug and it does not tell us what are the characteristics of that 1 patient which benefited from the drug.
The NNT is derived from the portion of the sample/population which benefits from an effective drug or intervention. But it does not reflect the portion of the population which does not benefit from the drug, is possibly harmed from the drug, or simply does not need the drug.
ARR and NNT tell us about the impact of an intervention, but do not tell us the whole story. They do not tell us how much risk remains in the intervention group. This is where the relative risk becomes hugely important.
NNT as a time dependent measure
Prof Darrel Francis beautifully unravels how ARR and NNT vary across the time course. Check out the tweetorial here created as a moment by Swap.
Even if the relative risk reduction from a long-term therapy is constant over time, the NNT will decrease with increasing follow-up as adverse events accrue with time and the absolute risk rises. The NNT defined for a certain time period of say 5 years cannot be extrapolated to a different time period of 10 years or 20 years.
NNT is an outcome specific measure
Most interventions affect more than one outcome, for example, Canagliflozin impacted renal and cardiovascular deaths as well as progression to ESKD. Hence the investigators in the CREDENCE trial have reported a different NNT for each of these outcomes as NNT for one outcome cannot be directly compared to another one even if the intervention remains the same.
Since NNT is specific to the outcome of interest in the specified time period, for the defined population which is exposed to the risk and the intervention, it may not be always applicable to more generalized patient population who may have a completely different baseline risk of disease or its complication. Suppose the baseline risk for a particular disease in one population is 30% and the intervention reduces the risk to 20% (ARR 10% and RRR of 33% and NNT 10). Now if this intervention which has a relative risk reduction of 33% is applied to a different population with a much lower baseline risk of 10%, the absolute risk in the intervention group will be 6.7% (ARR 3.3%, NNT 30). Thus lower the baseline risk, the higher the NNT. This shows how the baseline risk influences the NNT and the reason why NNT should always be interpreted in the context of the relative risk and the absolute risk reduction and never in isolation.
[Figure adapted from https://www.youtube.com/watch?v=QPXXTE8N4PY&t=418s]
NNT is not an effective measure of clinical efficacy of treatments for chronic stable conditions such as statins for CAD.
It can be used effectively to understand the effectiveness of interventions which lead to a definite end point, such as surgical procedures or their complications, in other words things that happen or don’t happen. Other examples of situations where NNT can effectively relay the usefulness of the intervention are assessing whether a particular vaccine is able to prevent a communicable disease or whether a preventive screening test is able to help in early diagnosis of a certain cancer.
In the above example, is it more effective communication when you say “ we needed to treat 10 patients, to prevent 1 bad outcome”? Or this can be simply stated as “ there were 10% fewer deaths with intervention” (ARR) or “the intervention reduced the risk by 33%” (RRR). NNT takes the results and flips it to the treater perspective. Despite being designed for effective communication of the result of an intervention in a study, systematic reviews have shown that NNT is one of the hardest statistic to understand effect size and may need more cognitive gymnastics to understand what it really means. (Read @hildabast post on the NNT hype here).
These are a few of the many challenges we face in effective communication of research, and as Hilda Bastian rightly says in her post, “be on full alert” when it comes to methods used and interpretation of results of new studies.
References-
End Games: What is the number needed to treat (NNT)? BMJ 2013;347:f4605 doi: 10.1136/bmj.f4605 (Published 24 July 2013)
On the clinical meaningfulness of a treatment’s effect on a time-to-event Variable. Statist. Med. 2011, 30 2341–2348