Completed Systematic Reviews Worksheet for Evidence-Based Mental Health
Lima M, Moncrieff J. Drugs versus placebo for dysthymia (Cochrane Review). In: The Cochrane Library, Issue 1, 1999. Oxford: Update Software.
Are the results of this systematic review of therapy valid?
Is it a systematic review of randomised trials of the treatment you're interested in?
Does it include a methods section that describes finding and including all the relevant trials?
Yes. The authors searched several electronic databases, and attempted to locate unpublished trials.
Does it include a methods section that describes assessing their individual validity?
Yes. The authors used two methods to assess this.
Were the results consistent from study to study?
Yes. Consistent results were obtained for studies making each comparison (such as tricyclics and placebo, or MAOIs and placebo).
Are the valid results of this systematic review important?
It is necessary to be able to understand the size of the effect of treatments. A study may show a statistically significant advantage of one treatment over another. However, the magnitude of this effect may not be clinically significant. Odds ratios (ORs) and numbers needed to treat (NNTs) help us to understand the size of this effect.
The systematic review presents the results in terms of odds ratios. These may be translated into NNTs, by using the following table. The numbers in the body of the table are the NNTs for the corresponding odds ratios at that particular control event rate (or patient's expected event rate, PEER).
|Patient's Expected Event Rate (PEER)||0.05||2091||139||104||83||69||59||52||46||412||34||29||26|
|Patient's Expected Event Rate (PEER)||0.05||212||86||44||30||23||18||16||14||12||10||9||7|
Can you apply this valid, important evidence from a systematic review in caring for your patient?
Do these results apply to your patient?
Is your patient so different from those in the systematic review that its results can't help you?
No. The patient appears similar.
How great would the potential benefit of therapy actually be for your individual patient?
In the table on page 1, find the intersection of the closest odds ratio from the overview and the CER that is closest to your patient's expected event rate if they received the control treatment (PEER):
Odds ratio (no treatment response, TCA versus placebo)
Risk to your patients similar to those in trial, therefore PEER @ CER
= 208 / 305
Closest intersection gives NNT less than 6 (as intersection to the right of OR = 0.5).
To calculate the NNT for any OR and PEER:
rounded up to nearest whole number
See below for a discussion of the relevance of confidence intervals around this NNT, and description of how it may be calculated.
To calculate the NNT from the original numbers in the studies. Calculate the ARR from the CER and EER. The NNT is the reciprocal of the ARR (i.e. 1/ARR).
If the risk of the outcome is different in your patient, divide the NNT by F, which is the risk of the outcome in your patient relative to the patients in the review (or trial), expressed as a decimal.
CER = 208 / 305 = 0.6820
EER = 133 / 295 = 0.4508
ARR = 0.6820 - 0.4508 = 0.2312
NNT = 1 / ARR = 1 / 0.23 = 4.3
rounded up to nearest whole number = 5
If risk higher, F = say 1.2
NNT = 4.3 / 1.2 = 3.6 (rounded to 4)
Are your patient's values and preferences satisfied by the regimen and its consequences?
Do your patient and you have a clear assessment of their values and preferences?
Needs to be assessed in each patient
Are they met by this regimen and its consequences?
Needs to be assessed in each patient
Should you believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients?
Only if you can say "yes" to all of the following:
- Do they really make biologic and clinical sense?
- Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant?
- Was this difference hypothesised before the study began (rather than the product of dredging the data), and has it been confirmed in other, independent studies?
- Was this one of just a few subgroup analyses carried out in this study?
Confidence intervals around NNTs:
95% confidence interval (CI) of an NNT = 1 / limits of the CI of the absolute risk reduction (ARR)
The 95% CIs for the ARR will be 1.96 x standard error (SE) each side of the point estimate of the ARR.
1.96 x 0.039 = 0.076
Therefore 95% CIs for ARR are 0.231 +/- 0.076, i.e. 0.307 and 0.155.
The NNT is the reciprocal of the ARR. Therefore, the NNT is 4.3, with 95% CIs of 3.3 and 6.5. These figures are given in the abstract at the beginning of the review.
This means that it is 95% probable that the real treatment effect lies within an NNT of 3.3. and 6.5.
Note that during this calculation, as we are subtracting small numbers from small numbers, it is best to work to several decimal places, otherwise the result becomes inaccurate.