KT Clearinghouse

Knowledge to Action

Login

Welcome

Signup

 

CEBM

Were the results consistent from study to study?

Did all the studies show results in the same direction? The results shouldn't be exactly the same for all studies but we should be concerned if some studies showed a significant beneficial effect and others significantly excluded any benefit or showed a significant harmful effect. The investigators should test to determine if this lack of consistency (or heterogeneity) was statistically significant. If indeed it was significant, we would like to see that they have explored the reasons why this may have been observed.

The authors of the Cochrane review tested for heterogeneity and it was not significant.

  1. Is this a systematic review of randomised trials?
  2. Does it include a methods section that describes: a) finding and including all the relevant trials, and b) assessing their individual validity?
  3. Were the results consistent from study to study?

    A finer point:
  4. Were individual patient data (or aggregate data) used in the analysis?