Does it include a methods section that describes finding and including all the relevant trials and assessing their individual validity?
We need to review the study to determine how the authors identified the studies and if this process was sufficiently complete. For example, we'd like to see that they contacted authors looking for nonpublished studies, checked bibliographies of all the articles retrieved, handsearched journals and included articles from all languages and from drug company databases. Negative studies are less likely to get published and therefore if these are not searched for, it could lead to a false positive conclusion.
We'd also want to see that the study includes information on how individual studies were appraised for validity. We'd also be reassured if multiple reviews were done and if there was good agreement amongst the reviewers.
In the Cochrane review we identified, the authors searched the Cochrane trials registry, the reference lists of all articles and contacted researchers to identify published and nonpublished studies. Two reviewers independently assessed the quality of the trials and determined their eligibility for inclusion in the review. Although a formal scoring system was not used to record methodological quality, the authors did record the method of allocation concealment, the duration of follow-up, whether or not an intention to treat analysis was done and whether theirs was a blinded assessment of outcomes.
- Is this a systematic review of randomised trials?
- Does it include a methods section that describes: a) finding and including all the relevant trials, and b) assessing their individual validity?
Were the results consistent from study to study?
A finer point:
- Were individual patient data (or aggregate data) used in the analysis?