Are the results of this study valid?
Evidence about harm can come from a number of different study types. Ideally we'd like to see a high quality systematic review of randomised trials but these aren't easy to find because RCTs aren't always feasible for issues of harm. As a result, we usually find evidence about harm in cohort studies (groups of patients who are and aren't exposed to the treatment are followed up for the outcome of interest) and case-control studies (patients with the outcome of interest are matched with patients without the outcome and investigators look retrospectively to determine exposure). Case-control studies are useful when the outcome of interest is rare or when the required follow-up is long. The strength of inference that can be drawn from a case-control study is limited because they are more susceptible to bias.
Returning to our clinical scenario from the question formulation tutorial:
You see a 50 year old man who asks for a repeat prescription of sotalol which he has been taking for extrasystoles for several years. He has a remote history of an MI. You haven't seen him previously and are concerned about the proarrhythmic properties of sotalol given what is known about other antiarrhythmics.
During the tutorial on clinical questions we formulated the question: In a man with extrasystoles and a remote history of MI, does treatment with sotalol increase his risk of death?
How do we critically appraise this harm paper? We'll start off by considering validity first and the following list outlines the questions that we need to consider when deciding if a harm paper is valid.
- Were there clearly defined groups of patients, similar in all important ways other than exposure to the treatment or other cause?
- Were treatments/exposures and clinical outcomes measured in the same ways in both groups? (Was the assessment of outcomes either objective or blinded to exposure?)
- Was the follow-up of the study patients sufficiently long (for the outcome to occur and complete)?
Do the results of the harm study fulfil some of the diagnostic tests for causation?
- Is it clear that the exposure preceded the onset of the outcome?
- Is there a dose-response gradient?
- Is there any positive evidence from a 'dechallenge-rechallenge' study?
- Is the association consistent from study to study?
- Does the association make biological sense?