Completed Diagnosis Worksheet for Evidence-Based Purchasing
Whited JD, Hall RP, Simel DL, Horner RD. Primary care clinicians' performance for detecting actinic keratoses and skin cancer. Arch Intern Med 1997;157:985-990.
Are the results of this diagnostic study valid?
Was there an independent, blind comparison with a reference ("gold") standard of diagnosis?
Yes - of GP examinations versus those of dermatologists - considered a "pragmatic" gold standard. In a second stage, the classic gold standard, biopsy, was performed only on lesions thought malignant by the dermatologists.
Was the diagnostic test evaluated in an appropriate spectrum of patients (like those in whom it would be used in practice)?
Probably not. The prevalence in this group is quite high for British patients at 32%. Patients were recruited from dermatology and general internal medical out patients. Although neither group had been seen by a dermatologist for 4 months, and those from medical outpatients did not necessarily have a known skin condition, the dermatology patients did.
Was the reference standard applied regardless of the diagnostic test result?
Yes- when the dermatologist examination was the reference standard. All patients were examined both by a GP and a dermatologist.
No - when the reference standard was biopsy, to compare with the dermatologist examination. Biopsies were only performed on lesions thought to be malignant by the dermatologist.
Are the valid results of this diagnostic study important?
(skin cancer, according to dermatologist exam)
|Diagnostic Test Result
a + b
c + d
a + c
b + d
Sensitivity = a/(a+c)
Sensitivity = 35/61
Sensitivity = 57%
Specificity = d/(b+d)
Specificity = 114/129
Specificity = 88%
Likelihood Ratio for a positive test result = LR+
Likelihood Ratio for a positive test result = sens/(1-spec)
Likelihood Ratio for a positive test result = 57%/12%
Likelihood Ratio for a positive test result = 4.9 (rounding off at the final stage)
Likelihood Ratio for a negative test result = LR-
Likelihood Ratio for a negative test result = (1-sens)/spec
Likelihood Ratio for a negative test result = 43%/88%
Likelihood Ratio for a negative test result = 0.48 (rounding off at the final stage)
Positive Predictive Value = a/(a+b)
Positive Predictive Value = 35/50
Positive Predictive Value = 70%
Negative Predictive Value = d/(c+d)
Negative Predictive Value = 114/140
Negative Predictive Value = 81%
Pre-test Probability (prevalence) = (a+c)/(a+b+c+d)
Pre-test Probability (prevalence) = 61/190
Pre-test Probability (prevalence) = 32%
Pre-test-odds = prevalence/(1-prevalence)
Pre-test-odds = 32%/68%
Pre-test-odds = 0.47
Post-test odds = Pre-test odds x Likelihood Ratio for a positive result
Post-test odds = 0.47x4.9
Post-test odds = 2.3
Post-test Probability = Post-test odds/(Post-test odds + 1)
Post-test Probability = 2.3/3.3
Post-test Probability = 0.70
Can you apply this valid, important evidence about a diagnostic test in caring for your patient?
Is the diagnostic test available, affordable, accurate, and precise in your setting?
It is available and affordable. The GPs' lack of accuracy and precision would probably be worrying for most GPs and patients.
Can you generate a clinically sensible estimate of your patient's pre-test probability (from practice data, from personal experience, from the report itself, or from clinical speculation)?
Based upon Health Authority data, the population's pre-test probability is known to be about 5%.
Will the resulting post-test probabilities affect your management and help your patient? (Could it move you across a test-treatment threshold?; Would your patient be a willing partner in carrying it out?)
Using the nomogram and figures from the study, a 5% pre-test probability will yield a post-test probability of a positive result of about 23%.
Would the consequences of the test help your patient?
Yes - when the GP identifies a problem. However, the low sensitivity and prevalence means that a negative result is not so reassuring. Hence, GPs may need more training, or new ways of identifying skin cancer investigated.
The paper answered the question "how good are GPs versus dermatologists at diagnosing skin cancer." It does not answer the question "how good are GPs or dermatologists versus biopsy at diagnosing skin cancer," as biopsies were only performed on lesions thought to be malignant by dermatologists. The study might have been strengthened by performing biopsies on all lesions, in order to use a real "gold standard" against GP performance, rather than these authors' "pragmatic gold standard."