Systematic Reviews

Much like the question formulation, search, and selection processes, the process of evaluating the quality of the literature in a systematic review should be structured. In the critical appraisal stage, the researchers analyze the design of each included study, the validity of the results in light of the design, potential for bias, and the relevance of each study to the larger literature and the research question. There are tools that researchers can use to help them approach this process systematically and objectively. 

These are key issues to consider when appraising various types of medical studies. 

Key Issues in Appraising Therapy Studies:

• Randomization and concealed allocation
• Follow-up of all patients (ideally 80% or better)
• Blinding (concealment) of patients, clinicians and study personnel to the treatment being provided
• Intention to Treat analysis
• Baseline similarities between groups (established at the start of the trial)

Key Issues in Appraising Diagnostic Studies:

• Independent blind comparison with a gold standard
• Appropriate spectrum of patients
• All patients receive both tests
• Key Issues in Appraising Prognosis Studies:
• Well-defined sample of patients
• Similar prognostic factors
• Follow-up
• Objective outcome criteria

Key Issues for Etiology/Harm Studies:

• Similarity of comparison groups
• Outcomes and exposure measured the same for both groups
• Follow-up of sufficient length