Getting Started with Systematic Reviews: A Complete Beginner's Guide

Systematic reviews are the cornerstone of evidence-based practice, providing the highest level of evidence for clinical and policy decisions. If you're new to systematic reviews, this comprehensive guide will walk you through every step of the process.

What is a Systematic Review?

A systematic review is a research method that uses systematic and explicit methods to identify, select, and critically appraise relevant research, and to collect and analyze data from studies included in the review.

Key Characteristics

• Systematic approach - Follows a predefined protocol
• Comprehensive search - Attempts to identify all relevant studies
• Explicit methods - All steps are clearly documented
• Critical appraisal - Quality of evidence is assessed
• Synthesis - Results are combined using appropriate methods

Systematic Review vs. Narrative Review

Step 1: Formulate Your Research Question

The PICO Framework

Use PICO to structure your research question:

• Population: Who are you studying?
• Intervention: What intervention/exposure?
• Comparison: What is the comparison?
• Outcome: What outcomes are measured?

Example PICO Question

Question: "In adults with type 2 diabetes, does metformin compared to sulfonylureas reduce cardiovascular mortality?"

• P: Adults with type 2 diabetes
• I: Metformin
• C: Sulfonylureas
• O: Cardiovascular mortality

Additional Frameworks

• SPIDER (for qualitative studies): Sample, Phenomenon of Interest, Design, Evaluation, Research type
• SPICE (for mixed methods): Setting, Perspective, Intervention, Comparison, Evaluation

Step 2: Develop and Register Your Protocol

Why Register a Protocol?

• Reduces bias - Prevents selective reporting
• Increases transparency - Methods are public
• Avoids duplication - Others can see ongoing work
• Improves quality - Peer review before starting

Where to Register

1. PROSPERO - International prospective register of systematic reviews
2. Cochrane Library - For Cochrane reviews
3. Open Science Framework - General research registry

Protocol Components

Your protocol should include:

1. Background and rationale
2. Research question (PICO)
3. Inclusion/exclusion criteria
4. Search strategy
5. Study selection process
6. Data extraction plan
7. Risk of bias assessment
8. Data synthesis methods
9. Timeline and resources

Step 3: Design Your Search Strategy

Database Selection

Core Medical Databases:

• MEDLINE (via PubMed or Ovid)
• Embase
• Cochrane Central Register of Controlled Trials

Discipline-Specific Databases:

• PsycINFO (psychology)
• ERIC (education)
• Web of Science (multidisciplinary)

Search Strategy Development

1. Identify Key Concepts

Break down your PICO into searchable concepts:

• Population terms
• Intervention terms
• Outcome terms

2. Use Boolean Operators

• AND - Narrows search (all terms must be present)
• OR - Broadens search (any term can be present)
• NOT - Excludes terms (use cautiously)

3. Apply Search Techniques

• Truncation (*) - child* finds child, children, childhood
• Wildcards (?) - wom?n finds woman, women
• Phrase searching - "systematic review" for exact phrase
• Field searching - [ti] for title, [ab] for abstract

Example Search Strategy

Population: (diabetes OR "diabetes mellitus" OR diabetic)
AND
Intervention: (metformin OR "metformin hydrochloride")
AND
Outcome: (mortality OR death OR "cardiovascular death")
AND
Study Design: (randomized OR RCT OR "controlled trial")

Step 4: Study Selection Process

Inclusion and Exclusion Criteria

Develop specific criteria based on:

• Study design - RCTs, cohort studies, case-control
• Population - Age, condition, setting
• Intervention - Specific treatments or exposures
• Outcomes - Primary and secondary outcomes
• Language - Usually no restrictions recommended
• Publication date - Set appropriate limits

Screening Process

Two-Stage Screening

1. Title/Abstract Screening - Quick initial filter
2. Full-Text Screening - Detailed assessment

Best Practices

• Two independent reviewers for all stages
• Pilot testing to ensure consistency
• Regular calibration exercises
• Conflict resolution process
• Clear documentation of decisions

Using Screening Tools

Modern systematic reviews benefit from screening software:

Manual Tools:

• Rayyan (web-based)
• Covidence (Cochrane)
• DistillerSR (paid platform)

AI-Powered Tools:

• Our platform offers AI-assisted screening
• Reduces screening time by 80%
• Maintains high sensitivity (95%+)
• Provides confidence scoring

Step 5: Data Extraction

Planning Data Extraction

Create a standardized form including:

• Study characteristics - Author, year, country, setting
• Population details - Sample size, demographics, inclusion criteria
• Intervention details - Dose, duration, comparison
• Outcome data - Results, effect sizes, confidence intervals
• Quality indicators - Risk of bias elements

Data Extraction Process

1. Pilot the form with 2-3 studies
2. Two extractors work independently
3. Compare extractions and resolve discrepancies
4. Maintain detailed records of decisions

Common Extraction Challenges

• Missing data - Contact authors when possible
• Different outcome measures - May need conversion
• Multiple publications - Avoid double-counting
• Unclear reporting - Document assumptions

Step 6: Risk of Bias Assessment

Tools for Different Study Types

Randomized Controlled Trials:

• RoB 2 (Cochrane Risk of Bias tool 2.0)
• Covers: randomization, deviations, missing data, measurement, selection

Non-Randomized Studies:

• ROBINS-I (Risk Of Bias In Non-randomized Studies)
• Newcastle-Ottawa Scale (for cohort and case-control studies)

Diagnostic Studies:

• QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies)

Risk of Bias Domains

Typical domains include:

1. Selection bias - How participants were selected
2. Performance bias - Differences in care received
3. Detection bias - Differences in outcome assessment
4. Attrition bias - Differences in withdrawals
5. Reporting bias - Selective outcome reporting

Step 7: Data Synthesis

Types of Synthesis

Meta-Analysis (Quantitative)

• When appropriate - Similar studies, outcomes, populations
• Statistical pooling of effect estimates
• Forest plots to visualize results
• Heterogeneity assessment using I² statistic

Narrative Synthesis (Qualitative)

• When meta-analysis inappropriate - High heterogeneity
• Structured approach to combining findings
• Thematic analysis of results
• Vote counting or effect direction plots

Assessing Heterogeneity

• Clinical heterogeneity - Differences in populations, interventions
• Methodological heterogeneity - Differences in study design
• Statistical heterogeneity - Variation in results beyond chance

Step 8: Assess Certainty of Evidence

GRADE Approach

The Grading of Recommendations Assessment, Development and Evaluation system assesses:

1. Risk of bias - Study limitations
2. Inconsistency - Unexplained heterogeneity
3. Indirectness - Applicability concerns
4. Imprecision - Wide confidence intervals
5. Publication bias - Selective reporting

Evidence Certainty Levels

• High - Very confident in effect estimate
• Moderate - Moderately confident
• Low - Limited confidence
• Very Low - Very little confidence

Step 9: Report Your Review

PRISMA Guidelines

Follow PRISMA 2020 (Preferred Reporting Items for Systematic Reviews and Meta-Analyses):

• 27-item checklist covering all aspects
• Flow diagram showing study selection
• Transparent reporting of methods and results

Key Sections

1. Title and Abstract - Clear identification as systematic review
2. Introduction - Rationale and objectives
3. Methods - All procedures described
4. Results - Study selection, characteristics, synthesis
5. Discussion - Summary, limitations, implications
6. Funding - Sources of support

Common Reporting Issues

• Incomplete search strategy - Provide full strategies
• Missing PRISMA flow diagram - Essential for transparency
• Inadequate risk of bias reporting - Show assessments clearly
• Poor presentation of results - Use appropriate figures/tables

Tools and Resources

Essential Software

Reference Management:

• Zotero (free)
• EndNote (institutional license)
• Mendeley (free with premium options)

Screening and Review:

• Our AI-powered platform
• Rayyan (free web-based)
• Covidence (paid)

Meta-Analysis:

• Review Manager (RevMan) - Free from Cochrane
• R with meta packages (free)
• Stata (paid)
• Comprehensive Meta-Analysis (paid)

Helpful Resources

Guidelines and Training:

• Cochrane Handbook
• PRISMA Statement
• York Centre for Reviews and Dissemination

Online Courses:

• Cochrane Interactive Learning
• Johns Hopkins Coursera course
• University-specific training programs

Timeline and Planning

Typical Timeline

A systematic review typically takes 12-24 months:

• Protocol development: 2-3 months
• Search and screening: 3-6 months
• Data extraction: 2-4 months
• Analysis and synthesis: 2-3 months
• Writing and review: 3-6 months

Team Requirements

Minimum team:

• Principal investigator - Overall leadership
• Information specialist - Search strategy
• 2 reviewers - Screening and extraction
• Statistician - Meta-analysis (if needed)

Larger teams may include:

• Content experts
• Junior researchers/students
• Additional reviewers for large reviews

Common Pitfalls and How to Avoid Them

1. Poor Research Question

Problem: Overly broad or unclear question Solution: Use PICO framework, seek expert input

2. Inadequate Search Strategy

Problem: Missing key databases or terms Solution: Consult information specialist, pilot test

3. Inconsistent Screening

Problem: Reviewers apply criteria differently Solution: Pilot screening, regular calibration

4. Inappropriate Meta-Analysis

Problem: Combining incompatible studies Solution: Assess heterogeneity carefully

5. Poor Documentation

Problem: Cannot reproduce methods Solution: Document everything, use protocols

Getting Started: Action Plan

Week 1: Planning

• Form your team
• Develop research question (PICO)
• Conduct preliminary literature search
• Define inclusion/exclusion criteria

Week 2-3: Protocol

• Write full protocol
• Get protocol reviewed
• Register protocol (PROSPERO)
• Set up project management system

Week 4-6: Search Strategy

• Develop comprehensive search strategy
• Consult with librarian/information specialist
• Pilot test search strategy
• Execute searches in all databases

Month 2-4: Screening

• Set up screening software
• Conduct pilot screening
• Screen titles/abstracts
• Obtain full texts
• Screen full texts

Month 4-6: Data Extraction

• Develop extraction forms
• Pilot extraction process
• Extract data from included studies
• Assess risk of bias

Month 6-8: Analysis

• Assess heterogeneity
• Conduct meta-analysis (if appropriate)
• Perform sensitivity analyses
• Apply GRADE assessment

Month 8-12: Reporting

• Write first draft
• Internal review and revision
• Submit for peer review
• Revise based on feedback
• Final publication

Conclusion

Conducting your first systematic review can seem overwhelming, but breaking it down into manageable steps makes it achievable. Remember:

1. Start with a clear question using PICO
2. Follow established guidelines like PRISMA
3. Register your protocol before starting
4. Work in teams with independent reviewers
5. Document everything for transparency
6. Seek expert help when needed

The most important thing is to start. Every experienced systematic reviewer was once a beginner. With careful planning, attention to detail, and the right tools, you can conduct a high-quality systematic review that contributes meaningfully to the evidence base.

Ready to Begin?

If you're ready to start your systematic review journey:

• Try our AI-powered screening tool to see how technology can accelerate your review
• Download our protocol template to get started quickly
• Join our community of systematic reviewers for support and advice

Start Your Free Trial | Try Demo | Contact for Support

Your journey to becoming a systematic review expert starts with a single step. Take that step today!