Critical Appraisal
Critical appraisal is the process of systematically examining research evidence to assess its validity, results and relevance before using it to inform a decision. This guide aims to introduce you to the process and the tools used to appraise different types of paper.
We also offer training in critical appraisal. Please see the Current Training Timetable for dates and times.
Critical appraisal is an essential part of evidence based practice. It is a process which helps practitioners decide whether a piece of research is good enough to be used in decision making. It is about evaluating the whole research process, rather than looking just at the results.
To help you undertake critical appraisal, several user-friendly tools are available. These are usually in the form of checklists and many of them are freely available online. These checklists assist you in asking appropriate questions when reading a paper and help you to maintain an objective view of the research. See below for more details about critical appraisal check-lists.
We also offer training in critical appraisal. Please see the Current Training Timetable for dates and times.
Critical appraisal is an essential part of evidence based practice. It is a process which helps practitioners decide whether a piece of research is good enough to be used in decision making. It is about evaluating the whole research process, rather than looking just at the results.
To help you undertake critical appraisal, several user-friendly tools are available. These are usually in the form of checklists and many of them are freely available online. These checklists assist you in asking appropriate questions when reading a paper and help you to maintain an objective view of the research. See below for more details about critical appraisal check-lists.
|
Reading a paper
When approaching a paper, there are several things to consider: Look at extrinsic factors. These are factors not related directly to the study itself, but which may have some influence on it. Ask questions like:
|
|
|
Intrinsic factors are all factors relating to the study itself Consider issues such as:
|
- A good paper will be divided into sections. This will help you to locate relevant pieces of information in order to answer your questions.
- Is it of interest? - the title and abstract should give you enough information to decide whether a paper is of interest
- Why was it done? - the introduction will give you the background, the context and maybe some information about previous research
- How was it done? - the methodology will go into detail about how the study was undertaken
- What has it found? - the results section should go into detail about the results and their meaning
- What are the implications? - the discussion section will consider the implications and whether any further study is required
For example:
Critical appraisal tools
There are standard critical appraisal questions. These relate to the validity, results and relevance of the research. Whichever type of check-list you choose to use, the questions you ask will revolve around these three issues. While check-lists will ask questions in slightly different ways, they will cover the same ground. The questions you ask, and therefore the check-list you use, will depend on what kind of study is being reported.
There are several checklists available from different sources:
More detailed information about how to read different papers can be found at:
Other useful information about types of study and their relative value can be found at:
There are standard critical appraisal questions. These relate to the validity, results and relevance of the research. Whichever type of check-list you choose to use, the questions you ask will revolve around these three issues. While check-lists will ask questions in slightly different ways, they will cover the same ground. The questions you ask, and therefore the check-list you use, will depend on what kind of study is being reported.
There are several checklists available from different sources:
- Critical Appraisal Skills Programme (CASP) Tools & Checklists
- Best Bets Critical Appraisal Worksheets
- Centre for Evidence Based Medicine Critical Appraisal Sheets
- Birmingham Critical Appraisal Skills Programme
- SIGN Critical Appraisal Notes and Checklists
More detailed information about how to read different papers can be found at:
- BMJ Resources
- Centre for Health Evidence (Canada)
- The Users' Guides to the Medical Literature form a series of articles published in JAMA on which many of the checklists are based. See below for examples. (You will need an NHS Athens user name and password to access some of these articles).
Other useful information about types of study and their relative value can be found at:
- University of Minnesota - Understanding Research Study Designs
- Centre for Evidence Based Medicine - Guide to Study Designs and Levels of Evidence
Glossary
Some technical terminology is used for the statistical aspects of research and you may find it useful to consult a glossary for definitions of these.
See our detailed Critical Appraisal Glossary for definitions and links to explanations.
Other useful information can be found at:
BMJ Evidence-Based Medicine - Down with odds ratios
Bandolier - Describing results of trials and reviews
CMAJ - Tips for Learners of evidence-based medicine
CEBM - Likelihood ratios
CEBM - CATMaker (computer-assisted critical appraisal tool) and EBM calculators
Some technical terminology is used for the statistical aspects of research and you may find it useful to consult a glossary for definitions of these.
See our detailed Critical Appraisal Glossary for definitions and links to explanations.
Other useful information can be found at:
BMJ Evidence-Based Medicine - Down with odds ratios
Bandolier - Describing results of trials and reviews
CMAJ - Tips for Learners of evidence-based medicine
CEBM - Likelihood ratios
CEBM - CATMaker (computer-assisted critical appraisal tool) and EBM calculators
Further reading
Journal articles - Users’ Guides Series:
Guyatt GH, Sackett DL, Cook DJ. 1993. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA: the journal of the American Medical Association. 270(21), pp2598-601
Guyatt G.H. et al. 1994. Users' guides to the medical literature: II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA: Journal of the American Medical Association. 271(1), pp59-63
Jaeschke R., Guyatt G., Sackett D.L. 1994. Users' guides to the medical literature: III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA: Journal of the American Medical Association. 271(5), pp389-391
Jaeschke R., Guyatt G.H., Sackett D.L. 1994. User's guides to the medical literature: III. How to use an article about a diagnostic test: B. What are the results and will they help me in caring for my patients? JAMA: Journal of the American Medical Association. 271(9), pp703-707
Oxman A.D., Cook D.J., Guyatt G.H. 1994. Users' guides to the medical literature: VI. How to use an overview. JAMA: Journal of the American Medical Association. 272(17), pp1367-1371
Oxman AD, Sackett DL, Guyatt GH. 1993. Users' guides to the medical literature: I. How to get started. JAMA: Journal of the American Medical Association. 270(17), pp2093-2095
For other papers in the series see The Users' Guides to the Medical Literature. (You will need an NHS Athens user name and password to access the full text).
Books
There are several books about critical appraisal. These are examples of the kind of titles you might find:
AJETUNMOBI, O. 2002. Making sense of critical appraisal. London: Arnold.
BOWERS, D. HOUSE, A. OWENS, D. 2014. Understanding clinical papers. 3rd ed. Chichester: Wiley.
CUTCLIFFE, J. and WARD, M. 2007. Critiquing nursing research. 2nd ed. London: Quay Books.
GREENHALGH, T. 2010. How to read a paper: the basics of evidence-based medicine. 4th ed. Chichester: Wiley-Blackwell.
Also available as an ebook (You will need an NHS Athens user name and password to access the text).
GUYATT, G. RENNIE, D. et al. 2008. Users’ guides to the medical literature. 2nd ed. New York: McGraw-Hill
For other books on critical appraisal, search the online library catalogue
There are also many ebooks on critical appraisal, explore our ebooks collection
Remember, if you are interested in undertaking training in critical appraisal, please see the Current Training Timetable.
Journal articles - Users’ Guides Series:
Guyatt GH, Sackett DL, Cook DJ. 1993. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? JAMA: the journal of the American Medical Association. 270(21), pp2598-601
Guyatt G.H. et al. 1994. Users' guides to the medical literature: II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients? JAMA: Journal of the American Medical Association. 271(1), pp59-63
Jaeschke R., Guyatt G., Sackett D.L. 1994. Users' guides to the medical literature: III. How to use an article about a diagnostic test. A. Are the results of the study valid? JAMA: Journal of the American Medical Association. 271(5), pp389-391
Jaeschke R., Guyatt G.H., Sackett D.L. 1994. User's guides to the medical literature: III. How to use an article about a diagnostic test: B. What are the results and will they help me in caring for my patients? JAMA: Journal of the American Medical Association. 271(9), pp703-707
Oxman A.D., Cook D.J., Guyatt G.H. 1994. Users' guides to the medical literature: VI. How to use an overview. JAMA: Journal of the American Medical Association. 272(17), pp1367-1371
Oxman AD, Sackett DL, Guyatt GH. 1993. Users' guides to the medical literature: I. How to get started. JAMA: Journal of the American Medical Association. 270(17), pp2093-2095
For other papers in the series see The Users' Guides to the Medical Literature. (You will need an NHS Athens user name and password to access the full text).
Books
There are several books about critical appraisal. These are examples of the kind of titles you might find:
AJETUNMOBI, O. 2002. Making sense of critical appraisal. London: Arnold.
BOWERS, D. HOUSE, A. OWENS, D. 2014. Understanding clinical papers. 3rd ed. Chichester: Wiley.
CUTCLIFFE, J. and WARD, M. 2007. Critiquing nursing research. 2nd ed. London: Quay Books.
GREENHALGH, T. 2010. How to read a paper: the basics of evidence-based medicine. 4th ed. Chichester: Wiley-Blackwell.
Also available as an ebook (You will need an NHS Athens user name and password to access the text).
GUYATT, G. RENNIE, D. et al. 2008. Users’ guides to the medical literature. 2nd ed. New York: McGraw-Hill
For other books on critical appraisal, search the online library catalogue
There are also many ebooks on critical appraisal, explore our ebooks collection
Remember, if you are interested in undertaking training in critical appraisal, please see the Current Training Timetable.
Journal Club Support Service
We offer advice and practical assistance on how to setup and run a journal club, providing you with support and resources to help you do this. Please visit our Journal Club Support Service page for more information.