Evidence-based decision-making: practical issues in the appraisal of evidence to inform policy and practice
Bronwyn J. CarterA Skills Enhancement for Public Health Project, School of Public Health, La Trobe University, Bundoora, VIC 3083, Australia.
B Division of Health Studies Research, La Trobe University, Bundoora, VIC 3083, Australia. Email: bronwyn.carter@latrobe.edu.au
Australian Health Review 34(4) 435-440 https://doi.org/10.1071/AH09778
Submitted: 30 April 2009 Accepted: 28 March 2010 Published: 25 November 2010
Abstract
Objective. To highlight the differences between a systematic review of the literature and a systematic review of the best available evidence; to discuss practical issues in the appraisal of evidence to inform public health policy and practice; and to make recommendations for next steps in the development of evidence-based decision making in public health.
Data sources and selection. Literature and other sources were reviewed including the subject reading list, recommended texts and websites for the La Trobe University postgraduate subject Evidence Based Public Health Practice 2007 and other relevant sources identified.
Data extraction and synthesis. Relevant opinions were extracted to summarise debate in relation to definitions of evidence, usefulness of systematic reviews, tools for critical appraisal and other practical issues in the translation of evidence into practice.
Conclusions. Evidence relevant to decisions regarding public health policy and practice may include evidence from the literature including experimental and observational studies as well as other sources, including policies and opinions of stakeholders. Further development of skills and approaches to the critical appraisal of evidence are required. Recommendations include: mapping of Australian competencies to public health education; development of national guidelines to inform the appraisal of evidence for public health decision making; and promotion of leadership and education in evidence-based approaches, discussion and debate in relation to definitions of evidence, and public health research that generates the best possible evidence.
What is known about the topic? Systematic reviews are a well recognised tool for the critical appraisal of evidence to inform decision making. There is a lack of agreement about what constitutes valid evidence for inclusion in such reviews and many policy makers have no training or qualifications in the use of systematised approaches to the critical appraisal of evidence from a range of sources to inform decisions. Approaches to critical appraisal of evidence and skills in appraisal of evidence and evidence-based decision making require further development.
What does this paper add? This paper reviews current opinions on what constitutes valid evidence and discusses important differences between a systematic review of the literature as distinct from a systematic review of available evidence. The desirable approach is recognised as the utilisation of the best available evidence from a range of sources to inform decision making, including evidence from observational studies including qualitative data, as well as contextual and colloquial evidence. This paper calls for: mapping of Australian competencies to public health education; development of national guidelines to inform the appraisal of evidence for public health decision making; and the promotion of leadership and education in evidence-based approaches, discussion and debate in relation to definitions of evidence and the promotion of public health research that generates the best possible evidence.
What are the implications for practitioners? This paper calls upon practitioners to further develop skills in critical appraisal of evidence from a range of sources to inform policy and practice, and to foster collaborative partnerships between researchers, policy makers, educators, managers and clinicians.
References
[1] Brownson RC, Baker E, Leet T, Gillespie K. Evidence-based Public Health. New York: Oxford University Press; 2003.[2] Healthy People 2010. US Department of Health and Human Service; 2010. Available at http://www.healthypeople.gov/About/goals.htm [verified 7 October 2010].
[3] Frommer M, Rychetnik L. From evidence-based medicine to evidence-based public health. In: Lin V, Gibson B, editors. Evidence-based health policy: problems and possibilities. Melbourne: Oxford University Press; 2003. pp. 56–69.
[4] Rychetnik L, Frommer M. A Schema for Evaluating Evidence on Public Health Interventions; Version 4. Melbourne: National Public Health Partnership; 2002. Available at http://www.dhs.vic.gov.au/nphp/publications/phpractice/schemaV4.pdf [verified 7 October 2010].
[5] Jack SM. Utility of qualitative research findings in evidence-based public health practice. Public Health Nurs 2006; 23 277–83.
| Utility of qualitative research findings in evidence-based public health practice.Crossref | GoogleScholarGoogle Scholar | 16684207PubMed |
[6] Petticrew M. Systematic reviews from astronomy to zoology: myths and misconceptions. BMJ 2001; 322 98–101.
| Systematic reviews from astronomy to zoology: myths and misconceptions.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD3M3itl2itQ%3D%3D&md5=e2c3fef7b0dd2bdc47c4c373d7c792b4CAS | 11154628PubMed |
[7] Mulrow CD. Systematic reviews: rationale for systematic reviews. BMJ 1994; 309 597–9.
| 1:STN:280:DyaK2cznt1OmtQ%3D%3D&md5=c581da7022e29567582fc9c253ce6830CAS | 8086953PubMed |
[8] Jackson N. Health Promotion and Public Health Systematic Review Handbook. Cochrane Health Promotion and Public Health Field. Undated. Available at www.vichealth.vic.gov.au/cochrane/welcome/index.htm [verified 1 June 2007].
[9] Rychetnik L, Hawe P, Waters E, Barratt A, Frommer M. A glossary for evidence based public health. J Epidemiol Community Health 2004; 58 538–45.
| A glossary for evidence based public health.Crossref | GoogleScholarGoogle Scholar | 15194712PubMed |
[10] Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997; 127 380–7.
| 1:STN:280:DyaK2svhtVOmsQ%3D%3D&md5=1eb1aafa34a5aaa553ddfe6b2b7b4b88CAS | 9273830PubMed |
[11] Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Chichester, UK: Wiley; 2005. Available at http://www.cochrane.org/training/cochrane-handbook [verified 7 October 2010].
[12] Holmes D, Murray S, Peron A, Rail G. Deconstructing the evidence-based discourse in health sciences: truth, power and fascism. Int J Evid-Based Healthc 2006; 4 180–6.
| Deconstructing the evidence-based discourse in health sciences: truth, power and fascism.Crossref | GoogleScholarGoogle Scholar | 21631765PubMed |
[13] Nutbeam D. Evaluating health promotion – progress, problems and solutions. Health Promot Int 1998; 13 27–44.
| Evaluating health promotion – progress, problems and solutions.Crossref | GoogleScholarGoogle Scholar |
[14] Petticrew M, Roberts H. Evidence, hierarchies, and typologies: horses for courses. J Epidemiol Community Health 2003; 57 527
| Evidence, hierarchies, and typologies: horses for courses.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD3s3psVaqtw%3D%3D&md5=56615ec92fa61a1138d7f170f6855e76CAS | 12821702PubMed |
[15] Petticrew M. Why certain systematic reviews reach uncertain conclusions. BMJ 2003; 326 756–8.
| Why certain systematic reviews reach uncertain conclusions.Crossref | GoogleScholarGoogle Scholar | 12676848PubMed |
[16] Victora CG, Habicht J-P, Bryce J. Evidence-based public health: moving beyond randomised trials. Am J Public Health 2004; 94 400–5.
| Evidence-based public health: moving beyond randomised trials.Crossref | GoogleScholarGoogle Scholar | 14998803PubMed |
[17] MythBusters Teaching Resource. Ottawa: Canadian Health Services Research Foundation; 2008. p. 11. Available at http://www.chsrf.ca/mythbusters/pdf/teaching-resource1_e.pdf [verified 7 October 2010].
[18] Directly observed therapy for treating tuberculosis. Wiley InterScience. 2006. Available at http://mrw.interscience.wiley.com/cochrane/clsysrev/articles/CD003343/frame.html [verified 30 April 2009].
[19] Munro SA, Lewin SA, Smith HJ, Engel ME, Fretheim A, Volmink J. Patient adherence to tuberculosis treatment: a systemative review of qualitative research. PLoS Med 2007; 4 e238 [Review]
| Patient adherence to tuberculosis treatment: a systemative review of qualitative research.Crossref | GoogleScholarGoogle Scholar | 17676945PubMed |
[20] Lavis JN, Davies HT, Gruen RL, Walshe K, Farquhar CM. Working within and beyond the Cochrane Collaboration to make Systematic Reviews more useful to healthcare managers and policy makers. Healthc Policy 2006; 1 21–33.
| 19305650PubMed |
[21] Carlton A-L. Regulation of traditional Chinese medicine in Victoria. In: Lin V, Gibson B, editors. Evidenced-based health policy: problems and possibilities. Melbourne: Oxford University Press; 2003.
[22] Communicating with consumers and carers. Victoria Quality Council, Department of Human Services; 2007. Available at http://www.health.vic.gov.au/qualitycouncil/downloads/project_report.pdf [verified 7 October 2010].
[23] Brehaut JD, Juzwishin D. HTA Initiative #18. Bridging the Gap: the use of research evidence in policy development. Edmonton, AB: Alberta Heritage Foundation for Medical Research; 2005.
[24] How to use the evidence: assessment and application of scientific evidence. NHMRC; 2005. Available at http://www.nhmrc.gov.au/publications/synopses/cp69syn.htm [verified 7 October 2010].
[25] How to review the evidence: systematic identification and review of the scientific literature. NHMRC; 2005. Available at http://www.nhmrc.gov.au/publications/synopses/cp65syn.htm [verified 7 October 2010].
[26] Daly J, Willis K, Small R, Green J, Welch N, Kealy M, et al A hierarchy of evidence for assessing qualitative health research. J Clin Epidemiol 2007; 60 43–9.
| A hierarchy of evidence for assessing qualitative health research.Crossref | GoogleScholarGoogle Scholar | 17161753PubMed |
[27] Denzin N, Lincoln Y. Handbook of qualitative research. London: Sage; 1994.
[28] Heron J, Reason P. A participatory inquiry paradigm. Qual Inq 1997; 3 274–94.
| A participatory inquiry paradigm.Crossref | GoogleScholarGoogle Scholar |
[29] Interprofessional Health Education in Australia: the Way Forward – Final Report. Learning and Teaching for Interprofessional Practice Australia (L-TIPP). 2009. Available at http://www.aippen.net/docs/LTIPP_proposal_apr09.pdf [verified 24 October 2010].
[30] Weightman A, Ellis S, Cullum A, Sander L, Turley R. Grading evidence and recommendations for public health interventions: developing and piloting a framework. London: Health Development Agency; 2005. Available at www.hda.nhs.uk [verified June 2007].
[31] Critical Appraisal Skills Programme. Public Health Research Unit; 2007. Available at http://www.sph.nhs.uk/what-we-do/public-health-workforce/resources/critical-appraisals-skills-programme [verified 30 April 2009].
[32] Armstrong R, Waters E, Jackson N, Oliver S, Popay J, Shepherd J, et al. Guidelines for systematic reviews of health promotion and public health interventions. Version 2. Melbourne: University of Melbourne; 2007. Available at http://ph.cochrane.org/sites/ph.cochrane.org/files/uploads/Guidelines%20HP_PH%20reviews.pdf [verified 24 October 2010].
[33] Quality Assessment Tool for Quantitative Studies. Effective Public Health Practice Project; 2003. Available at http://www.ephpp.ca/tools.html [verified 30 April 2009].
[34] Lewin SA, Dick J, Pond P, Zwarenstein M, Aja GN, van Wyk B, Bosch-Capblanch X, Patrick M. Lay health workers in primary and community health care. Cochrane Database Syst Rev 2005; CD004015 [Review]
| Lay health workers in primary and community health care.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD2M%2Fks1ahuw%3D%3D&md5=f2ca45974bd0452c1693ced3e6bccbb4CAS | 15674924PubMed |
[35] Core Competencies for Public Health in Canada: Release 1.0. Public Health Agency of Canada; 2007. Available at http://www.phac-aspc.gc.ca/ccph-cesp/pdfs/cc-manual-eng090407.pdf [verified 7 October 2010].
[36] Good public health practice. Standards for public health physicians and specialists in training. Faculty of Public Health of the Royal Collages of Physicians of the United Kingdom; 2001. Available at http://www.fphm.org.uk/prof_standards/downloads/general_standards/D_Stds_for_PHPs_Specialists.pdf [verified 8 February 2010].
[37] Core competencies for public health professionals. US Public Health Foundation; 2009. Available at http://www.phf.org/link/corecompetencies.htm#view [verified 24 October 2010].
[38] Genat B, Robinson P, Parker E. Foundation Competencies for Master of Public Health Graduates in Australia. Australian Network of Public Health Institutions (ANAPHI); 2009.
[39] Genat B, Robinson P. New competencies for public health graduates: a useful tool for course design. Aust N Z J Public Health 2010; 34 513–6.
| New competencies for public health graduates: a useful tool for course design.Crossref | GoogleScholarGoogle Scholar | 21040181PubMed |
[40] Troy C. Online Public Health Professional Development in Canada and Internationally. In: Rossiter J, editor. Canadian Network for Innovation in Education Annual Conference; 10–13 May 2009; Ottawa, ON; 2009.
[41] ASPHER’s European Public Health Core Competencies Programme (EPHCC). The Association of Schools of Public Health in the European Region (ASPHER); 2010. Available at http://www.aspher.org/index.php?site=ccc&PHPSESSID=174f0de931758fedb901b16ee1e950d0 [verified 8 February 2010].
[42] Skills Enhancement for Public Health. Australian Skills Enhancement Project La Trobe University; 2008. Available at http://www.phac-aspc.gc.ca/sehs-acss/whatnew-eng.php [verified 8 February 2010].
[43] Carter B, Ervin L, Lowe J, Wright E, Robinson P. Mapping Public Health Competencies to Online Education for Public Health Practitioners, Public Health Association of Australia 40th Annual Conference, 27–29 September 2010; Adelaide; 2010.
[44] Young T. Cochrane Column Response from SA Lewin, J Dick, P Pond, M Zwarenstein, G Aja, B van Wyk, X Bosch-Capblanch, and M Patrick. Response from Gill Walt. Response from Richard H Morrow. Response from Simon Lewin, Merrick Zwarenstein, and Judy Dick. Int J Epidemiol 2005; 34 1250–3.
| Cochrane Column Response from SA Lewin, J Dick, P Pond, M Zwarenstein, G Aja, B van Wyk, X Bosch-Capblanch, and M Patrick. Response from Gill Walt. Response from Richard H Morrow. Response from Simon Lewin, Merrick Zwarenstein, and Judy Dick.Crossref | GoogleScholarGoogle Scholar |
[45] Potter M, Quill B, Aglipay G, Anderson E, Rowitz L, Smith L, et al. Demonstrating excellence in practice-based research for public health. Association of Schools of Public Health; 2006. Available at http://www.asph.org/document.cfm?page=600 [verified 24 October 2010].
[46] Family Choice Program. Melbourne: Royal Children’s Hospital. Available at http://www.rch.org.au/hacc/services/index.cfm?doc_id=1192#FCP [verified 7 October 2010].
[47] Cooper C, Wheeler DM, Woolfenden S, Boss T, Piper S. Specialist home-based nursing services for children with acute and chronic illnesses. Cochrane Database Syst Rev 2006; CD004383
| Specialist home-based nursing services for children with acute and chronic illnesses.Crossref | GoogleScholarGoogle Scholar | 1:STN:280:DC%2BD28nitVKkuw%3D%3D&md5=463350adbea955be05bb344c0c51fe80CAS | 17054202PubMed |
