Applying an after-action review process to examine a complex public health response in New South Wales (NSW), Australia: lessons for reflective practice
Caroline H. Sharpe A * , Alexander Willems B , Amanda Robinson C , ToveLysa Fitzgerald D , Julie Letts B , Craig Dalton F G H and Andrew J. Milat B EA
B
C
D
E
F
G
H
Abstract
After-action reviews (AARs) are used to systematically examine the functions, capabilities and barriers impacting effective pandemic responses. This paper describes the methods used for and the lessons learnt from undertaking the first formal state-wide AAR of the public health response to COVID-19 in New South Wales (NSW), Australia.
A state-wide AAR was applied to examine the public health response to COVID-19 conducted by Health from January 2020 until May 2022.
The AAR was conducted between March and November 2022. The World Health Organization ARR approach was used and involved six stages including: (1) AAR design, (2) AAR planning, (3) team debriefs, (4) workshop preparation, (5) consensus workshop and (6) AAR report review and finalisation.
The AAR process involved over 100 people across the NSW network through surveys, team debriefs and workshops. The stepped process used to complete the review, with standardised templates, was found to be acceptable and feasible. The preparatory stage elicited important insights, provided an opportunity for structured reflection and helped identify themes for discussion in the workshop. Feedback methods included two participant satisfaction surveys and one post-implementation review session, which identified strengths in the process and areas that could be modified for future iterations of other public health reviews in NSW.
The AAR process successfully engaged multi-disciplinary pandemic response staff in a systematic reflection process. The process was perceived by most participants as a highly valuable opportunity to reflect and it led to important findings to improve public health emergency responses. It is important that the scope of the AAR is well understood by participants and that the psychological needs of the workforce are considered in the AAR process. There is merit in applying such reviews as standard practice in future public health emergencies.
Keywords: after-action review, COVID-19, emergency preparedness, operational review, public health debrief, quality improvement, reflective practice, resilience planning.
References
1 NSW Ministry of Health. Public Health – NSW COVID-19 Response. Sydney: NSW Ministry of Health, Population and Public Health Division; 2023. Available at www.health.nsw.gov.au/Infectious/covid-19/evidence-hub/Publications/phr-report.pdf [cited 18 May 2023].
2 NSW Health. COVID-19 Weekly Data Overview – Epidemiological week 25, ending 25 June 2022. Sydney: NSW Government; 2022. Available at www.health.nsw.gov.au/Infectious/covid-19/Documents/weekly-covid-overview-20220625.pdf [cited 2 November 2022].
3 World Health Organization. Guidance for after action review (AAR). Geneva: WHO; 2019. Available at www.who.int/publications/i/item/WHO-WHE-CPI-2019.4 [cited 30 January 2022].
4 World Health Organization. After Action Reviews and Simulation Exercises under the International Health Regulations 2005 M&E Framework (IHR MEF). Geneva: WHO; 2018. Available at https://extranet.who.int/sph/sites/default/files/document-library/document/WHO-WHE-CPI-2018.48-eng.pdf [cited 12 February 2022].
5 Aledort JE, Lurie N, Ricci KA, Dausey DJ, Howard S. RAND Corporation. Facilitated Look-Backs: A New Quality Improvement Tool for Management of Routine Annual and Pandemic Influenza. Santa Monica, CA; 2006. Available at www.rand.org/pubs/technical_reports/TR320.html [cited 9 November 2022].
6 Singleton CM, Debastiani S, Rose D, Kahn EB. An analysis of root cause identification and continuous quality improvement in public health H1N1 after-action reports. J Public Health Manag Pract 2014; 20(2): 197-204.
| Crossref | Google Scholar | PubMed |
7 Mayigane LN, de Vázquez CC, Vente C, Charles D, Copper FA, Bell A, et al. The necessity for intra-action reviews during the COVID-19 pandemic. Lancet Glob Health 2020; 8: e1451-e1452.
| Crossref | Google Scholar | PubMed |
8 World Health Organization. Intra-Action Review (IAR). Strategic Partnership for Health Security and Emergency Preparedness (SPH) Portal. Geneva: WHO; 2023. Available at https://extranet.who.int/sph/intra-action-review
9 Zhelyazkova A, Fischer PM, Thies N, Schrader-Reichling JS, Kohlmann T, Adorjan K, Huith R, Jauch KW, Prückner SM. COVID-19 management at one of the largest hospitals in Germany: Concept, evaluation and adaptation. Health Serv Manage Res 2022; 36(1): 63-74.
| Crossref | Google Scholar | PubMed |
10 European Centre for Disease Prevention and Control. One-day in-action review (IAR) protocol in the context of COVID-19. Stockholm: ECDC; 2021. Available at www.ecdc.europa.eu/sites/default/files/documents/One-day-in-action-review-protocol.pdf [cited 28 May 2023].
11 World Health Organization. Guidance for conducting a country COVID-19 intra-action review (IAR). Geneva: WHO; 2020. Available at https://iris.who.int/bitstream/handle/10665/333419/WHO-2019-nCoV-Country_IAR-2020.1-eng.pdf?sequence=1 [cited 30 April 2022].
12 Dalton C. What just happened? A guide to conducting after-action reviews of significant public health events without stress or blame. Callaghan, NSW: University of Newcastle; 2020. Available at http://hdl.handle.net/1959.13/1420587 [cited 1 May 2022].
13 Federation University Australia. Post Implementation Review Guide. Sydney, NSW, Australia; 2010. Available at https://policy.federation.edu.au/forms/18.1%20Post%20Implementation%20Review%20Guide.pdf [cited 30 May 2023].
14 Dalton C, Kirk MD, Durrheim DN. Using after-action reviews of outbreaks to enhance public health responses: lessons for COVID-19. Med J Aust 2022; 216(1): 4-9.
| Crossref | Google Scholar | PubMed |
15 Quach HL, Nguyen KC, Vogt F. After-action reviews for emergency preparedness and response to infectious disease outbreaks. Western Pac Surveill Response J 2023; 14(1): 1-8 PMCID: PMC10090030.
| Crossref | Google Scholar |
16 Kolbe M, Schmutz S, Seelandt JC, Eppich WJ, Schmutz JB. Team debriefs in healthcare: aligning intention and impact. BMJ 2021; 374: n2042.
| Crossref | Google Scholar | PubMed |
17 World Health Organization. The global practice of After Action Review: A Systematic Review of Literature. Geneva: WHO; 2019. Available at https://iris.who.int/bitstream/handle/10665/331432/WHO-WHE-CPI-2019.9-eng.pdf?sequence=1 [cited 16 November 2022].
18 Centers for Disease Control and Prevention. Morbidity and Mortality Weekly Report. Assessment of Epidemiology Capacity in State Health Departments - United States, 2009. Atlanta: CDC; 2009. Available at www.cdc.gov/mmwr/preview/mmwrhtml/mm5849a1.htm [cited 24 April 2023].
19 Piltch-Loeb R, Kraemer JD, Nelson C, Stoto MA. A Public Health Emergency Preparedness Critical Incident Registry. Biosecur Bioterror 2014; 12(3): 132-143.
| Crossref | Google Scholar | PubMed |
20 National Association of Country and City Health Officials. The Public Health Emergency Preparedness Landscape Findings from the 2018 Preparedness Profile Assessment. Washington: NACCHO; 2018. Available at www.naccho.org/uploads/downloadable-resources/2018-Preparedness-Profile-Report_external_final.pdf [cited 25 May 2022].
