The impact of self-assessment and surveyor assessment on site visit performance under the National General Practice Accreditation scheme
David T. McNaughton
A * , Paul Mara B and Michael P. Jones A
A
B
Abstract
There is a need to undertake more proactive and in-depth analyses of general practice accreditation processes. Two areas that have been highlighted as areas of potential inconsistency are the self-assessment and surveyor assessment of indicators.
The data encompass 757 accreditation visits made between December 2020 and July 2022. A mixed-effect multilevel logistic regression model determined the association between attempt of the self-assessment and indicator conformity from the surveyor assessment. Furthermore, we present a contrast of the rate of indicator conformity between surveyors as an approximation of the inter-assessor consistency from the site visit.
Two hundred and seventy-seven (37%) practices did not attempt or accurately report conformity to any indicators at the self-assessment. Association between attempting the self-assessment and the rate of indicator non-conformity at the site visit failed to reach statistical significance (OR = 0.90 [95% CI = 1.14–0.72], P = 0.28). A small number of surveyors (N = 9/34) demonstrated statistically significant differences in the rate of indicator conformity compared to the mean of all surveyors.
Attempt of the self-assessment did not predict indicator conformity at the site visit overall. Appropriate levels of consistency of indicator assessment between surveyors at the site visit were identified.
Keywords: accreditation, audit and feedback, general practice, health policy, health services, primary health care, quality and safety, reliability.
References
1 The Royal Australian College of General Practitioners. Standards for General Practices. 5th edn. RACGP; 2020. Available at https://www.racgp.org.au/running-a-practice/practice-standards/standards-5th-edition/standards-for-general-practices-5th-ed-1
2 RACGP. Changes to the RACGP Standards for general practices, 5th edn. Available at https://www.racgp.org.au/running-a-practice/practice-standards/standards-5th-edition/standards-for-general-practices-5th-ed/introduction-to-the-standards-for-general-practice/changes-from-the-previous-edition
3 Jones M, McNaughton D, Mara P. General practice accreditation–does time spent on-site matter? Aust Health Rev 2023; 47(6): 689-693.
| Crossref | Google Scholar | PubMed |
4 MP Consulting. Review of General Practice Accreditation Arrangements. 2021. Available at https://consultations.health.gov.au/primary-health-networks-strategy-branch/review-of-general-practice-accreditation-arrangeme/user_uploads/review-of-the-ngpa-scheme---consultation-paper---final---110821--004-.pdf
5 Khoury J, Krejany CJ, Versteeg RW, et al. A process for developing standards to promote quality in general practice. Fam Pract 2019; 36(2): 166-171.
| Crossref | Google Scholar | PubMed |
6 Service Australia. Practice Incentives Program. 2023. Available at https://www.servicesaustralia.gov.au/practice-incentives-program
7 Debono D, Greenfield D, Testa L, et al. Understanding stakeholders’ perspectives and experiences of general practice accreditation. Health Policy 2017; 121(7): 816-822.
| Crossref | Google Scholar | PubMed |
8 Braithwaite J, Greenfield D, Westbrook J, et al. Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study. Qual Saf Health Care 2010; 19(1): 14-21.
| Crossref | Google Scholar | PubMed |
10 Greenfield D, Pawsey M, Naylor J, Braithwaite J. Are accreditation surveys reliable? Int J Health Care Qual Assur 2009; 22(2): 105-116.
| Crossref | Google Scholar | PubMed |
11 Greenfield D, Braithwaite J. Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation. Qual Saf Health Care 2009; 18(3): 162-163.
| Crossref | Google Scholar | PubMed |
12 Australian Commission on Safety and Quality in Healthcare. Policy - Approval under the National General Practice Accreditation (NGPA) Scheme to conduct assessments. 2022. Available at https://www.safetyandquality.gov.au/publications-and-resources/resource-library/policy-approval-under-national-general-practice-accreditation-ngpa-scheme-conduct-assessments
14 Erwin PC. A Self-Assessment Process for Accreditation Preparedness: A Practical Example for Local Health Departments. J Public Health Manag Pract 2009; 15(6): 503-508.
| Crossref | Google Scholar | PubMed |
