Australian Health Review Australian Health Review Society
Journal of the Australian Healthcare & Hospitals Association

Just Accepted

This article has been peer reviewed and accepted for publication. It is in production and has not been edited, so may differ from the final published form.

Accuracy of national KPI reporting from two Aboriginal Medical Services: potential to underestimate the performance of primary health care

Isaac Hill , David Johnson , David Scrimgeour , Robyn McDermott

Abstract

Objective: To assess the accuracy of current methods of extracting national Key Performance Indicator (nKPI) health data via the OCHREStreams program from Communicare, a commonly used patient information management system (PIMS) in Aboriginal primary care. Methods: Two Aboriginal Community Controlled Health Services (ACCHSs) were recruited for this study. A sample of regular clients aged 55 years from each ACCHS was selected and a subset of thirteen nKPIs were examined. A manual case note audit of the nKPI subset within Communicare was undertaken by a clinician at each participating ACCHS and acted as a ‘gold standard’ comparator for three query methods: (1) internal Communicare nKPI reports; (2) PenCS CAT4 nKPI reports (a third party data extraction tool) and; (3) nKPI data submitted to the Improvement Foundation (IF) Web Portal. Results: No errors were found in the nKPI data extraction from Communicare by Pen Computer Systems’ Clinical Audit Tool 4 (PenCS CAT4) and the subsequent submission to the IF Web Portal. The Communicare internal nKPI report, however, included deceased clients and past patients and we can be very confident that deceased clients and past patients are also included in the IF Web Portal data. This resulted in inflation of client denominators and an underestimation of health service performance - particularly for nKPIs recording activity in the last 6 months. Communicare internal nKPI reports were found to be reasonably accurate although a number of minor errors were detected. Conclusions: PenCS CAT4 accurately extracts a subset of nKPI data from Communicare. Given the widespread use of Communicare in ACCHSs, however, the inclusion of deceased clients and past patients in the OCHREStreams program is likely to have resulted in systematic under-reporting of health service performance nationally through this program.

AH16251  Accepted 28 March 2017

© CSIRO 2017