Australian health service organisation assessment outcome data for the first 2 years of implementing the Comprehensive Care Standard

Keywords: accreditation, health data, health standards, healthcare, outcome measurement, patient safety, quality of care, risk management

Patients in Australia receive health care that has been ranked highly for safety and quality when compared to other countries.¹ There are around 11 million patient separations per year in Australian public and private hospitals.² Most patients receive safe care; however, more than 180 000 hospital-acquired complications are recorded in admitted patient data each year. In 2016–17, more than 1000 serious clinical incidents were reported, indicating further initiatives could improve the safety and quality of Australian health care.³

The Australian Commission on Safety and Quality in Health Care (the Commission) is a national agency with functions specified in the National Health Reform Act 2011,⁴ including promotion and support for implementation of safety and quality matters in health care, and collection, interpretation and dissemination of safety and quality healthcare data. The Commission is responsible for the development of standards, including the National Safety and Quality Health Service (NSQHS) Standards that apply to a wide variety of health service organisations (HSOs) operating across a range of settings. Part of this responsibility includes coordinating the Australian Health Service Safety and Quality Accreditation (AHSSQA) Scheme.⁵

Standards are recognised as a method for improving the provision of health care.⁶ Since 2013, the NSQHS Standards have been used as a framework for improving the quality of health service provision in Australia, and minimising the risk of harm to patients.^7,8 Evaluation of the first edition of the Standards demonstrated improvements in some patient outcomes.⁹ The increasing complexity and frailty of patients at risk of adverse events was a recognised safety and quality gap.⁹ Identification of these gaps contributed to the development and inclusion of the Comprehensive Care Standard (CCS) in the second edition.⁸

The actions in the CCS aligned with the best available evidence on areas of greater risk of harm to patients and were developed in collaboration with the Australian Government, states and territories, the private sector, clinical experts, patients and carers.⁸ The actions in the CCS strengthen the role of consumers and carers in making decisions and setting goals for their health care. This approach of providing care that meets the clinical and personal needs and goals of individual patients is an expectation of the Australian health system.

There are eight NSQHS Standards in the second edition, with 148 associated actions. Accreditation to the NSQHS Standards every 3 years is mandatory for all public and private hospitals, day procedure services and most public dental practices in Australia.¹⁰ To achieve accreditation, HSOs need to meet the NSQHS Standards. Accreditation may still be granted with some actions met with recommendations. Actions not met may be remediated by HSOs within specific timeframes to complete assessment.¹⁰

The number of HSOs requiring accreditation fluctuates. In December 2020, 1323 HSOs required accreditation by assessment to the NSQHS Standards. Commission advisories provide context about assessment to the actions in different circumstances. Assessment is undertaken by one of seven Commission-approved independent accrediting agencies. Agencies are responsible for ensuring assessors understand the requirements of rating hospital performance. Ratings are applied to each action in the Standards (Table 1). Completed assessment outcome data are submitted to the Commission and collated in a secure database. Assessment to the CCS commenced in January 2019. The CCS includes 36 actions listed under four criteria (Table 2), totalling 24% of the NSQHS Standards actions.⁸

Table 1.  Definitions of ratings for each action in the NSQHS Standards.¹¹

Table 2.  Actions and criteria of the Comprehensive Care Standard.

Healthcare outcomes are not the same for all Australians, although health care is generally considered high-quality when measured by life expectancy and other clinical indicators.¹² The CCS aims to ensure that patients receive safe, effective care that aligns with their needs and preferences. The actions in the CCS aim to promote continuous service improvement and mitigate risks in areas where patient harm continues.

Resources were released between 2016 and 2020 to support CCS implementation. Existing related resources were already available. The implementation resources included the foundational, cultural and organisational aspects important to implementation. The resources were derived from anticipated need, by reviewing common queries submitted to the Commission, and stakeholder consultation. Consumer resources were also developed for specific issues identified by consumer representatives.

This paper describes a retrospective examination of CCS assessment outcome data to identify areas of underperformance (actions not met and met with recommendations) and potential misinterpretation of the requirements (inappropriate application of ratings). This was part of a larger strategy to evaluate CCS implementation, which included qualitative components and evaluation of resource use. Post hoc evaluation prior to completion of the 3 year accreditation cycle was part of the Commission’s organisational work plan for comprehensive care.

A retrospective examination of assessment outcome data routinely submitted by accrediting agencies to the Commission was undertaken as a measure of progress of CCS implementation. The recorded status of actions in the CCS at the completion of initial assessment to the NSQHS Standards for HSOs between January 2019 and December 2020, was retrieved from the Commission database for analysis.

The number of CCS actions met, not met, met with recommendations and not applicable were calculated. Not applicable actions were reviewed to assess compliance with the relevant advisory.¹³ Non-compliant actions assigned not applicable were coded as inappropriate.

Ethics approval was not sought as the intended purpose of the data analysis was for quality improvement. This was a retrospective examination of de-identified assessment outcome data routinely submitted to the Commission by accrediting agencies. There was no testing of novel policies or departure from standard care. The data related to HSOs and no individual patients or minority or vulnerable groups were identifiable.

A total of 495 assessments relating to 468 unique HSOs were undertaken between January 2019 and December 2020. Although all Australian states and territories were represented in the assessment outcome data, 24 HSOs were registered for short notice assessment that may not have included assessment to the CCS. Most (71.1%, n = 352) assessed HSOs met the requirements or received not applicable status for the 36 actions of the CCS after completion of initial assessment (Fig. 1, Table 3). For the remaining assessments, CCS actions were not met (11.7%, n = 58) and met with recommendations (14.9%, n = 74) (Table 4).

Fig. 1.  Percentage of HSOs that met the actions in the NSQHS Standards on initial assessment. HAI, Healthcare-Associated Infection; RR, Recognising and Responding.

Table 3.  Outcomes of initial assessment: total assessments.^A

Table 4.  HSO type and initial outcome.

There have been 17 820 CCS actions assessed during the 495 assessments. CCS actions were not assessed (n = 866) in most of the HSOs that undertook short notice assessments (n = 24).

All 36 actions in the CCS were rated not met or met with recommendations by at least one HSO. The actions not met, met with recommendations and inappropriately assigned not applicable are outlined in Fig. 2. All actions are accounted for in at least one category. The not met rating (assigned 325 times) was issued for all but one action (Preventing falls and harm from falls 5.25), met with recommendations (assigned 331 times) was issued for all but five actions (collaboration and teamwork 5.05, documenting advance care plans 5.09, preventing and managing pressure injury 5.21 and 5.23, preventing falls and harm from falls 5.24) and not applicable status (n = 3028) was issued for all but 12 actions (5.01–5.08, 5.11–5.14) and was inappropriately assigned at least once for 23 actions.

Fig. 2.  CCS actions rated not met and met with recommendations (reccs) and non-compliant – not applicable.

Appropriately assigned not applicable ratings most often related to minimising restrictive practices-seclusion (5.36 n = 265), followed by end-of-life care actions including identifying end-of-life 5.15 (n = 177), accessing specialist palliative care 5.16 (n = 191), accessing supervision and support 5.18 (n = 190), reviewing care provided 5.19 (n = 189), and shared decision -making 5.20 (n = 178). The not applicable rating included 208 occurrences (9%) that were not compliant with the advisory¹³ and therefore coded inappropriate. Inappropriately rated actions also often concerned end-of-life care. These included action 5.15 (n = 17), 5.16 (n = 25), 5.18 (n = 24), 5.19 (n = 24) and 5.20 (n = 14). Action 5.31, predicting, preventing and managing self-harm and suicide, was also represented 15 times.

Examination of the assessment outcome data provided a preliminary review of CCS implementation, representing 35.4% of Australian HSOs requiring accreditation. Most HSOs (71%) met initial assessment requirements for the CCS more often than the Clinical Governance, Medication Safety and Preventing and Controlling Healthcare-Associated Infection Standards. An anticipated wash-in period, planned implementation, and actions already part of health service provision may have contributed to this outcome. Prior to implementation, the NSQHS Standards (2nd edn) were released for public consultation with extensive lead-time to provide opportunities for change.

Gap analysis for specific actions was a requirement of the staged implementation approach set out in advisories.^14,15 Initial consultation highlighted the potential inability of HSOs to meet actions related to risk screening and assessment, multidisciplinary teamwork, and comprehensive care plans. Resources supporting these actions were developed and released between 2016 and 2018. Results demonstrated that three of the top five actions not met and met with recommendations were from the Minimising patient harm criterion. Action 5.5 requiring teamwork and collaboration was only allocated not met twice. The advisories^14,15 allowed additional time for HSOs to implement actions identified as problematic, with scheduled milestones up to 2022 when full implementation was expected. The date was extended to 2023 as a result of the COVID-19 pandemic. Accreditation was suspended for much of 2020 and HSO accreditation status was maintained during the pandemic response. Hybrid assessments were also permitted to complete initial assessment for HSOs that were not accessible to accreditation teams. There was no capacity to separate the data in this evaluation.

Integrating clinical governance and applying quality improvement systems (5.01 and 5.02) were the most frequently not met actions. This aligns with the poor performance at initial assessment in relation to the Clinical Governance Standard, which was only met by 45% of HSOs. Clinical governance is an integrated component of corporate governance. It ensures that frontline clinicians, managers and board members are accountable to patients and the community for assuring the delivery of safe care.¹⁶ The relevant CCS actions are integral to ensuring organisational culture and governance supports comprehensive care delivery, and were described in the earliest tranche of implementation resources.¹⁷ Website analytics data from the Commission indicates that although the CCS is highly accessed, the implementation resources are not. Increased use of these resources could support improvement.

The CCS promotes continuous quality improvement in person-centred care. It does not prescribe condition-specific care activities, although the Minimising patient harm criterion includes requirements related to high-frequency adverse events. None of the CCS actions were met by every HSO, including actions relating to pressure injuries and falls. Both were previously addressed as separate Standards to which many organisations were assessed. Pressure injury and falls prevention were encompassed by the CCS with significantly truncated actions.

Falls¹⁸ and pressure injuries¹⁹ continue to be a major source of harm for patients in hospital. These risks were added to a national list of Hospital-Acquired Complications (HACs) made up of 16 agreed, high-priority complications, which clinicians, managers and others can address and improve patient care. Action 5.22 preventing and managing pressure injuries appears in the top five not met actions and action 5.25 related to falls prevention appears in the top five met with recommendations. The actions most frequently requiring remediation are likely to change as the accreditation cycle continues and more HSOs are assessed.

There are a number of HACs defined and reported from the administrative data that could be related to the CCS. HAC data could not be linked for this evaluation. In the 2019–20 financial year, the HACs rate for pressure injury was 43.8 per 100 000 separations, for falls 43.7 per 100 000 separations and delirium 357.1 per 100 000 separations. The CCS could be strengthened by linking with programs such as HACs funding penalties.²⁰ The actions in the CCS are unlikely to fully reduce harm to patients from these adverse events and the funding penalty is intended to improve outcomes by encouraging strong risk management processes. Clinical care standards may also have a role in supporting CCS implementation.

Day procedure services were slightly more likely to meet the actions at initial assessment, although no statistical test was performed. This is probably due to the number of actions that are permitted a not applicable rating. The majority of not applicable ratings were allocated in day procedure services. There were 208 inappropriate not applicable ratings,¹³ mostly related to end-of-life care. The end-of-life care actions were also highly represented in the not met, met with recommendations, and not applicable ratings. The only end-of-life care action where HSOs were performing consistently was receiving and documenting advance care plans (5.17). The number of non-compliant not applicable end-of-life care actions increased in April 2020. This was the result of an update to the advisory¹⁴ where some actions that did not previously apply to day procedure services were reconsidered. The end-of-life care actions were included in the CCS to support implementation of the National Consensus Statement: essential elements for safe and high-quality end-of-life care.²¹ End-of-life care is a recognised area for improvement in Australia,²² which is demonstrated in the assessment outcome data. Although it is unlikely that day procedure services frequently provide care to patients in their last days, some procedures may be appropriate for people who are at the end of life. Day procedure services should have capacity to meet the specified actions when these patients are admitted. Increased oversight of the application of the not applicable rating may also decrease the variability of assessments between HSOs.

The accreditation scheme was reviewed in 2018 to improve reliability of accreditation processes. This necessitated changes to the way in which assessor teams conduct assessments. Assessors would have been developing assessment skills required by the changes; however, it is difficult to assess what effect this had on the assessment outcome data. Assessments are undertaken by different assessor teams and some variation in the outcomes was expected. Additionally, there was no capacity to determine variation between assessors and accrediting agencies in their application of ratings. The advisories and appeals process provide a balancing feature to the AHSSQA scheme. At the time of this evaluation, all HSOs had been awarded accreditation. Accreditation is important to ensuring quality and safety, but is not the panacea to preventing all patient harm. Repeated review of the assessment outcome data would provide a more complete understanding of progress towards CCS implementation and changes that may be required in future editions of the NSQHS Standards.

The CCS had 71% of actions met by the 35.4% of Australian HSOs assessed during the first 2 years of implementation of the NSQHS Standards (2nd edn) and performed well compared to other Standards. There was some indication of underperformance, particularly with regard to clinical governance systems that support delivery of comprehensive care. Measuring the implementation of the CCS is complicated and further investigation is needed to understand the extent of variation and attributable factors. Some actions in the CCS are linked to additional measures and incentives that also drive change, such as inclusion in the HACs list. Future evaluation of the implementation of the CCS may benefit from strengthening links between these synergistic quality and safety programs. To improve the implementation of comprehensive care within HSOs, effort should focus on clinical governance systems that support organisational safety culture, teamwork and clinicians to deliver comprehensive care.

The data for this article are not available in a public repository. Data will be shared by request to the corresponding author pending permission from the Australian Commission on Safety and Quality in Health Care.

The authors declare no conflicts of interest.

This research did not receive any specific funding.