Assessing the applicability of the Model for Understanding Success in Quality (MUSIQ) for primary care: a multi-case mixed methods analysis
Jane Cullen
1 2 * , Nihal Jayamaha 3 , Paul Childerhouse 4 , Lynn McBain 5
1
2
3
4
5
Handling Editor: Tim Stokes
Abstract
Effective quality improvement (QI) is vital to improve healthcare quality and outcomes. The context surrounding QI has a dynamic relationship that impacts QI efforts over time. Developing an understanding of context may provide opportunities to address barriers to success. Most studies into contextual influences have been conducted in secondary care. Primary care is key to improving population health, equity and value. The Model for Understanding Success in Quality (MUSIQ) is a commonly used context assessment tool that was developed in large, mainly secondary care organisations, and questions have been raised as to its applicability in primary care.
This study aims to assess the applicability of the MUSIQ in primary care settings and suggest adaptations for primary care.
A multi-case mixed methods approach was followed with quantitative data from the MUSIQ survey tool, compared with qualitative data from Aotearoa New Zealand primary care interviews. The Consolidated Framework for Implementation Research (CFIR) guided qualitative data collection and analysis.
Scores were duplicated between common microsystem and organisational factors. External motivation created mixed reactions and scores depending on interpretation, but consistent themes of community responsibility and network relationships were identified as strengths. A lack of QI infrastructure and triggering events were consistent findings.
Total MUSIQ scores were impacted by scores given for factors at different MUSIQ levels that are combined in many primary care organisations. A primary care adaptation of MUSIQ is proposed that removes the duplicated levels and adds key primary care contextual factors not included in MUSIQ.
Keywords: barriers and facilitators, context, general practice, implementation science, model for understanding success in quality, MUSIQ, primary care, quality improvement.
| WHAT GAP THIS FILLS |
| What is known about the topic: Variations in context are thought to drive the variable outcomes from quality improvement in health care. Despite the importance of primary care to the health system, research into primary contextual factors and tools to understand and measure factors in primary care are rare. |
| What this study adds: This research evaluates the use of MUSIQ in primary care and provides direction for improving its applicability in this area. Duplicated levels have been removed and specific primary care factors added to improve applicability, particularly for small to medium-sized general practice settings. The primary care adaptation of MUSIQ can be used to assess the primary care context surrounding quality improvement efforts and guide action to improve the likelihood of success. |
Introduction
Healthcare quality improvement (QI) is vital as there are too many examples of poor access, inequity, harm, unwarranted variation, and care deviating from evidence-based knowledge.1,2 Research identifies context as contributing to the variation in QI outcomes.3–5 The context surrounding QI has a dynamic relationship with the intervention and implementation process over time. Developing an understanding of context may provide opportunities to address barriers to success6–10 and/or choose contextually appropriate approaches.5,6
Contextual assessment and measurement instruments have been developed so this knowledge can be used to understand and modify contextual factors for QI success.6–11 The Model for Understanding Success in Quality (MUSIQ) is a model widely used by QI practitioners to assess contextual factors.7,12 In MUSIQ, context is defined as everything that is not part of the intervention and the implementation process.13
It is recognised that ‘high-quality primary care is the foundation of a robust health care system’, critical for achieving health of the population, addressing inequities and improving care experience, system value and sustainability.14 Most research into contextual factors influencing QI success is conducted in large, predominantly secondary care organisations where contextual factors are likely different to those in more numerous primary care settings with different funding, structures, culture, care delivery models and aims.15–18
MUSIQ was developed based on quantitative data from predominantly hospital-based studies and provides statistical analysis of the relationship between contextual factors.8 MUSIQ has not been widely tested in primary care.12,13 Therefore, this study asks:
Is the Model for Understanding Success in Quality (MUSIQ) applicable in primary care?
How might the MUSIQ be adapted for use in primary care?
Theoretical background
Implementation science has generated a wide range of heuristics to understand or explain the determinants of implementation outcomes with subsequent attempts to provide consolidated models, taxonomies, templates, checklists and selection guides.19–25 Context is a common factor across these heuristics but one that is variably defined and described.23,25 This contributes to variation in the associated factors, frameworks, measures and measurement tools.20,22,25,26 The number of heuristics developed attempting to explain the role of context in QI demonstrates the complexity and variation inherent to the topic.5,10,19,22,25 A systematic review of measurement tools of context in primary care found few validated tools for primary care and less coverage of identified contextual factors.26
The Consolidated Framework for Implementation Research (CFIR) is a theory-based determinant framework20 frequently used to assess contextual factors affecting implementation in health care.27,28 It has proven applicability in primary care with use in pre-, during- and post-QI initiative implementation29–36 including in Aotearoa New Zealand (NZ).35,36 The CFIR has been used for qualitative studies, mixed methods, quantitative studies37 and in a hospital-based study as a qualitative tool to compare with quantitative MUSIQ data.7 The broad coverage and applicability of CFIR as a qualitative assessment tool allows for a deeper understanding and comparison of contextual factors relevant to primary care that may not be included in the MUSIQ instrument.
MUSIQ is one of few healthcare models where relationships between contextual factors and improvement outcomes have been explored.8,10,38 It is theory-based, with well-defined constructs built from a systematic review of quantitative studies and structured expert opinion.10,13,38,39 MUSIQ has been used for qualitative evaluation within secondary care,38,40 the health system,41 mixed-methods evaluation of a collaborative initiative7 and as a categorisation framework for systematic review.42 Within these studies, only the evaluation by Dewan et al. used quantitative MUSIQ scores.7 Recent reviews of MUSIQ conducted by secondary analysis of hospital QI through a complexity lens produced additional constructs and levels of context,10 and an adapted version for low-income countries.43 However, the applicability of MUSIQ in primary care is still in question.
Methods
A multi-case mixed methods approach was followed as quantitative and qualitative methods together (mixed methods) have potential to provide a more complete picture of context and are frequently used in QI and implementation science.3,4,22 Cases were purposefully selected to include high-performing general practice and Primary Health Organisation (PHO) settings typical of NZ primary care. Case study sites chose a completed successful QI intervention (QII) for this study. Case characteristics and interview participants are outlined in Table 1.
| Case 1 (C1) | Case 2 (C2) | Case 3 (C3) | Case 4 (C4) | Case 5 (C5) | Case 6 (C6) | ||
|---|---|---|---|---|---|---|---|
| Organisation | A PHO and VLCAA practice co-located with Community Mental Health and other health services | A kaupapa MāoriB VLCA practice | An integrated family health service | General practice | A kaupapa Māori PHO with varied primary health care services including 5 VLCA general practices | A general practice co-located with a range of other health services | |
| Setting | Remote rural practice | An urban practice within a satellite town of a large city | An urban practice in a large city | Urban practice in a large city | Sited in urban suburbs surrounding a large city | Urban practice in a large city | |
| Practice size (100th percentile is the largest at approx. 29,000 patients) | Approx. 6000 enrolled patients | Approx. 3000 enrolled patients | Approx. 6000 enrolled patients | Approx. 13,000 enrolled patients | PHO approx. 18,000 enrolled patients. Practices: 2500–5500 | Approx. 20,000 enrolled patients | |
| 76th percentile | 40th percentile | 76th percentile | 95th percentile | From 38–67th percentile | 99th percentile | ||
| FTE unless otherwise stated | 7 general practitioners (GPs) employed over varying hours | GP: 2.5 | GP: 5 | Head count of 13 GPs with various FTE | Practice headcount with various FTE: | Chief executive | |
| Nurse practitioner: 1 | GPs: 25 | GP director | |||||
| Practice nurses: 2 | PNs: 30 | Finance/admin: 2 | |||||
| Community health team 3.6 | Practice Nurses (PNs): 2.35 | PHO: Approx. 120 staff in a range of roles | GPs: 11.9 | ||||
| PNs: 9.5 | |||||||
| Reception: 10 | |||||||
| Interview participants | PHO clinical director | GP director | GP director | Clinical nurse lead | Quality leader | General manager | |
| CMH district manager | Practice manager | DHB service integration facilitator | GP director | Clinical lead nurse | Medical director | ||
| General practice (mental health) liaison nurse | Nurse lead | PHO project manager | Clinic RN | Hub senior lead | GP Associate | ||
| Practice nurse | Administration team lead | Practice nurse | General manager | Hub administrator | |||
| General practitioner (GP) | Community health worker/receptionist | Receptionist | Administration manager | ||||
| Community health workerC | PHO data analyst | ||||||
| Practices nurses (2)C | PHO nurse director | ||||||
| Intervention | Physical health for patients with a Mental Health diagnosis | Telephone triage project | Patient prioritisation at reception | Nurse-led acute paediatric (initially 13 years and under) clinic | A central telephone call answering and appointment booking service | Model of care change from acute care and general practice to general practice only | |
| Date of QII | 2017 | 2014 | 2017 | 2018 | 2018 | 2018 | |
| Date of interviews | October 2018 | October 2018 | January 2019 | August 2019 | January 2020 | December 2019 |
The qualitative data collection methods and the consent process have previously been presented in a 3-case comparative study45 and 6-case comparative study46 and were based on onsite interviews utilising CFIR to guide data collection and analysis.20 CFIR was used to identify primary care contextual factors because of the frameworks’ broad coverage, its similarities with MUSIQ13 and its applicability to primary care.29–36
The quantitative data were collected via the MUSIQ excel calculator (Cincinnati Children’s Hospital Medical Centre, CC BY 3.0), attached in Word format in the Supplementary material, Section S1 and is also available online7 (https://qi.elft.nhs.uk/resource/the-model-for-understanding-success-in-quality-2/). Participants completed the MUSIQ survey post-interview, choosing who would participate and how to complete it (individually or group).
The qualitative comments were first themed inductively, then deductively according to the CFIR codes and MUSIQ factors, which were then compared. MUSIQ domains were mapped to CFIR domains to aid consistency during analysis. Alignment between MUSIQ factors and the CFIR domains and subconstructs are shown in Table 2. CFIR domains and constructs that do not align to the MUSIQ factors are not shown. The MUSIQ data and CFIR codes were compared at each stage of thematic analysis, during within-case analysis and again at cross-case analysis.
| MUSIQ system levels | Factors | CFIR domains | Constructs | |
|---|---|---|---|---|
| Environment | External motivators | Outer setting | Peer pressure | |
| External project sponsorship | External policy and incentives | |||
| Intervention construct ‘Intervention Source’ aligns to external project sponsorship | ||||
| Organisation | QI leadership | Inner setting | Culture | |
| Senior leader sponsor | Implementation climate | |||
| Organisational QI culture | Tension for change | |||
| QI maturity | Relative priority | |||
| QI support and capacity | Data infrastructure | Organisational incentives and rewards | ||
| Resource availability | Goals and feedback | |||
| Workforce focus on QI | Learning climate | |||
| Microsystem | QI leadership | Readiness for implementation | ||
| Motivation | Leadership engagement | |||
| QI capability | Available resources | |||
| Microsystem QI culture | Access to knowledge and information | |||
| QI Team | Team leadership | Characteristics of individuals | Knowledge and beliefs about the innovation | |
| Team diversity | Self-efficacy | |||
| Team subject matter expertise | Individual stage of change | |||
| Decision making process | Implementation processA | Opinion leaders | ||
| Team norms | Champions | |||
| QI skill | Reflecting and evaluating | |||
| Prior QI experience | ||||
| Physician involvement | ||||
| Team tenure | ||||
| Miscellaneous | Task strategic importance to the organisation | Inner setting | Relative priority | |
| Triggering event |
Results
A total of 33 participants were interviewed from practices and PHOs participating in six QI projects. A subset of participants completed the MUSIQ tool (14 out of 33) who were given the MUSIQ definitions to consider beforehand and while answering questions.
Cross-case analysis is reported under the MUSIQ headings, comparing MUSIQ and CFIR results. The MUSIQ interpretation guide and quantitative results are shown in Appendix 1. C1 scores are shown for the DHB, and the PHO and practice participants and are an agreed score, as is C2 (2 participants), C3 (4 scores averaged) and C4–6, which are single participant scores. The MUSIQ survey is made up of 35 questions that are grouped to provide scores for the contextual factors, with a maximum possible score of 7. The overall MUSIQ scores are between 125 and 140 out of a possible 168. Participants QIIs were scored retrospectively, and all are within the highest range. Representative interview quotes themed to MUSIQ factors and some additional CFIR constructs are shown in Section S2 of the Supplementary material.
Outcome
This study was based on participants perception of QII success and was not scored. Cases provided data to support achievement of the QII aims and other related benefits that demonstrate success.
Context
The factor ‘external motivators’ was understood and interpreted differently by the teams, which affected scoring. Some cases included patient need in their score for this factor and scored this highly. Lower scoring cases were C2, a Kaupapa Māori organisation that did not view patients as external to the organisation and disliked ‘incentives’, C3 and C6 also felt they did not need external sponsors or incentives. The CFIR construct of ‘patient needs and resources’ was identified as a key factor, ‘If it’s not good for patients, why are we doing it?’
Case 1 was the only team with financial sponsorship, but most teams had some degree of external support and/or team participation or collaboration from the PHO and/or DHB (C5 is a PHO). This is shown in Table 1 and contributed to positive scores for external sponsorship. High scores for external sponsorship were associated with cases where support was provided from the PHO (C4) and Healthcare Home Collaborative (C5), even though there was no financial support. In C6 the score of 1 reflects that external sponsorship was not desired or required.
The CFIR construct ‘cosmopolitanism’ refers to the degree to which organisations are networked with other organisations.20 All teams had strong networks with national groups, PHOs and/or DHBs and within local communities. These collaborative relationships differed from external project sponsorship and were an important factor in most of these QIIs and how all cases operated. C2–C4 and C6 have staff in governance positions at the PHO or the DHB. Several of these practices provide examples of success within their networks. C1, C3, C5 and C6 are integrated health services working alongside other primary health services.
MUSIQ includes QI leadership and QI culture at both organisation and microsystem levels, and in smaller primary care organisations, there is no separation between microsystem and organisation. Similarly, QI maturity (sophistication of the QI programme) at the organisation level and capability for QI (staff ability to use QI for change) had substantial overlap in the smaller organisations.
QI workforce focus (training and reward systems to support QI)13 did not score highly. Only C4 had received training in QI, and no case rewarded QI, although it was encouraged. C4 and C6 were both part of a local primary care QI programme that ‘did quite a bit with the PDSA improvement methodology’ and several C4 participants had completed a short course on QI.
Microsystem QI leadership was generally scored highly. The high scores for QI culture duplicate the high culture scores for organisational QI culture as, apart from C1, the microsystem and organisation were the same. There was some variation in motivation scores as C1, C3 and C4 reported the microsystem (or organisation) outside the QII team as less motivated than the QI team. QI capability was the lowest score in this section. Only C3 had some confidence in their abilities to use QI methods, with an average score of five, and C4, the only case to use formal QI methods, scored themselves three.
MUSIQ measures sources of motivation both external to and within the microsystem. This caused some confusion in the participants who felt intrinsically motivated by their sense of responsibility for the communities they care for and did not regard the community as external.
QI team scores were generally high, which included leadership, diversity, subject matter expertise and physician involvement. This is reflected in the mixed team membership shown in Table 1. The lowest scoring factor in this group was for QI skill, with C1 and C5 only scoring 2 and 3 for the use of technical QI skills. However, as they had all been working to improve the quality of healthcare services for some time, they rated prior QI experience more highly. All teams had capable members and leaders with confidence in leading change. Both C4 and C6 had team members with experience in formal QI, but it was only C4 that applied formal QI methods.
Team decision-making, team norms and team tenure received high scores reflecting established team relationships and collaborative decision-making, a feature of teamwork that was supported by the qualitative data. Teamwork was commented on by every team, and several teams spoke of having a family-like relationship. In C2 and C5 the unique cultural characteristics of kotahitanga (unity, strength within, solidarity and accord) and whanaungatanga (relationships and kinships) supported strong internal and networked community relationships. These cultural perspectives are not able to be captured by a Western-based measurement tool. Every team agreed on the importance of team leadership and described shared or distributed leadership models involving both formal and informal leadership that was reflected in the high scores given for QI Team Leadership and Diversity but is not fully captured in the leadership factors defined in MUSIQ.
An inner setting team and organisational theme of community responsibility was identified that was not included in either MUSIQ or CFIR. Community responsibility was expressed by each case and the need to do this work ‘to address the needs of the community that we serve.’ The Kaupapa Māori organisations felt an even deeper connection, described as manaakitanga (value, respect and care for) and whanaungatanga.
This section includes the tasks of strategic importance to the organisation and the presence of a triggering event prompting the QII. All teams assigned a score of one for ‘triggering event’, indicating that this was not a factor. The QIIs were internally motivated, reflected in the high score for strategic importance.
Patient needs and resources and a deeper form of cosmopolitanism were identified as important factors underpinning QII success. These are CFIR constructs that are not included in MUSIQ. The community responsibility underpinning high levels of motivation to succeed is not part of either model, although motivation (MUSIQ), compatibility and relative priority (CFIR) are related. Table 3 shows these additional two CFIR constructs and one new construct.
| MUSIQ/CFIR Domains | Construct | |
|---|---|---|
| Environment/outer setting | Patient needs and resources | |
| Cosmopolitanism (deeper) | ||
| Microsystem/inner setting | Community responsibilityA |
Discussion
This study used the quantitative MUSIQ tool alongside CFIR to qualitatively assess the applicability of MUSIQ to primary care. Qualitative data guided understanding of the thinking behind participant scores and allowed exploration of additional contextual factors, demonstrating the utility of mixed methods design. MUSIQ has recently been revised,10 referred to as MUSIQ2.0, and we include consideration of MUSIQ2.0 changes in this discussion.
The overall MUSIQ scores are high, indicating likelihood of success because of the strength of the strong human-centred contextual factors of QI culture, team leadership, diversity, tenure and norms. However, the multiple levels attracted several scores for the same item when those different levels were not present. In the small to medium-sized practices, QI experience and leadership were shared and distributed throughout the team without the three levels defined in MUSIQ. A primary care adaptation of MUSIQ is proposed in Fig. 1 that combines the duplicated features between organisation and microsystem to increase applicability in the small to medium-sized practices common in NZ, definitions of the factors are provided in Section S3 of the Supplementary material.
External environment
MUSIQ2.0 has two new external contextual factors: QI and implementation (QI&I) external knowledge-general, and -project specific, regarding the ‘extent to which the team or organisation values and acquires knowledge from external sources’10, (p.5). This study and other primary care research support the ‘QI&I external knowledge-general’ factor, while finding that these beneficial, trusting and two-way relationships and whanaungatanga take time to build.47,48 The adapted version of MUSIQ (Fig. 1) adds Network relationships specific to QI (but not project specific) to external factors. This is defined as: The degree to which an organisation and its staff are networked, combining complementary strengths with other external organisations, their local community and population to collaborate for QI and draws on the work of Coombe et al.49 This is similar to, but distinct from, cosmopolitanism20 and the MUSIQ2.0 factors of external knowledge-general and -project specific.10
Network relationships provide resources in knowledge, connections to others working on similar work and, in some cases, data, and this is shown by the arrow from network relationships to QI support and capacity as well as the organisation/microsystem. This was identified in our cases from the interconnectivity present with local community and other health and social care organisations, including PHOs and professional communities.
External motivators are relevant, although some intended motivators or incentives can disincentivise primary care.50,51 The triggering events factor was not present in any of the cases and was not valued by participants. A previous external factor, ‘task strategic importance’ was found to be an organisational factor.10
Organisation/microsystem
The key primary care strengths of distributed leadership, QI culture, team relationships and motivation engendered by community responsibility mitigated the identified weakness in formal QI capability. Formal QI methods were only applied by C4, where several staff are trained in QI, and they were initially supported by an improvement advisor. This is reflected in the low MUSIQ scores for microsystem QI capability by all teams and by C1 and C5 for QI team QI skill, who had the most exposure to QI methods.
Patient engagement was not included as a factor in MUSIQ, but it has been added as a team factor in MUSIQ2.0,10 reflecting the shift in healthcare QI thinking and practice.52 Consumer engagement and co-design are an important aspect of NZ healthcare QI, but this knowledge was still developing and not a factor in these cases, although a focus on patient needs was expressed in every case. The patient needs and resources construct from CFIR was added to organisation/microsystem as this was a key motivator and guided QII selection. Participants identified it as an inner setting factor rather than an outer setting factor.
QI team
Teams described ‘family-like’ relationships, kinship, unity and togetherness, expressed in the Kaupapa Māori organisations as whanaungatanga and kotahitanga. This is not included in any of the definitions of MUSIQ team factors, thus was unable to capture the essence of team strengths and teamwork characteristics described. Team tenure, diversity, norms and decision-making processes all scored highly and were important.
QI support and capacity
The two-way arrows between Microsystem/Organisation (MS/O) and QI support and capacity reflect the interrelationships between several of the factors and that QI support and capacity comes from the organisation, which is also the microsystem in small-to-medium sized primary care organisations.
MUSIQV2.0 has the additional infrastructure factor of specialist QI&I staff and QI&I portfolio management,10 both are extremely rare in the small to medium-sized organisations that figure predominantly in NZ general practice and are not included in the adapted model. Workload has been identified as a barrier to QI, and the supportive QI infrastructure, including personnel, that exists in hospitals is rarely present in primary care.53,54 Instead, teams relied on the varied personal skills and experience of team members and supportive networks.
Implications for primary care
Team members found value in the thinking and conversation generated by completing the quantitative snapshot of context, and, as intended, it highlighted modifiable aspects of context for their QII in their setting. The MUSIQ principles and factors, such as leadership, motivation and QI culture, are applicable in primary care. Studies across other sectors agree on the importance of leadership and a QI culture,55 but do not consider motivation from community and patient need. Patient need is a key driver in primary care where the relationship is usually longitudinal, continuous and provides motivation for improvement.56 Community responsibility, a unique primary care factor that is particularly strong in Indigenous communities45,46,48 and connected to patient need, is not included in MUSIQ. However, consideration of motivation at different parts of the system is applicable and useful.
The structures and relationships within and between the inner and outer settings differ in primary care; some of the layers are not present and external sponsorship can receive a mixed reception in primary care.50,51 As Kaplan et al.13 suggest, the multiple levels and roles within the MUSIQ were difficult to apply in primary care and created confusion and duplicated scores.
Network relationships may provide missing infrastructure factors, with shared knowledge and resources in reciprocal two-way relationships supporting spread and sustainability of improvement. This is a key enabler in primary care, particularly in under-resourced settings.48,57 Internationally, investments have been made in, and via, network relationships to provide the QI infrastructure and expertise that is not often present in small to medium-sized practices.16,58
Strengths and limitations
This is one of few attempts to study the applicability of MUSIQ in primary care, thus building the understanding of how context influences QI success in primary care and proposing a primary care adaptation of MUSIQ. Limitations of this study include the case sample being of only successful projects, meaning that it is unknown how these factors might operate in unsuccessful primary care projects. Only six NZ cases were studied, and with the heterogeneity of primary care, applicability will need to be evaluated from site characteristics and tested in other settings.
Another limitation is the appropriateness of using Western models such as CFIR or MUSIQ to understand Māori QI approaches. This risks misinterpretation, misrepresentation and devaluing of Indigenous knowledge, and one should be cautious interpreting the findings in Indigenous settings. This risk was mitigated by the feedback from these teams but still exists as only the experience of being Māori can allow a full understanding of Māori concepts and ways of viewing the world.
Future directions
Further research is required to test this primary care model on successful and unsuccessful primary care QI efforts, in particular the values assigned to the likelihood of successful projects. Is the removal of triggering events supported by a larger sample? Is the best practical use of the MUSIQ as a combined qualitative and quantitative tool or only one or the other? Are the different uses of the tool more applicable at different levels of the system, eg microsystem: qualitative and meso- or macrosystem: quantitative? These questions all prompt avenues for future research.
Conclusion
MUSIQ can be a useful tool to guide thinking about enablers and barriers to QI in primary care, with a view to enhancing enablers and acting on barriers before commencing a QII. However, we caution against making comparisons based only on numerical MUSIQ scores, with the variable estimation of QI skills noted here and elsewhere.53,54 Instead, use these tools to understand local strengths, enablers and barriers and collectively plan to use strengths and enablers to mitigate and reduce or remove barriers.
Data availability
Deidentified interview transcripts are available on request from the lead author, ORCID: https://orcid.org/0000-0002-0830-4784. This paper forms part of the PhD thesis of Jane Cullen (2023): https://mro.massey.ac.nz/items/4ca56d27-2825-4057-9352-9d1c9fcb7e58.
References
1 Braithwaite J, Glasziou P, Westbrook J. The three numbers you need to know about healthcare: the 60-30-10 Challenge. BMC Med 2020; 18: 102.
| Crossref | Google Scholar | PubMed |
2 Doty MM, Tikkanen RS, FitzGerald M, et al. Income-related inequality in affordability and access to primary care in eleven high-income countries. Health Aff 2021; 40: 113-120.
| Crossref | Google Scholar | PubMed |
3 Bate P, Robert G, Fulop N, et al. Perspectives on context: A selection of essays considering the role of context in successful quality improvement. London: Health Foundation; 2015. Available at https://www.health.org.uk/publications/perspectives-on-context
4 Kao LS. Implementation science and quality improvement. In Success in Academic Surgery: Health Services Research. Springer; 2014. pp. 85–100. 10.1007/978-1-4471-4718-3_8.
5 Van Kemenade E. Patterns emerging from the TQM paradigm in relation to the 21st century complex context within TQM journal. TQM J 2022; 34(3): 494-514.
| Crossref | Google Scholar |
6 Ovretveit J, Mittman B, Rubenstein L, et al. Using implementation tools to design and conduct quality improvement projects for faster and more effective improvement. Int J Health Care Qual Assur 2017; 30(8): 755-768.
| Crossref | Google Scholar | PubMed |
7 Dewan M, Parsons A, Tegtmeyer K, et al. Contextual factors affecting implementation of in-hospital pediatric CPR quality improvement interventions in a resuscitation collaborative. Pediatr Qual Saf 2021; 6: 455.
| Crossref | Google Scholar | PubMed |
8 Kaplan HC, Froehle CM, Cassedy A, et al. An exploratory analysis of the model for understanding success in quality. Health Care Manage Rev 2013; 38: 325-338.
| Crossref | Google Scholar | PubMed |
9 Ovretveit J, Mittman B, Rubenstein L, et al. Using implementation tools to design and conduct quality improvement projects for faster and more effective improvement. Int J Health Care Qual Assur 2017; 30(8): 755-768.
| Crossref | Google Scholar | PubMed |
10 Reed JE, Kaplan HC, Ismail SA. A new typology for understanding context: qualitative exploration of the model for understanding success in quality (MUSIQ). BMC Health Serv Res 2018; 18: 584.
| Crossref | Google Scholar | PubMed |
11 Coles E, Wells M, Maxwell M, et al. The influence of contextual factors on healthcare quality improvement initiatives: what works, for whom and in what setting? Protocol for a realist review. Syst Rev 2017; 6: 168.
| Crossref | Google Scholar | PubMed |
12 Douglas LC, Heo M, Azad N, et al. Contextual factors associated with quality improvement success in a multi-site ambulatory setting. J Healthc Qual 2019; 41: 317-328.
| Crossref | Google Scholar | PubMed |
13 Kaplan HC, Provost LP, Froehle CM, et al. The Model for Understanding Success in Quality (MUSIQ): building a theory of context in healthcare quality improvement. BMJ Qual Saf 2012; 21: 13-20.
| Crossref | Google Scholar | PubMed |
14 National Academies of Sciences, Engineering, and Medicine. Implementing High-Quality Primary Care: Rebuilding the Foundation of Health Care. Washington, DC: The National Academies Press; 2021. 10.17226/25983
15 Auraaen A, Slawomirski L, Klazinga N The Economics of Patient Safety in Primary and Ambulatory Care: Flying Blind. OECD Health Working Papers. OECD; 2018. 10.1787/baf425ad-en
16 Crabtree BF, Nutting PA, Miller WL, et al. Primary care practice transformation is hard work: insights from a 15-year developmental program of research. Med Care 2011; 49: S28-S35.
| Crossref | Google Scholar | PubMed |
17 Fiscella K, McDaniel SH. The complexity, diversity, and science of primary care teams. Am Psychol 2018; 73: 451-467.
| Crossref | Google Scholar | PubMed |
18 Lau R, Stevenson F, Ong BN, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci 2016; 11: 40.
| Crossref | Google Scholar | PubMed |
19 Birken SA, Rohweder CL, Powell BJ, et al. T-Cast: an implementation Theory Comparison and Selection Tool. Implement Sci 2018; 13: 143.
| Crossref | Google Scholar | PubMed |
20 Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of Health Services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009; 4: 50.
| Crossref | Google Scholar | PubMed |
21 Flottorp SA, Oxman AD, Krause J, et al. A checklist for identifying determinants of practice: a Systematic Review and Synthesis of Frameworks and Taxonomies of Factors that prevent or enable improvements in Healthcare Professional Practice. Implement Sci 2013; 8: 35.
| Crossref | Google Scholar | PubMed |
22 Nilsen P, Birken, SA Handbook on Implementation Science. Edward Elgar Publishing; 2020. 10.4337/9781788975995
23 Stange K, Glasgow R Considering and reporting important contextual factors in research on the patient-centered medical home. Rockville, MD: Agency for Healthcare Research and Quality; 2013. Available at www.ahrq.gov/sites/default/files/wysiwyg/ncepcr/tools/PCMH/contextual-factors.pdf
24 Tabak RG, Khoong EC, Chambers DA, et al. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med 2012; 43(3): 337-350.
| Crossref | Google Scholar | PubMed |
25 Nilsen P, Bernhardsson S. Context matters in implementation science: a scoping review of determinant frameworks that describe contextual determinants for implementation outcomes. BMC Health Serv Res 2019; 19(1): 189.
| Crossref | Google Scholar | PubMed |
26 Brennan SE, Bosch M, Buchan H, et al. Measuring organizational and individual factors thought to influence the success of quality improvement in Primary Care: a systematic review of instruments. Implement Sci 2012; 7: 121.
| Crossref | Google Scholar | PubMed |
27 Birken SA, Powell BJ, Shea CM, et al. Criteria for selecting Implementation Science Theories and Frameworks: results from an international survey. Implement Sci 2017; 12: 124.
| Crossref | Google Scholar | PubMed |
28 Skolarus TA, Lehmann T, Tabak RG, et al. Assessing citation networks for dissemination and implementation research frameworks. Implement Sci 2017; 12: 97.
| Crossref | Google Scholar |
29 Garbutt JM, Dodd S, Walling E, et al. Barriers and facilitators to HPV vaccination in primary care practices: a mixed methods study using the Consolidated Framework for Implementation Research. BMC Fam Pract 2018; 19: 53.
| Crossref | Google Scholar | PubMed |
30 Ilott I, Gerrish K, Booth A, et al. Testing the Consolidated Framework for Implementation Research on health care innovations from South Yorkshire. J Eval Clin Pract 2013; 19: 915-924.
| Crossref | Google Scholar | PubMed |
31 Keith RE, Crosson JC, O’Malley AS, et al. Using the Consolidated Framework for Implementation Research (CFIR) to produce actionable findings: a Rapid-Cycle Evaluation Approach to improving implementation. Implement Sci 2017; 12: 15.
| Crossref | Google Scholar | PubMed |
32 Kowalski CP, Veeser M, Heisler M. Formative evaluation and adaptation of pre-and early implementation of diabetes shared medical appointments to maximize sustainability and adoption. BMC Fam Pract 2018; 19: 109.
| Crossref | Google Scholar | PubMed |
33 McHugh M, Brown T, Liss DT, et al. Practice facilitators’ and leaders’ perspectives on a facilitated quality improvement program. Ann Fam Med 2018; 16: S65-S71.
| Crossref | Google Scholar | PubMed |
34 Meyers D, Miller T, Genevro J, et al. EvidenceNOW: balancing primary care implementation and implementation research. Ann Fam Med 2018; 16: S5-S11.
| Crossref | Google Scholar | PubMed |
35 Oetzel J, Ngawati R, Penetito-Hemara D, et al. Facilitators and barriers for implementation of health programmes with Māori communities. Implement Sci Commun 2024; 5(1): 26.
| Crossref | Google Scholar |
36 Stokes T, Tumilty E, Doolan-Noble F, et al. HealthPathways implementation in a New Zealand health region: a qualitative study using the Consolidated Framework for Implementation Research. BMJ Open 2018; 8: e025094.
| Crossref | Google Scholar | PubMed |
37 Kirk MA, Kelley C, Yankey N, et al. A systematic review of the use of the consolidated framework for implementation research. Implement Sci 2015; 11: 72.
| Crossref | Google Scholar | PubMed |
38 Wilson WJ, Jayamaha N, Frater G. The effect of contextual factors on quality improvement success in a lean-driven New Zealand healthcare environment. Int J Lean Six Sigma 2018; 9(2): 199-220.
| Crossref | Google Scholar |
39 Tomoaia-Cotisel A, Scammon DL, Waitzman NJ, et al. Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Ann Fam Med 2013; 11 Suppl 1(Suppl 1): S115-S123.
| Crossref | Google Scholar | PubMed |
40 Griffin A, McKeown A, Viney R, et al. Revalidation and quality assurance: the application of the MUSIQ framework in independent verification visits to healthcare organisations. BMJ Open 2017; 7: e014121.
| Crossref | Google Scholar | PubMed |
41 Barson S, Doolan-Noble F, Gray J, et al. Healthcare leaders’ views on successful quality improvement initiatives and context. J Health Organ Manag 2017; 31: 54-63.
| Crossref | Google Scholar | PubMed |
42 Kringos DS, Sunol R, Wagner C, et al. The influence of context on the effectiveness of hospital quality improvement strategies: a review of systematic reviews. BMC Health Serv Res 2015; 15: 277.
| Crossref | Google Scholar | PubMed |
43 Reed J, Ramaswamy R, Parry G, et al. Context matters: adapting the Model for Understanding Success in Quality Improvement (MUSIQ) for low and middle income countries. Implement Sci 2017; 12(48): 23.
| Crossref | Google Scholar |
44 Durie M, Hoskins TK, Jones A. Interview: Kaupapa Maori: shifting the social. N Z J Educ Stud 2012; 47(2): 21.
| Google Scholar |
45 Cullen J, Childerhouse P, McBain L. Contextual antecedents of quality improvement: a comparative case study in rural, urban and Kaupapa Māori general practice. J Prim Health Care 2022; 14: 179-186.
| Crossref | Google Scholar | PubMed |
46 Cullen J, Childerhouse P, Jayamaha N, et al. Developing a model for primary care quality improvement success: a comparative case study in rural, urban and Kaupapa Māori organisations. J Prim Health Care 2023; 15: 333-342.
| Crossref | Google Scholar | PubMed |
47 Lanham HJ, Palmer RF, Leykum LK, et al. Trust and reflection in primary care practice redesign. Health Serv Res 2016; 51: 1489-1514.
| Crossref | Google Scholar | PubMed |
48 Larkins S, Carlisle K, Turner N, et al. At the grass roots level it’s about sitting down and talking’: exploring quality improvement through case studies with high-improving Aboriginal and Torres Strait Islander primary healthcare services. BMJ Open 2019; 9: e027568.
| Crossref | Google Scholar | PubMed |
49 Coombe CM, Chandanabhumma PP, Bhardwaj P, et al. A Participatory, Mixed Methods Approach to define and measure partnership synergy in long-standing equity focused CBPR partnerships. Am J Community Psychol 2020; 66(3–4): 427-438.
| Crossref | Google Scholar | PubMed |
50 Avby G, Kjellström S, Andersson Bäck M. Tending to innovate in Swedish primary health care: a qualitative study. BMC Health Serv Res 2019; 19: 42.
| Crossref | Google Scholar |
51 Middleton L, Dunn P, O’Loughlin C, et al. Taking Stock: Primary care innovation. A report for the New Zealand Productivity Commission. Wellington, New Zealand: Victoria University of Wellington, Health Services Research Centre; 2018. Available at https://www.treasury.govt.nz/sites/default/files/2024-05/pc-inq-ssp-taking-stock-primary-care-innovation-victoria-university-wellington-v2.pdf
52 Damschroder LJ, Reardon CM, Widerquist M, et al. The updated consolidated framework for implementation research based on user feedback. Implement Sci 2022; 17: 75.
| Crossref | Google Scholar | PubMed |
54 Parchman ML, Anderson ML, Coleman K, et al. Assessing quality improvement capacity in primary care practices. BMC Fam Pract 2019; 20: 103.
| Crossref | Google Scholar | PubMed |
55 Leite H, Radnor Z, Bateman N. Meaningful inhibitors of the lean journey: a systematic review and categorisation of over 20 years of literature. Prod Plan Control 2020; 33(5): 403-426.
| Crossref | Google Scholar |
56 Hashim MJ. Principles of family medicine and general practice–defining the five core values of the specialty. J Prim Health Care 2016; 8(4): 283-287.
| Crossref | Google Scholar | PubMed |
57 Lim S, Islam NS. Small Practices, Big (QI) Dreams: customizing Quality Improvement (QI) efforts for under-resourced Primary Care Practices to improve diabetes disparities. JMIR Diabetes 2022; 7: e23844.
| Crossref | Google Scholar | PubMed |
58 Knight AW, Dhillon M, Smith C, et al. A quality improvement collaborative to build improvement capacity in regional primary care support organisations. BMJ Open Qual 2019; 8: e000684.
| Crossref | Google Scholar | PubMed |
Appendix 1.MUSIQ results
| Contextual factor | C1A PHO & GP | C1A DHB | C2A | C3A | C4A | C5A | C6A | Max score | |
|---|---|---|---|---|---|---|---|---|---|
| External environment | |||||||||
| External motivators | 7 | 7 | 1 | 5 | 7 | 7 | 5 | 7 | |
| External project sponsorship | 5 | 5 | 5 | 5.3 | 7 | 7 | 1 | 7 | |
| Organisation | |||||||||
| QI leadership | 7 | 2 | 7 | 7 | 7 | 3 | 7 | 7 | |
| Senior leader sponsor | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | |
| QI culture | 7 | 7 | 7 | 6.3 | 7 | 7 | 7 | 7 | |
| QI maturity | 6 | 4 | 5 | 5.8 | 2 | 5 | 6 | 7 | |
| QI support and capacity | |||||||||
| Data infrastructure | 6 | 6 | 2 | 6.5 | 6 | 4 | 6 | 7 | |
| Resource availability | 4 | 4 | 7 | 5 | 7 | 7 | 5 | 7 | |
| QI workforce focus | 2.8 | 3.3 | 4.3 | 4.9 | 4.8 | 5 | 4.7 | 7 | |
| Microsystem | |||||||||
| QI leadership | 4 | 4 | 7 | 6 | 7 | 7 | 7 | 7 | |
| QI culture | 7 | 7 | 7 | 6.8 | 7 | 7 | 7 | 7 | |
| Motivation | 4 | 4 | 7 | 5.5 | 5 | 7 | 7 | 7 | |
| QI capability | 1 | 1 | 3 | 5 | 2 | 2 | 2 | 7 | |
| QI team | |||||||||
| Team leadership | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | |
| Team diversity | 7 | 7 | 7 | 7 | 7 | 6 | 6 | 7 | |
| Subject matter expert | 7 | 7 | 7 | 6.3 | 7 | 6 | 7 | 7 | |
| Team decision-making processes | 7 | 7 | 6.7 | 6.5 | 7 | 6.8 | 6.8 | 7 | |
| Team norms | 7 | 7 | 7 | 6 | 7 | 6.8 | 7 | 7 | |
| Team QI skill | 2 | 2 | 7 | 5.3 | 7 | 3 | 5 | 7 | |
| Physician involvement | 7 | 7 | 7 | 7 | 7 | 6 | 7 | 7 | |
| Prior QI experience | 5 | 5 | 7 | 5.3 | 7 | 6 | 6 | 7 | |
| Team tenure | 7 | 7 | 7 | 7 | 7 | 7 | 7 | 7 | |
| Miscellaneous | |||||||||
| Triggering event | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 7 | |
| Task strategic importance to the organisation | 7 | 6 | 7 | 6.8 | 7 | 7 | 7 | 7 | |
| Total score | 132 | 125 | 140 | 140 | 147 | 138 | 139 | 168 | |
AC1 completed the MUSIQ survey together as two groups (DHB and PHO/general practice) and provided an agreed score for each group. C2: two participants gave agreed scores.,C3: four participants scored separately and then the scores were averaged. C4, C5 and C6: only one person completed the survey.
MUSIQ score interpretation guide
| Total score | Interpretation | |
|---|---|---|
| 168 | Highest possible MUSIQ score | |
| 120–168 | Project has a reasonable chance of success | |
| 80–119 | Project could be successful, but possible contextual barriers | |
| 50–79 | Project has serious contextual issues and is not set up for success | |
| 25–49 | Project should not continue as is; consider deploying resources to other improvement activities | |
| 24 | Lowest possible MUSIQ score when all questions are answered | |
| 1 | Lowest possible MUSIQ score (questions recorded as ‘don’t know’ or ‘N/A’) |
