Efficacy of submissions as an advocacy strategy: piloting the Public Health Association of Australia’s submission evaluation tool

Introduction

Public health advocates use a variety of strategies to influence policymakers regarding public health policies.¹ Evidence-based submissions to government inquiries and other decision-making bodies are a comparatively low-cost advocacy activity used by public health advocates globally to outline arguments and propose specific strategies.¹ To date, limited studies internationally, and none in Australia, have examined the effectiveness of submissions on public health policy. To address this gap, the Public Health Association of Australia (PHAA) developed an evaluation tool in 2019 to determine submission effectiveness.

Evaluating submissions may identify barriers that generally limit this advocacy strategy’s effectiveness.² This study aims to pilot the PHAA submission evaluation tool by examining submission writing as an advocacy strategy in the Australian context.

Methods

We piloted the tool using submissions by the PHAA (sourced from its website) to Federal public health nutrition policy inquiries from 2010–2019. We limited our data to submissions from the PHAA, as it is one of Australia’s leading non-government advocacy organisations for public health and offered a pragmatic sample to demonstrate the tool’s usefulness. Submissions were included if there was a corresponding, publicly available government report published on the inquiry and if they focused on public health nutrition broadly (rather than additive-specific changes to Australia’s Food Standards Code). The time period studied captures periods of government by both major parties, which have differing ideologies. The ideology of the Federal Liberal National Coalition (the Coalition), which was in power from 2013–2022, emphasises minimal state involvement and free market economics³, while the social democratic ideology of the Australian Labor Party (ALP), which was in power from 2007–2013, strives for a balance between a market economy and state intervention.³

To investigate policy outcomes of government inquiries, corresponding government reports were extracted from government websites. The PHAA submission evaluation tool (see Supplementary File 1, available from: doi.org/10.6084/m9.figshare.21938714.v1) draws on Schumaker’s levels of responsiveness.⁴ to evaluate the quality of submission content, the government’s receptiveness to policy change, and the submitter’s organisational influence.

Submission quality includes three factors:

a) The submission responds to specific inquiry questions

b) The submission identifies and provides high quality evidence for claims

c) The submission outlines priorities and strategic approaches for the issue.

Government receptiveness was measured by analysing two factors:

a) The features of the inquiry, including:

i) Objective of the policy/bill under inquiry

ii) Whether recommendations from submissions were included in the inquiry outcome

iii) Stance taken in the call for submissions

b) Previous momentum/resistance to the proposal, including hearings, inquiries, funding, and policy outcomes.

The tool measured organisational influence by stakeholder category rather than individual groups. This was determined based on the number of times PHAA was: a) mentioned in the final report and b) the proportion of government recommendations aligned with PHAA submissions. The more times submissions were cited in the final report, the higher the organisational influence.⁴

Findings

Six of the 34 available submissions met the inclusion criteria (see Table 1). Despite a high- to very high-quality of all submissions, government alignment with PHAA recommendations, and considerable government receptiveness, this evalution found that only one of six included government inquiries, led to any policy outcomes (the Health Star Rating System Five Year Review). This inquiry and related submission scored ‘very high’ in all tool categories. Some previous studies have demonstrated that submissions are a means for advocates to communicate with current and future governments, record concerns, highlight evidence, raise awareness of public health issues, and impact policymaker perspectives.⁵ However, our study results did not demonstrate that PHAA submissions were related to policy outcomes, suggesting that submissions may not significantly influence policymaking in the short term.

Discussion & conclusions

A limitation of this pilot is that only one organisation’s submissions were assessed and that we present only a limited correlation in this brief report. Both in Australia and internationally, policymaking is a prolonged, complex process with many influential factors, including worldviews and vested interests of stakeholders, interpretation of evidence, and other advocacy strategies. Beyond submissions, studies have acknowledged the influence of relationships with policymakers as a lever for policy change, which can be challenging to establish and maintain for advocates.¹²

We encourage others to iteratively test and refine this tool to improve its usefulness in evaluating submission impact. Focusing on a specific public health topic and analysing a broader scope of submissions would be useful for future research using the tool. The tool may also be helpful for advocacy agencies prior to submission writing, that is, to align submission language to existing government policy and/or to share resources with other health advocacy organisations. As submissions take considerable time, the results from this pilot study are particularly important for advocates when considering the best allocation of their time and efforts.

Acknowledgements

We acknowledge the support and assistance of Ingrid Johnston and Sena Debia from the Public Health Association of Australia on an earlier version of this tool.

Peer review and provenance

Externally peer reviewed, not commissioned.

Author contributions

KC was responsible for conceptualising the study. JG performed data collection and analysis and wrote the first draft of the manuscript. CR performed further data analysis and wrote the second draft of the manuscript. KC and MF supervised the study, reviewed and edited the manuscript and provided analytical advice.

Competing interests

None declared.