Seasonal sea level forecasts for the Australian coast

1.Introduction

Coastal inundation is an increasingly significant hazard for many communities around Australia, and is 1 of the 10 priority hazards being assessed by Australia’s first National Climate Risk Assessment (NCRA, Department of Climate Change, Energy, the Environment and Water 2024). The frequency and severity of coastal flooding are expected to increase with sea level rise under climate change (Cooley et al. 2022), necessitating adaptation measures to protect coastal communities and infrastructure (Mortensen et al. 2024). Although extreme flooding events often receive the most attention, the cumulative cost of regular minor or nuisance flooding (‘high tide flooding’) may be comparable to that of extreme events in many locations, especially as sea level rise raises the baseline water levels around which tides vary (Moftakhari et al. 2017). In addition to affecting coastal communities through property damage, loss of access and loss of life, sea level extremes can also affect coastal ecosystems. For example, low sea level events can expose coral reefs causing bleaching (e.g. Ampou et al. 2017; Brown et al. 2019) or lead to mangrove die-backs (e.g. Abhik et al. 2021).

Seasonal forecasts of coastal sea level made up to a year into the future can provide valuable information for environmental managers, emergency services, local councils and port operators to prepare for and mitigate the impacts of extremes. The Australian Bureau of Meteorology’s seasonal ensemble prediction system, the Australian Community Climate and Earth-System Simulator-Seasonal prediction system version 2 (ACCESS-S2, Hudson et al. 2017; Wedd et al. 2022) provides seasonal forecasts of global sea level up to 8 months into the future on a low-resolution 1/4° grid. Studies have demonstrated that ensemble seasonal prediction systems, including ACCESS-S2, have skill relative to climatological forecasts in predicting low-frequency sea level variations out to and beyond 6 months into the future throughout broad regions of the tropical open oceans (Miles et al. 2014; Widlansky et al. 2017, 2023; Long et al. 2021). Although the processes governing coastal sea level variability differ from those in the open ocean and require finer grid resolutions to resolve, several studies have demonstrated that such models may also have skill at forecasting sea level along the coast (e.g. on the shelf and at tide gauge locations) where the impacts of sea level extremes are more keenly felt (McIntosh et al. 2015; Long et al. 2021, 2023; Jacox et al. 2023). In this study, we further demonstrate the skill and potential utility of these forecasts through a detailed assessment of ACCESS-S2’s inshore sea level predictions for the Australian coast.

Variability in coastal sea level is driven by a wide range of processes including but not limited to tides, storm surges, coastal upwelling, coastal trapped waves (CTWs), mesoscale eddies and wave processes (setup and run-up, not considered here). These processes act on different time scales, vary in amplitude around the coastline and differ in their predictability (Woodworth et al. 2019). Around Australia, tidal variability is strongest in the north, particularly on the north-west shelf, and weakest in the south and south-west (green line in Fig. 1a, d). By contrast, the high-frequency residual (sea level minus the tide) is strongest along the south coast associated with storm surges and mid-latitude synoptic variability (orange line in Fig. 1a). These signals, beyond weather time scales of a few days, tend to propagate anti-clockwise around the Australian coast, associated with CTW dynamics (White et al. 2014). The mean seasonal cycle is another source of sea level variability, reaching a standard deviation of 10 cm in the north (green line in Fig. 1b), peaking in austral summer (December–February) and autumn (March–May, Barroso et al. 2024) and weakening to the south, where it peaks in austral winter (June–August).

Fig. 1.

Standard deviations of various components of the sea level signal at the ANCHORS tide gauges over the 1982–2018 period. Tide gauges are listed in anti-clockwise order with corresponding locations marked in the map (d). (a) Standard deviation of the tide, the monthly averaged residual (sea level minus the tide) and the daily averaged residual. The standard deviation of the tidal signal is computed from hourly time series using the UTide software package (see Section 2), which includes the annual and semi-annual tidal constituents (thus the residual does not include the seasonal cycle). (b) Standard deviation of the monthly residual, the seasonal cycle (computed from the monthly climatology of the tidal signal) and the monthly averaged inverse barometer (IB). (c) Ratio of the variances of the monthly averaged to the daily averaged residual (blue) and the monthly averaged IB to residual (orange). Also shown are the fraction of the monthly averaged sea level variance associated with ENSO (here, the Niño 3.4 index, N34(t)) and with SAM (using the Marshall 2003 index) from a multi-linear regression on these two indices. For N34(t), with multi-linear regression coefficient B, this is computed as var (B × N34) ÷ var(SLA), where var is the variance operator and SLA(t) is the total (monthly averaged) residual sea level. (d) The colour of the symbols indicates the standard deviation of the tide (green line, a).

Of most interest for this study are the low-frequency (monthly) residual sea level anomalies (SLAs, from the mean seasonal cycle), as this is the signal, in addition to the high-frequency tides and mean seasonal cycle, that may be predictable on seasonal time scales. The low-frequency variability has an amplitude that decreases anticlockwise around the Australian coast (blue line in Fig. 1b, also see Middleton et al. 2007; White et al. 2014) and constitutes 15–80% of the total (daily) residual variability (blue line in Fig. 1c), highlighting the potential for seasonal predictions to provide useful information. Many of these low-frequency anomalies are associated with the El Niño-Southern Oscillation (ENSO), which, when characterised by sea surface temperature (SST) anomalies in the eastern Pacific Niño 3.4 region (5°N–5°S, 170–120°W), explains up to 60% of the variance on the north-west and west coasts (green line in Fig. 1c) due to oceanic teleconnections by CTWs (White et al. 2014). Lowe et al. (2021) showed that there was a 20% change in the 100-year extreme sea level event between El Niño and La Niña conditions in south-west Western Australia.

Other climate modes contribute to monthly sea level variability to a lesser extent. The Southern Annular Mode (SAM), corresponding to meridional shifts in the southern hemisphere westerly winds (Fogt and Marshall 2020), explains up to 15% of the variance along the south coast independently of ENSO (Fig. 1c). It has been demonstrated that the SAM has some predictability in ACCESS-S2 out to lead month 3, but only due to its connection to ENSO (Lim et al. 2013; Wedd et al. 2022). Other modes of seasonal climate variability, such as the Indian Ocean Dipole (IOD) or a second mode ENSO index (e.g. Frankcombe et al. 2015) have a relatively weak impact on Australian sea level when included in the multi-linear regression used to compute the variance fractions in Fig. 1c (not shown, also see Wang et al. 2021).

In addition to dynamic sea level variability, atmospheric sea level pressure variations (typically neglected in output from general circulation models) also drive variability in sea level through the ‘inverse barometer’ (IB) effect. On time scales beyond a few days, the IB can be approximated as follows:

where Pa′(x,t) is the deviation of the sea level pressure from its global ocean mean, g is the acceleration due to gravity and ρ_s is the surface sea water density (we neglect the small variations in surface sea water density and take gρ_s = 9.9 mm hPa⁻¹) (Ponte 1994; Gregory et al. 2019). Underneath strong, transient low pressure systems, the IB effect can often dominate the wind-driven dynamic impacts of the system on sea level (Ponte 1994). At longer time scales, the IB is smaller, but can still constitute a significant fraction of the total sea level variability. For Australia, monthly averaged IB anomalies are strongest along the south coast where they reach a standard deviation of ~4 cm (orange line in Fig. 1b) and account for 10–20% of the monthly residual sea level variance (orange line in Fig. 1c). The IB has been neglected in past studies of seasonal sea level predictability (e.g. Miles et al. 2014; McIntosh et al. 2015; Widlansky et al. 2023). However, the IB is indistinguishable from other components of sea level variability from a local impacts perspective. Thus, we consider it important to consider and, as suggested by Long et al. (2021), we explore the impact on forecast skill of adding the IB to the dynamic sea level in the ACCESS-S2 forecasts (yielding the total sea level) as a post-processing step.

In this paper we present a skill assessment of ACCESS-S2 seasonal coastal sea level forecasts up to 8 months into the future around the Australian coastline. We present both deterministic and probabilistic skill metrics and we assess the impact of adding the IB contribution to seasonal sea level, using ACCESS-S2’s sea level pressure forecasts. Finally, we provide a brief overview of the mechanisms responsible for predictability, and discuss the next steps required to operationalise these forecasts and maximise their utility for coastal decision makers.

2.Data and methods

2.4. Skill calculations

2.4.1. Deterministic skill

For deterministic (ensemble mean) skill metrics, we use the anomaly correlation coefficient (ACC) and root-mean squared error (RMSE) as commonly used in the literature (e.g. Widlansky et al. 2023). We also compute a RMSE based skill score (RMSESS):

(2)RMSESS=1−RMSERMSEref

where RMSE^ref is the RMSE of a reference forecast of zero SLA, being equivalent to the standard deviation of the observed SLA. Values of RMSESS above zero indicate better skill than the zero SLA reference forecast with a maximum of 1 (zero RMSE, corresponding to 100% of the variance in the observations explained by the forecast). Both ACC and RMSESS for ACCESS-S2 are also compared to the equivalent scores for an anomaly persistence forecast. This (deterministic) persistence forecast is equal to the observed SLA for the month prior to the given start date (e.g. the observed December SLA for the 1 January start date).

2.4.2. Probabilistic (ensemble) skill

Deterministic forecasts do not take advantage of the additional information on uncertainty available in the forecast ensemble and its spread (of 27 members for the ACCESS-S2 hindcast). To take this additional information into account, we also compute the continuous ranked probability score (CRPS):

(3)CRPS(F,SLAo)=∫−∞∞[F(a)−H(a− SLA o)]2da

where F is the cumulative distribution function of the ACCESS-S2 ensemble at a given lead month, target month and location, H is the Heaviside step function and SLA^o is the observed SLA. For a deterministic forecast

Fa=H(a−SLAf)

where SLA^f is the deterministic forecast and the CRPS reduces to the mean absolute error. For an ensemble forecast, tighter forecast distributions around the observed value SLA^o yield smaller, better CRPS scores. As for the RMSE (Eqn 2), we convert CRPS to a skill score CRPSS by comparing it with a reference forecast CRPS^ref of the climatological distribution of the observed SLA from the given target month. The CRPS values for ACCESS-S2 are also compared with those of the deterministic anomaly persistence forecast.

2.4.3. Probabilistic quantile exceedance skill

Finally, we also quantify the skill of ACCESS-S2 in forecasting the probability P of SLAs exceeding various quantiles of the observed climatological SLA distribution at each ANCHORS tide gauge location. We consider three quantile levels α: the median (α = 0.5), upper tercile (α = 2/3) and upper decile (α = 0.9). The quantile thresholds for each of these quantile levels are computed from the climatological distribution of observed SLAs over the ACCESS-S2 hindcast time period, 1982–2018, at each tide gauge. We consider both quantile thresholds computed individually for each calendar month and those computed over all calendar months together. An example for the Broome tide gauge is shown by the blue distribution in Fig. 2b.

Fig. 2.

Example probabilistic quantile exceedance forecasts for SLAs at the Broome tide gauge from ACCESS-S2 at 3-months lead. (a) Time series of observed (blue line), ACCESS-S2 ensemble mean (green line) and ACCESS-S2 95% ensemble confidence interval (green shading) sea level anomalies over the 1995–2005 period. (b) Climatological SLA distribution from the observations (blue shading) and the ACCESS-S2 ensemble (green shading) over the ACCESS-S2 hindcast period, 1982–2018. Note that the observational distribution is noisier because it has a factor of 27 less values. The solid lines are kernel density estimate plots only used for visualisation. The dashed (dotted) lines (a and b) indicate the median (upper decile) quantile thresholds for the observations (blue) and the ACCESS-S2 ensemble (green), in this case computed using data from all months across the hindcast period. (c and d) Example forecast probability (green) and observed exceedances (blue) for the (c) median and (d) upper decile.

The ACCESS-S2 forecast for the probability P of exceeding the given quantile of the observed climatological SLA distribution is constructed based on the fraction of ensemble members that exceed the quantile (for the corresponding quantile level α) from the ACCESS-S2 climatological SLA distribution over the 1982–2018 hindcast period. Each quantile is computed separately for each lead time and tide gauge. The value of this quantile threshold may differ from the value of the quantile threshold from the observational distribution due to model biases (e.g. compare blue and green distributions in Fig. 2b). This method thus constitutes a bias correction. Example time series forecasts for the median and upper decile exceedance probabilities at Broome are shown in Fig. 2c, d.

Verification for these forecasts is based on the squared error in probability space, the Brier Score (BS):

(4)BS=(p− p o)2¯

where p is the ACCESS-S2 probability forecast and p^o is the binary event time series, equal to 1 if the observed SLA exceeds the quantile of the observed climatological SLA distribution and 0 otherwise (e.g. blue lines in Fig. 2c, d). The overline indicates an average over all start dates (either for a single month or for all months grouped together). As for the RMSE and CRPS, we compute a BS score (BSS) by comparing the ACCESS-S2 BS against BS^ref, the BS achieved by the best constant reference forecast probability, being the observed frequency of threshold exceedances over the hindcast period. Owing to the finite set of verification times, this observed frequency may differ somewhat from the theoretical expectation of p^ref = 1 − α. The ACCESS-S2 BS is also compared to a deterministic anomaly persistence forecast (i.e. whether the observed SLA for the month prior to the given start date exceeds the threshold).

Finally, to better understand the quantile exceedance skill, we perform a Consistent, Optimally binned, Reproducible, and Pool-Adjacent-Violators algorithm based (CORP) decomposition of the BSS (Dimitriadis et al. 2021; Loveday et al. 2024), which uses nonparametric isotonic regression to robustly decompose BS into miscalibration (MCB), discrimination (DSC) and uncertainty (BS^ref) terms:

(5)BS=MCB−DSC+BSref

Small values of MCB indicate that the forecast is reliable, in that the observed frequency of quantile exceedance for a given forecast probability matches that forecast probability. Larger values of DSC indicate that the forecast can discriminate between exceedance and non-exceedance. Using Eqn 5 allows us to express the BSS as follows:

(6)BSS=DSCBSref−MCBBSref

where MCB/BS^ref = 0 indicates perfect reliability and DSC/BS^ref = 1 indicates perfect discrimination.

Note that the probabilistic (and deterministic) verification and computation of quantile threshold values is done here using all the data available. An alternative, which may be more appropriate for application to real-time forecasts, is to use a cross-validation approach where the skill is computed separately for each year (and then averaged across all years), with the data for the year being verified being dropped for the purpose of computing thresholds; also see McIntosh et al. (2015) for a discussion of the role of cross-validation under calibration. We tested this approach for the BSS skill metric and found only minor differences not exceeding 0.02 (relative to BSS values at skilful locations between 0.2 and 0.6) for the median and reaching ~0.1 (relative to 0.1–0.4) at a few isolated locations for upper tercile and upper decile exceedance. We thus chose not to use cross-calibration in our skill analyses.

2.4.4. Statistical significance

For comparisons to altimetry (spatially resolved skill), the statistical significance of the ACC values is computed using an effective sample size of N(1 − ρ_fρ_o) ÷ (1 + ρ_fρ_o), where ρ_f and ρ_o are the lag-1 autocorrelations of the given forecast and observed time series of length N (~444 = 37 years × 12 months) respectively. For computing the significance of ACC differences, a Fisher-z transformation is applied following Shin and Newman (2021) and McIntosh et al. (2015). For all tide gauge computations, the Diebold–Mariano test (Diebold and Mariano 1995), with the Hering and Genton (2011) modification, on the time series of raw score differences at each time (e.g. the normalised anomaly product for ACC or the squared error for RMSE) is used to evaluate significance while taking into account autocorrelation with lead time. Following convention, we use P = 0.05 for significance.

3. Results

We begin with a brief analysis of the ACCESS-S2 ocean reanalysis from which the forecasts are initialised (Section 3.1) and assess ACCESS-S2’s representation of monthly SLA variance (Section 3.2). We then describe the spatial skill distribution for ACCESS-S2 ensemble mean sea level forecasts (including the IB) across all target calendar months (Section 3.3). This is followed by a detailed analysis of the skill at selected tide gauges as a function of target calendar month (Sections 3.4 and 3.5). The contribution of the IB to the forecast skill is considered in Section 3.6 and the skill of quantile exceedance probability forecasts is considered in Section 3.7.

3.1. Performance of the reanalysis

As discussed in more detail for the global context in Widlansky et al. (2023), the skill of ACCESS-S2 forecasts of seasonal sea level variability in the open ocean is lower than seasonal prediction systems whose reanalysis systems do assimilate altimetry observations. Focusing on the Australian region, Fig. 3a shows that the correlation between the ACCESS-S2 ocean reanalysis and satellite altimetry is relatively low in the open ocean. However, correlations are much higher at the coast, suggesting that the lack of assimilation of altimetry in the ACCESS-S2 reanalysis has little impact on the representation of coastal sea level variability. Here, assimilation of in situ temperature and salinity data, as well as satellite SST data, may contribute to directly constraining the sea level variability, in addition to teleconnections from other regions. It is important to note though that altimetry is not typically used inshore in most assimilation systems (Feng et al. 2024) as the signal-to-noise ratio is much smaller than in the open ocean. The correlation coefficients between the ACCESS-S2 ocean reanalysis and the ANCHORS tide gauges remain above 0.8 around most of the coast, with values only dropping below 0.8 for the east coast gauges between Sydney and Cairns (dots in Fig. 3a).

Fig. 3.

(a) Anomaly correlation coefficient (ACC) of linearly detrended monthly SLAs from the ACCESS-S2 ocean reanalysis compared to IMOS sea level and ANCHORS tide gauge observations over the 1993–2018 period. (b) The standard deviation of monthly SLAs from the observations. The standard deviation of monthly SLAs from the ACCESS-S2 hindcast ensemble members divided by the IMOS/ANCHORS observed standard deviation at lead month (c) 0 (corresponding to the first month of the forecast after initialisation, note that the magnitude of the variability in the reanalysis is similar to the lead 0 forecast, not shown) and (d) 3. Note that there are no data in the Gulf of Carpentaria in the IMOS dataset, which is depicted by grey shading.

3.3. Ensemble mean forecast skill (all target months)

Consistent with past studies using the seasonal prediction system POAMA, the predecessor of ACCESS-S (Miles et al. 2014; McIntosh et al. 2015), the ACC skill of ACCESS-S2 is highest in the north-west and reduces anti-clockwise around the coast (Fig. 4a, c). The ACC on the east and south-east coasts decays rapidly beyond the first month, whereas in the west and north it remains above 0.7 out to lead month 3 (Fig. 5a, d). The ACC skill is statistically significant (i.e. better than a 0-anomaly forecast) around nearly the entire coastline, even out to lead month 3 (Fig. 4c). This is an improvement over POAMA (e.g. McIntosh et al. 2015), primarily as the longer ACCESS-S2 hindcast provides a larger sample size and improved statistics (e.g. lower ACCs can still be significant). At short lead times most of the skill on the west and north-west coasts comes from anomaly persistence. The ACC of the ACCESS-S2 forecast is not significantly better than an anomaly persistence forecast at most of these locations at lead 0, but does beat persistence as lead time increases (Fig. 4b, compare solid and dashed lines of Fig. 5a, d). Skill is significantly better than persistence along the south coast and particularly in the south-east at lead month 0 (Fig. 4b), highlighting the shorter decorrelation time scale of sea level variability in these regions. At longer leads (e.g. lead month 3, Fig. 4c, d) the model beats persistence on the west and much of the south coast, whereas on the east coast persistence and ACCESS-S2 forecasts have much lower, though still significant, ACC values.

Fig. 4.

Ensemble mean skill metrics for ACCESS-S2 sea level anomalies compared to satellite altimetry (continuous colour) and ANCHORS tide gauges (markers) over the 1993–2018 period. The anomaly correlation coefficient (ACC) at lead (a) 0 (corresponding to the first month of the forecast after initialisation) and (c) 3 months. (b, d) The ACCESS-S2 ACC minus the ACC of an anomaly persistence forecast. Hatching indicates values that are (a, c) not significant, or (b, d) not significantly better than an anomaly persistence, at P = 0.05. The RMSE Skill Score (RMSESS) at lead (e) 0 and (g) 3 months. A value of 0 or less indicates no skill compared to a 0-anomaly forecast, with a value of 1 being perfect (zero RMSE). (f) The ACCESS-S2 RMSESS minus the RMSESS of an anomaly persistence forecast compared to altimetry and tide gauges. (h) The ratio of the standard deviation of individual ensemble members of the ACCESS-S2 forecast (averaged across ensemble members) at lead 0 months to the observations. All metrics are for all target calendar months together. Note that there are no data in the Gulf of Carpentaria in the IMOS dataset, which is depicted by grey shading.

Fig. 5.

ACC skill at selected ANCHORS tide gauges. (a, d) ACC from ACCESS-S2 (solid line) and for persistence (dashed line) across all target calendar months as a function of lead month for the period 1982–2018. Solid squares mark values where the ACCESS-S2 ACC is significantly better than persistence at P = 0.05 according to the Diebold–Mariano test. (b, e) ACC from ACCESS-S2 as a function of lead and target calendar month. (c, f) ACC difference between ACCESS-S2 and an anomaly persistence forecast. Crosses mark values that are not significant (b, e) or not significantly better than persistence (c, f) at P = 0.05 according to the Diebold–Mariano test. See Fig. 1 for tide gauge locations.

The RMSESS illustrates similar spatial and temporal patterns as ACC (Fig. 4e–g). However, at longer leads the ACCESS-S2 RMSE is much better than persistence (Fig. 4h). This is because the variance of the ACCESS-S2 ensemble mean forecast reduces with lead (an appropriate representation of forecast uncertainty) whereas the variance of an anomaly persistence forecast remains constant with lead (an overconfident forecast).

3.5. Ensemble forecast skill

To take ensemble information into account (the ACC and RMSESS metrics consider only the ensemble mean forecast), we now analyse the CRPS and CRPSS metrics (Fig. 6, see Methods). ACCESS-S2’s CRPS is significantly better than a climatological reference forecast for most months on the north-west and west coasts, except for long leads in winter. Conversely, across the south and east coasts, CRPS is not significantly better than climatology for most leads and most months, although there is still some positive values at shorter lead times. Unlike the ACC, ACCESS-S2’s CRPSS is significantly better than anomaly persistence at the majority of leads and target months (compare solid and dashed lines in top row, and lack of red crosses in bottom row, of Fig. 6). For forecast use cases that properly utilise uncertainty information available in ACCESS-S2’s ensemble, the CRPSS is the appropriate skill score to base skill assessment on (rather than the ACC or RMSESS).

Fig. 6.

As for Fig. 5 but for the CRPS skill score CRPSS (see Methods). Values above 0 indicate skill compared to a climatological reference forecast (e.g. forecasting the climatological distribution of sea level), with a maximum value of 1 (corresponding to a perfect, certain forecast). Squares (a and c) indicate CRPS scores that are significantly better than both the climatological reference forecast (i.e. CRPSS is significantly greater than 0) and anomaly persistence (dashed line) according to the Diebold–Mariano test at P = 0.05. Black (red) crosses (b and d) indicate CRPS scores that are not significantly better than the climatological reference forecast (anomaly persistence forecast) according to the Diebold–Mariano test at P = 0.05.

3.6. Role of inverse barometer

We now consider the skill of forecasts of the IB, derived from ACCESS-S2’s sea level pressure forecast as a post-processing step, and whether including the IB affects the total sea level forecast skill.

As discussed in the Introduction, the standard deviation of the monthly averaged IB is highest in the mid-latitudes and reaches an amplitude of ~±4 cm (Fig. 7d) that is 10–20% of the total sea level variance (orange line in Fig. 1c). The spatial scales of the IB are large and so here we summarise skill at four representative tide gauges (Fig. 7). The ACC and RMSESS for ACCESS-S2’s IB forecasts are both highest in the north and lowest in the south. This is consistent with the larger role for chaotic, weather-driven variability in the mid-latitudes and the predominance of predictable, tropical climate mode-driven variability in the north. When computed across all target calendar months (Fig. 7a, b, e), the ACC is significant at all sites and leads. It remains above 0.4 in the north and south-west but reduces to 0.2 in the south-east beyond lead month 4. As a function of target month (Fig. 7g), ACC is significant in the north except at longer leads in winter. The ACC is a little lower in the south-west and considerably lower in the south-east, where the ACC is barely significant beyond the first month.

Fig. 7.

ACC, RMSESS and CRPSS of the ACCESS-S2 IB forecast compared to ERA-5 over the 1982–2018 period. ACC skill at lead (a) 0 and (b) 3 months. Values that are not significant are hatched. (c) RMSESS at lead 3 months. (d) The standard deviation of the observed IB (m). (e) ACC and (f) CRPSS for all target months together at selected tide gauge locations. (g) ACC and (h) CRPSS as a function of target calendar month. Squares (e and f) note significant skill, whereas black crosses (g and h) indicate skill that is not significantly better than a reference forecast of 0 anomaly at P = 0.05 according to the Diebold–Mariano test.

The spatial structure of the CRPSS (Fig. 7f, h) is similar to the ACC, and indicates that ACCESS-S2 is better than a reference climatology forecast in similar regions and times of year (indicated by red in Fig. 7h). However, as a function of target calendar month, the CRPSS is non-significant nearly everywhere. This is because, unlike the ACC that only considers the ensemble mean, the CRPSS takes into account the ensemble variability that is (compared to the ensemble mean) relatively large for the IB, due to the stochasticity of atmospheric variability.

Despite the fact that the forecast of the IB alone is not skilful everywhere, including IB in the ACCESS-S2 forecast improves the ACC, RMSESS and CRPSS sea level forecast skills in nearly all cases (Fig. 8). This is because including the IB leads to a better representation of the overall sea level variance and forecast uncertainty. The degree of improvement varies with location and lead time, depending on the skill of the IB forecast alone, skill of the dynamic sea level forecast and the fraction of variability accounted for by the IB. Improvements are largest on the east coast with ACC, RMSESS and CRPSS improvements reaching ~0.1.

Fig. 8.

Difference in skill when including the IB in the ACCESS-S2 forecast, compared to observations of the total sea level including the IB. ACC difference at lead (a) 0 and (b) 3 months over the 1993–2018 period. RMSESS difference at lead (c) 0 and (d) 3 months over the 1993–2018 period. (e, g) ACC and (f, h) CRPSS difference as a function of lead for (e, f) all target calendar months together and (g, h) as a function of target calendar month at selected tide gauges over the 1982–2018 period. (a–d) Note that there are no data in the Gulf of Carpentaria in the IMOS dataset, which is depicted by grey shading.

3.7. Quantile exceedance probability skill

Forecast users are often interested in the probability of exceeding various thresholds, for use in risk assessments. In this section, we perform a skill analysis for ACCESS-S2 forecasts of the probability of exceeding the median, upper tercile and upper deciles of the observed climatological SLA distribution. The construction of these forecasts is described in the Methods and an example is shown in Fig. 2. Here, the BS and BSS are used to quantify the skill of these forecasts (see Methods).

As for the total sea level forecast, the skill of the quantile exceedance forecasts is highest on the north and west coasts and reduces anti-clockwise around the coast (Fig. 9a, d, e), with a distinct drop in forecast skill at leads more than 1 month across the Great Australian Bight east of the Esperance tide gauge (consistent with similar drops in ACC and RMSESS, Fig. 4c, g). The BSS for median and upper tercile exceedance is significantly better than a constant reference forecast on the north and west coasts at all leads, reaching values ~0.6 out of a maximum of 1. For upper decile exceedance the significance of the skill is reduced at long leads compared to the median and upper tercile due to the reduced number of observed events. On the south and south-east coasts, the BSS for median exceedance is significant at leads of 0–3 months and for upper tercile exceedance at leads of 0–1 months, but there is little skill on the east coast. ACCESS-S2 has more accurate probability predictions than overly confident (i.e. no uncertainty) anomaly persistence forecasts (lack of white crosses in Fig. 9a, d, e).

Fig. 9.

The Brier Skill Score (see Methods) for ACCESS-S2 probability forecasts for exceedance of the (a) median α = 0.5, (d) upper tercile α = 2/3 and (e) upper decile α = 9/10 over the 1982–2018 period. 0 indicates no skill improvement over the constant reference forecast (the observed frequency of events) whereas 1 indicates a perfect score. Values that are not better than the constant reference forecast (anomaly persistence) at P = 0.05 according to the Diebold–Mariano test are indicated with a black (white) cross (note that these two sets of crosses are offset). The (b) miscalibration MCB and (c) discrimination DSC components of the BSS from the CORP decomposition (Eqn 6) for the median exceedance forecast. Values of MCB/BS^ref near zero indicate a well calibrated (or reliable) forecast whereas larger values of DSC/BS^ref indicate that the forecast can better discriminate between observed exceedances and non-exceedances.

At tide gauge locations with positive BSS values, the reliability of the forecast probabilities is very good (MCB is near 0, Fig. 9c). Consequently, the locations with the best BSS scores are those where the forecasts are best able to discriminate between events, as quantified by the CORP discrimination score (DSC, Fig. 9b). Note that DSC gives similar information to the area under the Relative Operating Characteristic curve (not shown). Example reliability diagrams illustrate these results in more detail (Supplementary Fig. S3), with reliability curves remaining within the uncertainty bounds around the 1-to-1 line at all locations except along the east coast (e.g. Brisbane, Supplementary Fig. S3f).

The results in Fig. 9 are computed across all target calendar months (with quantile thresholds computed using data from all months) and thus benefit from a large sample size. When considered as a function of target calendar month (with quantile thresholds computed individually for each month), the significance of the BSS values is reduced. However, the spatial pattern is similar to that for all months. Example BSS values as a function of target and lead month are shown in Fig. 10 for the Broome, Fremantle, Sydney and Townsville tide gauges. Note that the colour bar in Fig. 10 ranges between −1 and 1, rather than −0.6 and 0.6 in Fig. 9, as the BSS can be higher for specific months.

Fig. 10.

Brier Skill Score for ACCESS-S2 probability forecasts for exceedance of the (a) median α = 0.5, (b) upper tercile α = 2/3 and (c) upper decile α = 9/10 over the 1982–2018 period at the Broome, Fremantle, Sydney and Townsville tide gauges as a function of lead month and target calendar month. 0 indicates no skill improvement over the constant reference forecast (the observed frequency of events) whereas 1 indicates a perfect score. Values that are not significantly better than the constant reference forecast (anomaly persistence) at P = 0.05 according to the Diebold–Mariano test are indicated with a black (white) cross. Note that unlike in Fig. 9, here the quantile thresholds are computed individually for each target calendar month.

Finally, we note that inclusion of the IB improves the skill of the quantile exceedance forecasts, with BSS increases of order 0.05 around most of the coastline, increasing to more than 0.1 at leads of 0–3 months on the east coast (Fig. 11).

Fig. 11.

The difference in the Brier Skill Score (see methods) for ACCESS-S2 probability forecasts for exceedance of the (a) median q = 0.5, (b) upper tercile q = 2/3 and (c) upper decile q = 9/10 over the 1982–2018 period when including the IB component in the forecast (compared to exceedances of quantiles of observed total sea level anomalies including the IB in both cases). All target calendar months are considered together.

4.Mechanisms for predictability

As shown by Miles et al. (2014) and McIntosh et al. (2015), the major source for predictability of Australian coastal sea level at seasonal time scales is ENSO, which drives up to 60% of the monthly sea level variance on the north and west coasts (green line in Fig. 1c, also see White et al. 2014). ACCESS-S2 has skill in forecasting ENSO beyond 6 months of lead time compared to reference forecasts (Wedd et al. 2022, e.g. their fig. 4). The peak correlation between ENSO and Australian coastal sea level occurs at a relatively short lag of no more than ~1 month (not shown). This is consistent with an oceanic teleconnection propagated by CTW dynamics from the Western Pacific and Maritime Continent with the coast on their left (Fig. 12a, c, e, Miles et al. 2014; McIntosh et al. 2015), travelling at wave speeds near the first-baroclinic Kelvin wave speed in the tropics of ~3 m s⁻¹ (Chelton et al. 1998), which subsequently affect the shelf sea level through changes in coastal currents, mass and density (temperature and salinity). Other mechanisms that could explain the connection between ENSO and Australian coastal sea level include local air-sea fluxes and wind-driven Ekman transport (Roberts et al. 2016). The SSTs are generally cooler on the west and north-west coasts during El Niño events (blue contours in Fig. 12a, c, e), which drive positive (into the ocean) air–sea heat flux anomalies (confirmed through analysis of air–sea flux observations from OAFlux, see https://oaflux.whoi.edu/, not shown) of the wrong sign to contribute to the negative SLAs by locally driven thermosteric contraction. Wind anomalies during El Niño events do have an upwelling-favourable south-westerly component on the north-west shelf that may contribute to driving low SLAs (vectors in Fig. 12b, d, f). The Great Barrier Reef region also shows some evidence of weak northerly, upwelling favourable winds correlated with low coastal sea level, but elsewhere local wind-driven mechanisms at this time scale are not supported by the regression analysis. Therefore, the main mechanism is likely associated directly with CTWs and their impacts on currents, vertically integrated mass and density – Jacox et al. (2023) came to a similar conclusion for the California current.

Fig. 12.

Lag regressions of monthly mean sea level anomalies (a, c, e, including the IB) and the IB (b, d, f) onto the Niño 3.4 index at (a, b) −3, (c, d) 0 and (e, f) 3 months lag in the units of centimetres per standard deviation of the Niño 3.4 index (σ_N34) over the 1993–2018 period. Negative lags indicate the Niño 3.4 index leading. Contours (a, c and e) indicate lag regressions of SST anomalies (from NOAA’s OISST product over 1993–2018) onto the Niño 3.4 index, with red contours indicating positive values above 0.1°C/σ_N34 and blue contours indicating negative values below −0.1°C/σ_N34, with a contour level of 0.05°/σ_N34. (a, c and e) Note that there are no data in the Gulf of Carpentaria in the IMOS dataset, which is depicted by grey shading. Vectors (b, d and f) indicate lag regressions of wind anomalies (from ERA5 over 1993–2018) onto the Niño 3.4 index. The red arrow (a) indicates the pathway for coastal trapped waves propagating from the western equatorial Pacific Ocean.

There is a distinct reduction in most seasonal skill metrics across the Great Australian Bight (beyond the Esperance tide gauge, e.g. see Fig. 9), which may be associated with the widening of the shelf, increased dissipation of seasonal signals and an increased role for stochastic, high-frequency weather-driven variability, e.g. Woodham et al. (2013) showed that high-frequency CTW amplitudes increase in amplitude in this region. On the south-east, east and north-east coasts of Australia, the connection between ENSO and the local sea level is much weaker (Fig. 12a, c, e), resulting in a small monthly averaged sea level variance (Fig. 1c) and a lack of predictability (Fig. 4).

However, ENSO’s atmospheric teleconnections do have a local impact through the IB effect, reaching almost 1 cm per standard deviation of the Niño 3.4 index (Fig. 12b, d, f), and, possibly, through local wind-driven effects as discussed above. Although the strength of the IB connection to ENSO is relatively uniform around the coast, the biggest impact of IB on the skill of the total sea level forecast is on the east coast (Fig. 8, 11) because the relative strength of the IB (compared to the total sea level variance) is strongest here (orange line in Fig. 1c). The higher relative strength of the IB also explains why the skill on the north-east shelf (e.g. at Townsville, see Fig. 5) is highest in austral spring and early summer, when the atmospheric ENSO teleconnections to this region are strongest (e.g. Risbey et al. 2009). Note the distinct difference between the behaviour of the shelf and the open ocean here (Fig. 12a, c, e, Woodworth et al. 2019), with El Niño driving significant positive SSH anomalies off the shelf through thermosteric expansion associated with clear skies, atmospheric heating and warm SSTs, in contrast to the negative anomalies near the coast.

In Section 3.3 it was found that there was some skill compared to reference forecasts in the south-east at certain times of year beyond lead month 1 (e.g. August–October in Melbourne and Sydney, Fig. 5). The seasonal relationship between ENSO and south-eastern sea level is strongest around this time of year (see Supplementary Fig. S4), suggesting that this skill is likely due to ENSO teleconnections.

Other potential sources for seasonal predictability include the IOD, the SAM and the Madden–Julian Oscillation (MJO). Regression analysis suggests that the IOD does not drive much sea level variability along the Australian coastline, when isolated from its impact on ENSO (not shown, also see Frankcombe et al. 2015). However, note that the IOD does affect Australian coastal sea level through the IB (reaching ~1 cm, see fig. 2 of Roberts et al. 2016). The SAM explains up to 15% of the variance of sea level along the south coast (red line in Fig. 1c). However, beyond a month of lead time, the predictability of the SAM is limited and thought to be due to its connection to ENSO (Lim et al. 2013). The MJO has a strong influence on coastal sea level in the Gulf of Carpentaria at short lead times (e.g. Oliver and Thompson 2010), but weak elsewhere. A more detailed analysis of sea level forecast skill associated with the MJO will be undertaken in a subsequent study on subseasonal forecast skill. Given the weak connection of these other modes of variability to Australian coastal sea level, we conclude, in agreement with previous studies (Miles et al. 2014; McIntosh et al. 2015) that ENSO is the dominant source of predictability.

5.Summary and next steps

We have quantified the skill of the seasonal ensemble prediction system, ACCESS-S2, in forecasting monthly SLAs (from the mean climatology) around the Australian coast up to 8 months into the future as compared to satellite altimetry and tide gauge observations and various reference forecasts.

We showed that including the IB effect in the monthly SLA forecasts, as a post-processing step using ACCESS-S2’s atmospheric sea level pressure forecasts, improves skill at the majority of coastal locations (Fig. 8). Seasonal SLA forecast skill can be improved by adding the IB, even if the skill of the IB forecast alone is not significantly better than a reference forecast, due to an improved representation of the overall sea level variability and uncertainty. This improvement is particularly notable on the east coast, where the IB contributes a larger fraction of the total sea level variability.

Seasonal sea level forecast skill is highest on the north and west coasts, where oceanic teleconnections with ENSO by CTWs and their impacts on shelf currents, mass and density, are strongest (Section 4, also see Miles et al. 2014; White et al. 2014; McIntosh et al. 2015), and reduces anti-clockwise around the coast (Fig. 5). Atmospheric ENSO teleconnections also contribute some skill by the IB effect and wind-driven Ekman processes (Section 4), whereas other modes of variability and processes contribute minimal skill. Because ENSO is a relatively long-lived phenomenon, much of the skill of the ensemble mean SLA forecast is due to anomaly persistence. However, skill metrics that take into account the uncertainty information available in the ACCESS-S2 ensemble, such as the CRPS (Fig. 6) or the BSS for quantile exceedance probability forecasts (BSS; Fig. 9), do show significant improvement over reference forecasts at most leads on the north and west coasts, and at shorter leads on the south coast.

Real time ACCESS-S2 forecasts of seasonal SLAs (without the IB) are currently produced for the Pacific through the Climate and Oceans Support Program in the Pacific (COSPPac; see http://www.bom.gov.au/climate/pacific/outlooks/), and also provided to the Australian Defence Force by regular briefings. These basic offerings will be leveraged by the Australian Climate Service to develop more sophisticated products for the Australian coastal zone. A prototype seasonal high-water alert product suite for the Australian coastline, building on the approach of Dusek et al. (2022), is under development, incorporating real time ACCESS-S2 sea level forecasts, tide predictions and sea level rise estimates. The seasonal SLA forecast component of this system will be most useful along the north-west and west Australian coasts, where the predictable seasonal anomalies reach up to 50% of the total sea level variance (e.g. Fig. 1c). Note that the benefits and practicality of including sea level trends into such a product suite, as discussed in Section 2.3.3, still need to be determined. The potential benefit of coastal downscaling (e.g. Long et al. 2023; Jacox et al. 2023) and calibration approaches – such as the quantile–quantile matching used for ACCESS-S2 terrestrial forecasts, National Operations Centre (2019), also see McIntosh et al. (2015) – will also be explored during product development. Future work will also assess model sea level skill on subseasonal (1–6 weeks) time scales, with recent studies suggesting that there may be some potential at this time scale (e.g. Amaya et al. 2022) associated with Kelvin waves that have a peak period of 10–25 days around the Australian coastline (Woodham et al. 2013), and with the MJO in the Gulf of Carpentaria (Oliver and Thompson 2010). In the longer term, an operational service incorporating this new prototype product suite is planned to better inform decision making and increase Australian community resilience to coastal flooding hazards under climate change.

Supplementary material

Supplementary material is available online.