AEMO regional electricity demand forecasting.
Live saved forecasts · Separate September 2026 backtest · 15-minute intervals
How much forecast error do we avoid versus a naive forecast?
— FAV · — average absolute error avoided · — matched predictions.
·
Single baseline: 40% prior-year average (level-adjusted), 20% monthly, 25% weekly and 15% daily averages. Fixed before backtesting; see Methodology for exact lags.
* September 2026 held-out test, 30 daily origins, 0–24h in backtest mode; other modes use their matched scored forecasts. FAV = 100 × (1 − model MAE / naive MAE). Positive adds value; negative means more error. Same issue, target and actual revision. Error reduction is not monetary savings.
Hover or tap to inspect values; keyboard users can focus the chart and use arrow keys.
UTC · MW · Nominal 90% prediction ranges for selected models.
Daily MAE by target date (UTC), all scored lead times. FAV uses the same issue, target and actual revision. Partial days can change as actuals arrive; overlapping live forecasts count separately. These curves use the latest actual revisions.
Teal: selected model · gold: selected naive baseline
Hover, tap, or focus the chart and use arrow keys to inspect values.
Above zero: less error than the baseline. Below zero: more error.
Hover, tap, or focus the chart and use arrow keys to inspect values.
| UTC target date | Model MAE · MW | Naive MAE · MW | FAV | Matched predictions |
|---|
Hold a target day fixed and move through saved issue times. Solid teal: selected vintage · dashed gold: preceding vintage · charcoal: latest actuals, shown retrospectively.
Hover, tap, or focus the chart to inspect revisions.
All scored lead times combined: how well each model performs overall. Lower error is better; positive bias means overprediction. Skill uses matched weighted-seasonal forecasts.
| Model | Samples | MAE MW | RMSE MW | WAPE | Bias MW | 90% coverage | FAV vs baseline |
|---|
The same forecasts split into 0–6, 6–12, 12–18 and 18–24 hour buckets: how accuracy changes farther into the future. Leads are measured from each forecast issue time. Lower error is better.
Hover, tap, or focus the chart and use arrow keys to inspect values.
| Lead bucket | Weighted seasonal | LightGBM | SARIMA |
|---|
Compare the saved model’s training history with up to seven subsequent days. The windows do not overlap. LightGBM uses regional history; SARIMA uses hourly averages. The weighted baseline also uses prior-year history; NEM shows aggregate demand, not a separately fitted model.
Slate: training distribution · teal: new observations. Shaded Gaussian density curves use a shared bandwidth; area is approximately one. Hover or use arrow keys to inspect density.
| Diagnostic | Value | Interpretation |
|---|
KS and Cramér–von Mises test the null of equal distributions. Holm correction covers six tests per vintage. Their p-values assume independent observations; 15-minute demand is autocorrelated, so these are exploratory flags, not calibrated alarms or proof of model deterioration. PSI above 0.20 is a review signal. Seasonal residuals subtract training mean demand for the same quarter-hour of the week. Short windows and serial correlation can inflate apparent change. Backtest observations are later historical actuals; prospective mode requires newly ingested data.
Read these alongside FAV: narrow intervals are useful only when coverage is adequate. Positive bias means overprediction. Coverage target: 90%; lower interval score is better.
| Model | Bias MW | P95 absolute error MW | 90% coverage | Mean interval width MW | 90% interval score MW |
|---|
PSI > 0.20 flags investigation, not automatic retraining. Issue-time context only: previous 28 days versus latest 7 days, both before the latest forecast cutoff. This is separate from the selected post-training comparison above.