Detection of state-dependent prediction skill using an adaptable, machine learning-based approach

In weather and climate prediction, the predictive power of the initial atmosphere and ocean state is limited by the chaotic nature of the system. To address this, some researchers focus on identifying state-dependent prediction skill, which refers to configurations of the initial conditions that enhance prediction skill at longer time scales.    

Shackelford et al. (2025) present a framework for identifying state-dependent prediction skill using neural networks and self-organizing maps (SOMs), and offer an example application to prediction of North Atlantic atmospheric circulation. The framework consists first of an ensemble of neural networks that receive northern hemisphere sea surface temperature (SST) maps as input and predict a conditional distribution of 500 mb height over the North Atlantic Ocean 7-8 weeks later (Z500). Next, the authors identify regions of SST variability with known influence on the North Atlantic storm tracks, including El Nino Southern Oscillation (ENSO), the Pacific Decadal Oscillation (PDO), and the Atlantic Multidecadal Oscillation (AMO). Mean SST over these regions form the basis of the SOMs, which visualize the co-variability of the different ocean modes in a low dimensional space. The authors leverage the probabilistic nature of the neural networks to assess which SOM units, or regimes, often feature "confident" neural network predictions. This reveals four SST regimes that enhance Z500 prediction skill on seasonal time scales. Two regimes adhere to a previously documented linear relationship between ENSO and Z500, which imbues confidence in the methodology. The other two regimes feature El Nino states, but oppose the oft-cited ENSO-Z500 relationship. Instead, these regimes corroborate recent findings that emphasize the role of decadal SST variability in modulating ENSO teleconnections. 

Although the authors focus on seasonal prediction of Z500, they propose that the framework is broadly suitable for identifying state-dependent prediction skill on subseasonal-to-decadal timescales. However, the authors stress that domain expertise on the inputs/outputs is essential for this methodology, given the end-user choices often required when constructing the SOMs.