An investigation into improving El Niño-Southern Oscillation prediction based on temporal transformer architecture

Abstract

Precisely forecasting the EI Nino-Southern Oscillation (ENSO) holds significant importance in predicting seasonal climate. The Recurrent Neural Network (RNN) has been demonstrated to be the most efficacious approach for ENSO prediction. The localized nature of the recurrent neuron poses a challenge in capturing distant antecedents of ENSO. The transformer architecture has been utilized in the domain of natural language processing (NLP) for a significant period of time due to its capacity to attend to global features. This study presents the introduction of the ENSO transformer with recurrent neuron to the ENSO research community. The current investigation demonstrates the efficacy of the ENSO Transformer model in accurately predicting the upcoming monthly mean Nino index. As the lead time increased, a temporal progression was observed in the activation map values. The study's results indicate that various climatic precursors of ENSO events have a significant impact, and each of them exhibits distinct temporal patterns. This suggests that the transformer with recurrent neuron model could be a useful tool for diagnosis. The present research suggests employing the ENSO Transformer RNN in tandem with the variant-based deep learning approach to achieve short-term prediction. The present investigation utilizes a comprehensive dataset that covers the complete Nino region spanning from 1950 to 2019. In contrast to other frequently employed forecasting models, the model we put forth demonstrated superior performance in benchmark evaluations and exhibited greater accuracy in reproducing the variations in predictive precision.