Sea Surface Temperature Forecasting Using Foundational Models: A Novel Approach Assessed in the Caribbean Sea
Abstract
:1. Introduction
2. Materials and Methods
2.1. Observational Datasets
2.2. Models’ Architecture
2.2.1. Chronos Model
- Time series normalization: to avoid bias given by the Equation (1):
- 2.
- Quantization method: to convert the scaled time series into discrete tokens. For this, we select bin centers denoted by and edges separating them into for . The quantization function is defined by Equation (2):
- 3.
- Prediction: the model provides multiple realizations of the future using the predicted distribution using a loss function given by Equation (3):
- 4.
- Dequantization method: to obtain the final prediction, which implements this transformation defined in Equation (4):
2.2.2. Lag-Llama Model
- Tokenization: to construct lagged features from the prior values of the time series. Given a sorted set of lag indices the lag operation is defined by where each entry of is given by . In that way, to construct lag features for some context-length window we need to satisfy a larger window with more historical points denoted by .
- Projection: the tokens processed are passed through a shared linear projection layer mapping the features to the hidden dimension of the attention module.
- Distribution head: the outputs from the masked transformer layers are used to predict the parameters of the forecast distribution of the next step; these parameters are given by the parametric distribution head.
2.3. Other Models for Comparison
2.4. Model Evaluation
2.5. Case Study: Forecast Assessment of Extreme Upwelling Events
3. Results
3.1. Evaluation of Models’ Performance
3.2. SST Time Series Predictions
Region | Lag | MEAN | STD | MIN | MAX | RANGE |
---|---|---|---|---|---|---|
1 | D1 | 0.003 | 0.155 | −1.117 | 0.761 | 1.878 |
D7 | −0.018 | 0.369 | −1.389 | 1.202 | 2.591 | |
D14 | −0.017 | 0.447 | −1.516 | 1.416 | 2.931 | |
D28 | −0.039 | 0.505 | −1.724 | 1.695 | 3.420 | |
2 | D1 | 0.002 | 0.169 | −0.668 | 0.946 | 1.614 |
D7 | −0.032 | 0.411 | −1.339 | 1.771 | 3.111 | |
D14 | −0.049 | 0.472 | −1.504 | 1.542 | 3.047 | |
D28 | −0.075 | 0.534 | −1.981 | 1.574 | 3.555 | |
3 | D1 | −0.003 | 0.140 | −0.661 | 0.707 | 1.368 |
D7 | −0.021 | 0.309 | −1.143 | 0.751 | 1.894 | |
D14 | −0.019 | 0.371 | −1.415 | 1.196 | 2.611 | |
D28 | −0.015 | 0.443 | −1.742 | 1.252 | 2.994 | |
4 | D1 | 0.001 | 0.160 | −0.772 | 0.533 | 1.306 |
D7 | 0.002 | 0.454 | −2.141 | 1.435 | 3.576 | |
D14 | 0.005 | 0.613 | −2.224 | 2.041 | 4.264 | |
D28 | 0.009 | 0.781 | −2.295 | 2.435 | 4.731 |
3.3. SST Spatial Predictions
3.4. Observed Extreme Upwelling Events and Their Forecast
3.4.1. Observed Upwelling Seasonality
3.4.2. Observed Annual Variability
Intensity (°C) | Duration (Days) | Frequency (Total Events) | ||||
---|---|---|---|---|---|---|
Region 1 | Region 4 | Region 1 | Region 4 | Region 1 | Region 4 | |
Mean | −0.74 | −0.83 | 10.67 | 10.89 | 5.10 | 4.80 |
Std | 0.09 | 0.11 | 5.02 | 6.68 | 2.96 | 3.09 |
MaV | −0.60 | −0.58 | 23 | 31 | 11 | 11 |
Year MaV | 2021 | 2011 | 2015 | 2002 | 2014/18 | 2009 |
MiV | −0.92 | −1.15 | 3 | 4 | 0 | 1 |
Year MiV | 2002 | 2002 | 2022 | 2022 | 2023 | 2005/11/16/22 |
3.4.3. Observed Interannual and Sub-Annual Variability
3.4.4. Model’s Ability to Forecast Upwelling Events
4. Discussion
Supplementary Materials
Author Contributions
Funding
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Bindoff, N.L.; Willebrand, J.; Artale, V.; Cazenave, A.; Gregory, J.M.; Gulev, S.; Hanawa, K.; Le Quere, C.; Levitus, S.; Nojiri, Y. Observations: Oceanic Climate Change and Sea Level; Cambridge University Press: Cambridge, UK, 2007. [Google Scholar]
- Levitus, S.; Antonov, J.I.; Boyer, T.P.; Locarnini, R.A.; Garcia, H.E.; Mishonov, A.V. Global Ocean Heat Content 1955–2008 in Light of Recently Revealed Instrumentation Problems. Geophys. Res. Lett. 2009, 36. [Google Scholar] [CrossRef]
- Doney, S.C.; Ruckelshaus, M.; Emmett Duffy, J.; Barry, J.P.; Chan, F.; English, C.A.; Galindo, H.M.; Grebmeier, J.M.; Hollowed, A.B.; Knowlton, N. Climate Change Impacts on Marine Ecosystems. Annu. Rev. Mar. Sci. 2012, 4, 11–37. [Google Scholar] [CrossRef] [PubMed]
- IPCC. The Physical Science Basis. In Contribution of Working Group I to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change; IPCC: Geneva, Switzerland, 2021. [Google Scholar]
- Clegg, P.; Mahon, R.; McConney, P.; Oxenford, H. The Caribbean Blue Economy; Routledge: London, UK, 2020; ISBN 1-00-022707-3. [Google Scholar]
- Martinez, S. THEMATIC REPORT FOR THE CENTRAL/SOUTH AMERICAN SUB-REGION. Unpublished. In CLME Project Implementation Unit, Centre for Resource Management and Environmental Studies (CERMES); University of the West Indies: Saint Michael, Barbados, 2007; Available online: https://clmeplus.org/web/app/uploads/2020/04/CLME-PDF-B-2007-Preliminary-TDA-Central-South-America.pdf (accessed on 1 February 2025).
- Wilkinson, T.A. Marine Ecoregions of North America; Instituto Nacional de Ecología: Mexico City, Mexico, 2009; ISBN 2-923358-41-4. [Google Scholar]
- Andrade-Amaya, C.A. The Circulation and Variability of the Colombian Basin in the Caribbean Sea. Ph.D. Tesis, University of Wales, Cardiff, Wales, 2000. [Google Scholar]
- Torres, R.R.; Latandret, S.; Salon, J.; Dagua, C. Water Masses in the Caribbean Sea and Sub-Annual Variability in the Guajira Upwelling Region. Ocean Dyn. 2023, 73, 39–57. [Google Scholar] [CrossRef]
- Penland, C.; Matrosova, L. Prediction of Tropical Atlantic Sea Surface Temperatures Using Linear Inverse Modeling. J. Clim. 1998, 11, 483–496. [Google Scholar] [CrossRef]
- Repelli, C.A.; Nobre, P. Statistical Prediction of Sea-Surface Temperature over the Tropical Atlantic. Int. J. Climatol. 2004, 24, 45–55. [Google Scholar] [CrossRef]
- Ashby, S.A.; Taylor, M.A.; Chen, A.A. Statistical Models for Predicting Rainfall in the Caribbean. Theor. Appl. Climatol. 2005, 82, 65–80. [Google Scholar] [CrossRef]
- Villegas, N.; Malikov, I.; Farneti, R. Sea Surface Temperature in Continental and Insular Coastal Colombian Waters: Observations of the Recent Past and near-Term Numerical Projections. Lat. Am. J. Aquat. Res. 2021, 49, 307–328. [Google Scholar] [CrossRef]
- Bell, M.J.; Schiller, A.; Le Traon, P.-Y.; Smith, N.R.; Dombrowsky, E.; Wilmer-Becker, K. An Introduction to GODAE OceanView. J. Oper. Oceanogr. 2015, 8, s2–s11. [Google Scholar] [CrossRef]
- Rawlins, F.; Ballard, S.P.; Bovis, K.J.; Clayton, A.M.; Li, D.; Inverarity, G.W.; Lorenc, A.C.; Payne, T.J. The Met Office Global Four-dimensional Variational Data Assimilation Scheme. Q. J. R. Meteorol. Soc. J. Atmospheric Sci. Appl. Meteorol. Phys. Oceanogr. 2007, 133, 347–362. [Google Scholar] [CrossRef]
- Hey, T.; Tansley, S.; Tolle, K.M. The Fourth Paradigm: Data-Intensive Scientific Discovery; Microsoft research Redmond: Redmond, WA, USA, 2009; Volume 1. [Google Scholar]
- Bommasani, R.; Hudson, D.A.; Adeli, E.; Altman, R.; Arora, S.; von Arx, S.; Bernstein, M.S.; Bohg, J.; Bosselut, A.; Brunskill, E. On the Opportunities and Risks of Foundation Models. arXiv 2021, arXiv:210807258. [Google Scholar]
- Jahanbakht, M.; Xiang, W.; Azghadi, M.R. Sea Surface Temperature Forecasting with Ensemble of Stacked Deep Neural Networks. IEEE Geosci. Remote Sens. Lett. 2021, 19, 1502605. [Google Scholar] [CrossRef]
- Taylor, J.; Feng, M. A Deep Learning Model for Forecasting Global Monthly Mean Sea Surface Temperature Anomalies. Front. Clim. 2022, 4, 932932. [Google Scholar] [CrossRef]
- Fei, T.; Huang, B.; Wang, X.; Zhu, J.; Chen, Y.; Wang, H.; Zhang, W. A Hybrid Deep Learning Model for the Bias Correction of Sst Numerical Forecast Products Using Satellite Data. Remote Sens. 2022, 14, 1339. [Google Scholar] [CrossRef]
- Ansari, A.F.; Stella, L.; Turkmen, C.; Zhang, X.; Mercado, P.; Shen, H.; Shchur, O.; Rangapuram, S.S.; Arango, S.P.; Kapoor, S. Chronos: Learning the Language of Time Series. arXiv 2024, arXiv:240307815. [Google Scholar]
- Rasul, K.; Ashok, A.; Williams, A.R.; Khorasani, A.; Adamopoulos, G.; Bhagwatkar, R.; Biloš, M.; Ghonia, H.; Hassen, N.V.; Schneider, A. Lag-Llama: Towards Foundation Models for Time Series Forecasting. arXiv 2023, arXiv:231008278. [Google Scholar]
- Zhou, T.; Niu, P.; Sun, L.; Jin, R. One Fits All: Power General Time Series Analysis by Pretrained Lm. Adv. Neural Inf. Process. Syst. 2023, 36, 43322–43355. [Google Scholar]
- Jin, M.; Wang, S.; Ma, L.; Chu, Z.; Zhang, J.Y.; Shi, X.; Chen, P.-Y.; Liang, Y.; Li, Y.-F.; Pan, S. Time-Llm: Time Series Forecasting by Reprogramming Large Language Models. arXiv 2023, arXiv:231001728. [Google Scholar]
- Montoya-Sánchez, R.A.; Devis-Morales, A.; Bernal, G.; Poveda, G. Seasonal and Interannual Variability of the Mixed Layer Heat Budget in the Caribbean Sea. J. Mar. Syst. 2018, 187, 111–127. [Google Scholar] [CrossRef]
- Varona, H.L.; Veleda, D.; Silva, M.; Cintra, M.; Araujo, M. Amazon River Plume Influence on Western Tropical Atlantic Dynamic Variability. Dyn. Atmos. Ocean. 2019, 85, 1–15. [Google Scholar] [CrossRef]
- Saha, A.; Serra, N.; Stammer, D. Growth and Decay of Northwestern Tropical Atlantic Barrier Layers. J. Geophys. Res. Oceans 2021, 126, e2020JC016956. [Google Scholar] [CrossRef]
- Balaguru, K.; Chang, P.; Saravanan, R.; Leung, L.R.; Xu, Z.; Li, M.; Hsieh, J.-S. Ocean Barrier Layers’ Effect on Tropical Cyclone Intensification. Proc. Natl. Acad. Sci. USA 2012, 109, 14343–14347. [Google Scholar] [CrossRef] [PubMed]
- Mignot, J.; Lazar, A.; Lacarra, M. On the Formation of Barrier Layers and Associated Vertical Temperature Inversions: A Focus on the Northwestern Tropical Atlantic. J. Geophys. Res. Oceans 2012, 117. [Google Scholar] [CrossRef]
- Foltz, G.R.; McPhaden, M.J. Impact of Barrier Layer Thickness on SST in the Central Tropical North Atlantic. J. Clim. 2009, 22, 285–299. [Google Scholar] [CrossRef]
- Bustos Usta, D.F.; Torres Parra, R.R.; Rodríguez-López, L.; Castillo Alvarez, M.; Bourrel, L. Observation and Projection of Marine Heatwaves in the Caribbean Sea from CMIP6 Models. Remote Sens. 2024, 16, 2357. [Google Scholar] [CrossRef]
- Rueda-Roa, D.T.; Muller-Karger, F.E. The Southern Caribbean Upwelling System: Sea Surface Temperature, Wind Forcing and Chlorophyll Concentration Patterns. Deep Sea Res. Part Oceanogr. Res. Pap. 2013, 78, 102–114. [Google Scholar] [CrossRef]
- Alonso del Rosario, J.J.; Vidal Pérez, J.M.; Blázquez Gómez, E. On the Prediction of Upwelling Events at the Colombian Caribbean Coasts from Modis-SST Imagery. Sensors 2019, 19, 2861. [Google Scholar] [CrossRef]
- Rueda-Roa, D.T.; Ezer, T.; Muller-Karger, F.E. Description and Mechanisms of the Mid-Year Upwelling in the Southern Caribbean Sea from Remote Sensing and Local Data. J. Mar. Sci. Eng. 2018, 6, 36. [Google Scholar] [CrossRef]
- Townhill, B.L.; Birchenough, S.N.; Engelhard, G.H.; Harrod, O.; McHarg, E.; Monnereau, I.; Buckley, P.J. Responding to Climate Change in Caribbean Fisheries and Aquaculture through Adaptation; CLME Hub: Cartagena, Colombia, 2021; pp. 1–57. [Google Scholar]
- GEBCO. GEBCO Gridded Bathymetry Data. Available online: https://www.gebco.net/data_and_products/gridded_bathymetry_data/ (accessed on 1 February 2025).
- Chin, T.M.; Vazquez-Cuervo, J.; Armstrong, E.M. A Multi-Scale High-Resolution Analysis of Global Sea Surface Temperature. Remote Sens. Environ. 2017, 200, 154–169. [Google Scholar] [CrossRef]
- Giesen, R.; Stoffelen, A.; Verhoef, A. Product User Manual; Copernicus Marine Service: 2024. Available online: https://documentation.marine.copernicus.eu/PUM/CMEMS-WIND-PUM-012-004-006.pdf (accessed on 31 January 2025).
- Raffel, C.; Shazeer, N.; Roberts, A.; Lee, K.; Narang, S.; Matena, M.; Zhou, Y.; Li, W.; Liu, P.J. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res. 2020, 21, 1–67. [Google Scholar]
- Ansari, A.F.; Heng, A.; Lim, A.; Soh, H. Neural continuous-discrete state space models for irregularly-sampled time series. In Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA, 23–29 July 2023; pp. 926–951. [Google Scholar]
- Zhang, B.; Sennrich, R. Root Mean Square Layer Normalization. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada, 8–14 December 2019. [Google Scholar]
- Su, J.; Ahmed, M.; Lu, Y.; Pan, S.; Bo, W.; Liu, Y. Roformer: Enhanced Transformer with Rotary Position Embedding. Neurocomputing 2024, 568, 127063. [Google Scholar] [CrossRef]
- Wang, H.; Huang, J.; Zhou, H.; Zhao, L.; Yuan, Y. An Integrated Variational Mode Decomposition and ARIMA Model to Forecast Air Temperature. Sustainability 2019, 11, 4018. [Google Scholar] [CrossRef]
- De Saa, E.; Ranathunga, L. Comparison between Arima and Deep Learning Models for Temperature Forecasting. arXiv 2020, arXiv:201104452. [Google Scholar]
- Nawi, W.; Lola, M.S.; Zakariya, R.; Zainuddin, N.H.; Abd Hamid, A.A.K.; Aruchunan, E.; Nazzrol, N.S.A. Improved of Forecasting Sea Surface Temperature Based on Hybrid Arima and Support Vector Machines Models. Malays. J. Fundam. Appl. Sci. 2021, 17, 609–620. [Google Scholar] [CrossRef]
- Taylor, S.J.; Letham, B. Forecasting at Scale. Am. Stat. 2018, 72, 37–45. [Google Scholar] [CrossRef]
- van de Sande, S.N.; Alsahag, A.M.; Mohammadi Ziabari, S.S. Enhancing the Predictability of Wintertime Energy Demand in The Netherlands Using Ensemble Model Prophet-LSTM. Processes 2024, 12, 2519. [Google Scholar] [CrossRef]
- Zrira, N.; Kamal-Idrissi, A.; Farssi, R.; Khan, H.A. Time Series Prediction of Sea Surface Temperature Based on BiLSTM Model with Attention Mechanism. J. Sea Res. 2024, 198, 102472. [Google Scholar] [CrossRef]
- Torres, R.R.; Tsimplis, M.N. Seasonal Sea Level Cycle in the Caribbean Sea. J. Geophys. Res. Oceans 2012, 117. [Google Scholar] [CrossRef]
- Benazzouz, A.; Mordane, S.; Orbi, A.; Chagdali, M.; Hilmi, K.; Atillah, A.; Pelegrí, J.L.; Hervé, D. An Improved Coastal Upwelling Index from Sea Surface Temperature Using Satellite-Based Approach—The Case of the Canary Current Upwelling System. Cont. Shelf Res. 2014, 81, 38–54. [Google Scholar] [CrossRef]
- Jacox, M.G.; Edwards, C.A.; Hazen, E.L.; Bograd, S.J. Coastal Upwelling Revisited: Ekman, Bakun, and Improved Upwelling Indices for the US West Coast. J. Geophys. Res. Oceans 2018, 123, 7332–7350. [Google Scholar] [CrossRef]
- Bograd, S.J.; Jacox, M.G.; Hazen, E.L.; Lovecchio, E.; Montes, I.; Pozo Buil, M.; Shannon, L.J.; Sydeman, W.J.; Rykaczewski, R.R. Climate Change Impacts on Eastern Boundary Upwelling Systems. Annu. Rev. Mar. Sci. 2023, 15, 303–328. [Google Scholar] [CrossRef]
- Fielding, P.J.; Davis, C.L. Carbon and Nitrogen Resources Available to Kelp Bed Filter Feeders in an Upwelling Environment. Mar. Ecol. Prog. Ser. 1989, 55, 181–189. [Google Scholar] [CrossRef]
- Orfila, A.; Urbano-Latorre, C.P.; Sayol, J.M.; Gonzalez-Montes, S.; Caceres-Euse, A.; Hernández-Carrasco, I.; Muñoz, Á.G. On the Impact of the Caribbean Counter Current in the Guajira Upwelling System. Front. Mar. Sci. 2021, 8, 626823. [Google Scholar] [CrossRef]
- Schlegel, R.W.; Smit, A.J. heatwaveR: A Central Algorithm for the Detection of Heatwaves and Cold-Spells. J. Open Source Softw. 2018, 3, 821. [Google Scholar] [CrossRef]
- Abrahams, A.; Schlegel, R.W.; Smit, A.J. Variation and Change of Upwelling Dynamics Detected in the World’s Eastern Boundary Upwelling Systems. Front. Mar. Sci. 2021, 8, 626411. [Google Scholar] [CrossRef]
- Bustos Usta, D.F.; Torres Parra, R.R. Projected Wind Changes in the Caribbean Sea Based on CMIP6 Models. Clim. Dyn. 2022, 60, 3713–3727. [Google Scholar] [CrossRef]
- Gamble, D.W.; Curtis, S. Caribbean Precipitation: Review, Model and Prospect. Prog. Phys. Geogr. Earth Environ. 2008, 32, 265–276. [Google Scholar] [CrossRef]
- Bustos, D.; Torres, R. Ocean and Atmosphere Changes in the Caribbean Sea during the Twenty-First Century Using CMIP5 Models. Ocean Dyn. 2021, 71, 757–777. [Google Scholar] [CrossRef]
Event | Date_Start | Date_End | Mean_Intensity (°C) | Duration (Days) |
---|---|---|---|---|
R1_I | 10 September 2021 | 14 September 2021 | −0.602 | 5 |
R1_D | 28 November 2022 | 4 December 2022 | −0.585 | 7 |
R4_I | 2 April 2022 | 4 April 2022 | −0.631 | 3 |
R4_D | 4 October 2021 | 7 October 2021 | −0.634 | 4 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Bustos Usta, D.F.; Rodríguez-López, L.; Torres Parra, R.R.; Bourrel, L. Sea Surface Temperature Forecasting Using Foundational Models: A Novel Approach Assessed in the Caribbean Sea. Remote Sens. 2025, 17, 517. https://doi.org/10.3390/rs17030517
Bustos Usta DF, Rodríguez-López L, Torres Parra RR, Bourrel L. Sea Surface Temperature Forecasting Using Foundational Models: A Novel Approach Assessed in the Caribbean Sea. Remote Sensing. 2025; 17(3):517. https://doi.org/10.3390/rs17030517
Chicago/Turabian StyleBustos Usta, David Francisco, Lien Rodríguez-López, Rafael Ricardo Torres Parra, and Luc Bourrel. 2025. "Sea Surface Temperature Forecasting Using Foundational Models: A Novel Approach Assessed in the Caribbean Sea" Remote Sensing 17, no. 3: 517. https://doi.org/10.3390/rs17030517
APA StyleBustos Usta, D. F., Rodríguez-López, L., Torres Parra, R. R., & Bourrel, L. (2025). Sea Surface Temperature Forecasting Using Foundational Models: A Novel Approach Assessed in the Caribbean Sea. Remote Sensing, 17(3), 517. https://doi.org/10.3390/rs17030517