A Framework for Joint Verification and Evaluation of Seasonal Climate Services across Socioeconomic Sectors

Abstract

Assessing the information provided by coproduced climate services is a timely challenge, given the continuously evolving scientific knowledge and its increasing translation to address societal needs. Here, we propose a joint evaluation and verification framework to assess prototype services that provide seasonal forecast information based on the experience from the Horizon 2020 (H2020) Climate forecasts enabled knowledge services (CLARA) project. The quality and value of the forecasts generated by CLARA services were first assessed for five climate services utilizing the Copernicus Climate Change Service seasonal forecasts and responding to knowledge needs from the water resources management, agriculture, and energy production sectors. This joint forecast verification and service evaluation highlights various skills and values across physical variables, services, and sectors, as well as a need to bridge the gap between verification and user-oriented evaluation. We provide lessons learned based on the service developers’ and users’ experience and recommendations to consortia that may want to deploy such verification and evaluation exercises. Last, we formalize a framework for joint verification and evaluation in service development, following a transdisciplinary (from data purveyors to service users) and interdisciplinary chain (climate, hydrology, economics, and decision analysis). SIGNIFICANCE STATEMENT: Tools to communicate climate-related information to users, typically dam managers, irrigation consortia, or energy producers, are fast evolving to answer societal needs. It is crucial to estimate the quality of the provided information, along with economic, environmental, and/or societal gains. Here, we exemplify how to assess information quality and potential gains in five services that provide data and information for hydropower, solar power, irrigation, and water reservoirs in Europe and South America. Based on this work, we recommend 1) service developers to well anticipate such quality and value assessments, due to the number of actors to be involved; 2) flexibility when screening how to quantify quality and gain to account for decision contexts; and 3) sustained funding or collaborating platforms to ensure the iterative coevaluation process.