Investigating Error Metrics for Satellite Rainfall Data at Hydrologically Relevant Scales

This paper addresses the following open question: What set of error metrics for satellite rainfall data can advance the hydrologic application of new-generation, high-resolution rainfall products over land? The authors’ primary aim is to initiate a framework for building metrics that are mutually interpretable by hydrologists (users) and algorithm developers (data producers) and to provide more insightful information on the quality of the satellite estimates. In addition, hydrologists can use the framework to develop a space–time error model for simulating stochastic realizations of satellite estimates for quantification of the implication on hydrologic simulation uncertainty. First, the authors conceptualize the error metrics in three general dimensions: 1) spatial (how does the error vary in space?); 2) retrieval (how “off” is each rainfall estimate from the true value over rainy areas?); and 3) temporal (how does the error vary in time?). They suggest formulations for error metrics specific to each dimension, in addition to ones that are already widely used by the community. They then investigate the behavior of these metrics as a function of spatial scale ranging from 0.04° to 1.0° for the Precipitation Estimation from Remotely Sensed Information Using Artificial Neural Networks (PERSIANN) geostationary infrared-based algorithm. It is observed that moving to finer space–time scales for satellite rainfall estimation requires explicitly probabilistic measures that are mathematically amenable to space–time stochastic simulation of satellite rainfall data. The probability of detection of rain as a function of ground validation rainfall magnitude is found to be most sensitive to scale followed by the correlation length for detection of rain. Conventional metrics such as the correlation coefficient, frequency bias, false alarm ratio, and equitable threat score are found to be modestly sensitive to scales smaller than 0.24° latitude/longitude. Error metrics that account for an algorithm’s ability to capture rainfall intermittency as a function of space appear useful in identifying the useful spatial scales of application for the hydrologist. It is shown that metrics evolving from the proposed conceptual framework can identify seasonal and regional differences in reliability of four global satellite rainfall products over the United States more clearly than conventional metrics. The proposed framework for building such error metrics can lay a foundation for better interaction between the data-producing community and hydrologists in shaping the new generation of satellite-based, high-resolution rainfall products, including those being developed for the planned Global Precipitation Measurement (GPM) mission.