Skill assessment of spatial maps for oceanographic modeling

In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier's archiving and manuscript policies are encouraged to visit: a r t i c l e i n f o a b s t r a c t Comparing data-derived and model-generated spatial maps of key variables is a critical part of oceanographic model skill assessment. Yet, many comparisons resort to presenting the maps side-by-side and saying that the two maps look similar. We describe a suite of techniques for comparing 2-dimensional spatial maps that allow for fuzziness in the comparison and that are included in the Map Comparison Kit software package. These techniques included Kappa, Fuzzy Kappa, Moving Window, and Aggregated Cells. We also introduce a new measure that we call " slip-and-slide. " The slip-and-slide technique is based on determining what has to be done (shift, rotate, re-scale axes) to the model map to get the model map to agree as close as possible to the data map. We simulated two commonly encountered spatial patterns on a 100 by 100 grid of cells (hotspot and gradient), and used the generated maps in two simulation experiments. The first experiment was designed to confirm that the new slip-and-slide method could recover known differences between the data and model maps (identical twin experiment), while the second experiment included all of the map comparison techniques applied to data and model maps that were generated from different response surfaces. The slip-and-slide method successfully recovered the known distortion parameters under ideal conditions and when the response variable differed in magnitude between the model and data maps. Slip-and-slide started to show some difficulty in recovering known distortion parameters when noise was added to the fit of the response surface used in the parameter searching. The second simulation experiment demonstrated the fuzzy techniques, and how the similarity between data and model maps can be quantified and evaluated using increasing degrees of fuzziness in the various techniques. The second simulation experiment also showed the success of the slip-and-slide method, as the various techniques all showed greater similarity between the data and distorted model maps than between the data and the original model maps. Many of the fuzzy methods were designed for comparing maps with many small features, whereas oceanographic maps typically involve large-scale patterns. We propose …