Validation and Uncertainties

The below information contains excerpts from The International SST Fiducial Reference Measurement Network paper, with permission from its authors.

SST Validation in the ISFRN

It is important that the SSTskin measurements produced in the ISFRN are fully traceable to Systeme International (S.I) standards and as such are considered Fiducial Reference Measurements (FRM). Such measurements are crucial to quantify uncertainties in the satellite-derived SST Climate Data Record (CDR) and are a fundamental component of satellite SST validation. More information on FRM can be found on the FRM4STS website, however the mandatory defining characteristics of an FRM can be summarised as (Donlon et al. 2014):

• FRM measurements have documented evidence of SI traceability via inter-comparison of instruments under operational-like conditions.
• FRM measurements are independent from the satellite SST retrieval process.
• FRM measurement protocols and community-wide management practices (measurement, processing, archive, documents etc.) are defined and adhered to.
• An uncertainty budget for all FRM instruments and derived measurements is available and maintained, traceable where appropriate to SI ideally directly through an NMI.

In addition to this, for satellite SST validation, GCOS request that “ship-mounted [TIR] radiometers, accurately calibrated before and after each deployment to traceable national standards, must be maintained as a truly independent reference data set for inter-calibration of follow-on satellite missions; this is particularly important where gaps in data exist between follow-on missions; a modest global array of ~10 repeat lines in different atmospheric regimes is required; in situ radiometer sampling strategies must consider the variable nature of SST skin dynamics” (Donlon et al. 2014).

The development and evolution of shipborne radiometer designs over the years has resulted in a number of shipborne radiometers that fulfil this request from GCOS and qualify as acceptable FRM for SST CRD, with an accuracy equal to or greater than 0.1K (Wimmer et al. 2012). Shipborne radiometers can also provide in situ SSTskin for satellite validation in regions devoid of sensored moored or drifting buoys, such as around the Australian coast and Southern Ocean (Beggs et al. 2012). Shipborne radiometers are self-calibrating and calibrations derived from their internal blackbodies are regularly verified against an SI-traceable laboratory calibration target (Donlon et al. 2014, Donlon et al. 2008).

Figure 1: Example laboratory calibration of the ISAR instrument pre- and post- deployment on the Pont Aven cruise ship

The traceability of both the shipborne radiometers and the laboratory calibration targets are also confirmed on a regular basis through inter-comparisons such as the ESA-funded FRM for validation of Surface Temperature from Satellites (FRM4STS) campaign, held in 2016 (Theocharous et al. 2019 and project technical report). The regular calibration of shipborne thermal infrared (TIR) radiometers, along with the fact that they measure, at source, the same SSTskin that is measured from space after modification by its propagation through the atmosphere (Minnett and Corlett, 2012) make them reliable FRM measurements for validating satellite SSTskin measurements. More information on the protocols used to maintain the SI traceability of shipborne radiometers in the ISFRN can be found here.

The ISAR Validation Tool

The ISAR validation tool uses the Felyx match-up database (MDB) files and follows the procedure as described in Wimmer et al. 2012 by using the Felyx MDB files and working from the central radiometer and SLSTR pixel match outwards until the edge of the match-up window is reached. This is done for 5 match-up windows (1: 0.5h 1km; 2a: 0.5h 20km; 2b: 2h 1km; 3: 2h 20km; 4 6h 25km ) and for Gaussian and robust statistics producing location plots of the central match, histograms, dependence plots for wind speed, latitude and uncertainty. The plots and statistics are produced for each level 2 SLSTR variable, including D3, D2, N3, N2, auxiliary SST products and water surface temperature (WST i.e. the best available SST).

The criteria for the match-up process is to get the best match within a match-up window, rather than to attempt to select match-ups with pixels in different parts of SLSTR’s swath. We match ISFRN radiometer data to any SLSTR data available, so both day and night time overpasses are validated.

Uncertainties

Another important factor in the ISFRN data is the uncertainty associated with the data. A comprehensive and reliable uncertainty budget allows a dataset to be compared either with themselves or with a reference standard and as such, can be used with confidence. Uncertainties arise due to many reasons but can be grouped into the following primary categories (Donlon et al. 2014a) as:

• Instrument measurement uncertainty: those relating to instrument hardware e.g. detector noise, optical misalignment, etc.
• Retrieval/algorithm uncertainty: those relating to derived quantities e.g. the value of sea water emissivity, temporal differences between sea surface and atmospheric radiance measurements, etc.
• Application uncertainty: those relating to a specific application e.g. differences in the type of SST measurement (e.g. of SSTdepth or SSTskin).

These uncertainties can further be split into two categories; random or systematic and are explored further in Donlon et al. 2014.

Possibly the most comprehensive uncertainty budget and analysis for a ship borne radiometer to date has been produced by Werenfrid Wimmer (Wimmer et al. 2012, Wimmer and Robinson, 2016) for the ISAR radiometer. The ISAR uncertainty model was created based on FRM standards and requirements and so not only are pre- and post-deployment calibrations performed but also a per-measurement model is used. This means that not only are shipborne radiometers fully traceable to SI standards (Preston-Thomas et al. 1990) but it also reduces the need for a large number of nominally independent observations to reduce the random uncertainty (for example, in the case of drifting buoys). The ISAR uncertainty model is based on a breakdown of uncertainties for the entire end-to-end instrument and data processing system, and was developed on a first principle bases by analysing these components of the ISAR instrument and propagating their associated uncertainties through the measurement equation that is shown in Figure 2 (Wimmer and Robinson, 2016). Here you can get an idea of how the primary uncertainty categories ‘instrument’, ‘retrieval/algorithm’ and ‘application’ are taken into account and addressed. A more thorough investigation into how these categories relate specifically to shipborne radiometer SST measurements can be found in Donlon et al. 2014. The ISAR post processor, which was implemented following this model, produces an uncertainty value for each SST. These uncertainties show the degree of confidence a user can have in the SST measurement. Figures 3 and 4 show a typical plot of ISAR SST data and associated total uncertainty, respectively. A detailed description of the uncertainty model can be found in Wimmer and Robinson, 2016.

Figure 2: The ISAR SST Measurements equation featuring a breakdown of some of the main elements of the ISAR SST processor to show the factors that introduce uncertainties. The boxes coloured in blue represent type A uncertainties, boxes coloured in red show type B uncertainties, and boxes in red and blue contain both type A and type B uncertainties. R2T stands for radiation to temperature transformation, Rsea is the radiation from the sea, Rsky the radiation from the sky, ε the seawater emissivity, RBB1,2 the radiation from the two on-board black bodies, SigSea, SigSky, SigBB1,2 are the signal from the detector when viewing the sea, sky of the two black bodies (Wimmer and Robinson, 2016).

Figure 3: The ISAR SST data from 2004 to 2019

Figure 4:  The ISAR total uncertainty for all data from 2004 to 2019