The CDF algorithm must be validated before applying it to real atmospheric data. Two main categories of tests are used: auto-consistency tests (which verify the internal coherence of input data) and mono-type fusion tests (which verify that the fusion produces physically meaningful results).

Auto-Consistency Tests

This test applies the CDF method to a single product using the same a priori information (profile and variance-covariance matrix) for both input and output. The test verifies that the input and output products differ only negligibly in terms of averaging kernel matrices, covariance matrices, and profiles.

The test passes when:

  • Differences between the original state vector and the one reconstructed with CDF formulas are well within the total error
  • The degrees of freedom (DOFs) calculated from the reconstructed averaging kernel match the original DOFs within 1%

Currently, compliance conditions are established heuristically. More rigorous mathematical conditions are needed and represent a subject for future development.

CDF(2022) Auto-Consistency

When CDF(2022) is applied with N=1 (single product) and using the same a priori variance-covariance matrix for input and output, the theory predicts that:

\mathbf{x}_{f} = \widehat{\mathbf{x}}_{1}

That is, the fused profile should be identical to the input profile. This identity holds if and only if the input data satisfy the fundamental relationship between the three characterisation matrices (see Prerequisites section, Eq. P2):

\mathbf{S}_{ai} = \left( \mathbf{I} – \mathbf{A}_{i} \right)^{- 1} \cdot \mathbf{S}_{i}

and numerical errors are negligible. Therefore, the CDF(2022) auto-consistency test is a powerful diagnostic: it verifies in a single step whether the basic requirements for CDF application are satisfied.

CDF(2015) Auto-Consistency

For CDF(2015), the auto-consistency condition is more stringent. When the same procedure is applied, the fused profile reduces to the input profile only if the following additional condition is met:

\mathbf{A}_{i}^{T}\mathbf{S}_{ni}^{- 1} \approx \mathbf{S}_{i}^{- 1}
(T1)

This condition, often not satisfied in practice, can be viewed as an additional requirement

Additional Auto-Consistency Checks

If the main auto-consistency tests (above) produce unsatisfactory results, supplementary checks can help identify the source of problems. These tests are typically data-dependent and empirical:

  • Symmetry — Verify that all covariance matrices are symmetric
  • AKM diagonal — Verify that all diagonal elements of averaging kernel matrices are non-negative (and typically ≤ 1)
  • Matrix relationship consistency — Test whether the three characterisation matrices satisfy the fundamental equations (P1)–(P3) from the Prerequisites section

Note: Problems often originate from corrupted input data or errors in data reading routines, rather than from the CDF algorithm itself.

Mono-Type Fusion Tests

This test applies the CDF method to two or more products of the same type (i.e., products retrieved using the same retrieval algorithm/code). It verifies that the fusion produces a physically meaningful result with improved precision.

Ideally, such tests should satisfy these conditions to isolate the fusion algorithm from other confounding factors:

  • Products retrieved on the same vertical grid (to avoid interpolation complications)
  • Products using the same a priori information (profile and variance-covariance matrix)
  • Alternatively, use the CDF itself to harmonize grids and a priori information before fusion testing

The test passes when the fused product exhibits:

  • Smooth, non-oscillating vertical structure
  • Total errors smaller than or equal to the input products’ errors
  • Degrees of freedom (DOFs) higher than the input products

As with auto-consistency tests, the conditions for determining test success are currently heuristic and require more rigorous mathematical formulation. Additionally, more rigorous conditions for determining test compliance need to be developed and tested on diverse datasets.

Test Index Test Description
1 CDF(2022) auto-consistency: total error covariance matrix inversion
2 CDF(2015) auto-consistency: noise error covariance matrix inversion

Table 1: Summary of auto-consistency test categories.