Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Publisher: Royal Meteorological Society
Languages: English
Types: Article
To improve the quantity and impact of observations used in data assimilation it is necessary to take into account the full, potentially correlated, observation error statistics. A number of methods for estimating correlated observation errors exist, but a popular method is a diagnostic that makes use of statistical averages of observation-minus-background and observation-minus-analysis residuals. The accuracy of the results it yields is unknown as the diagnostic is sensitive to the difference between the exact background and exact observation error covariances and those that are chosen for use within the assimilation. It has often been stated in the literature that the results using this diagnostic are only valid when the background and observation error correlation length scales are well separated. Here we develop new theory relating to the diagnostic. For observations on a 1D periodic domain we are able to the show the effect of changes in the assumed error statistics used in the assimilation on the estimated observation error covariance matrix. We also provide bounds for the estimated observation error variance and eigenvalues of the estimated observation error correlation matrix. We demonstrate that it is still possible to obtain useful results from the diagnostic when the background and observation error length scales are similar. In general, our results suggest that when correlated observation errors are treated as uncorrelated in the assimilation, the diagnostic will underestimate the correlation length scale. We support our theoretical results with simple illustrative examples. These results have potential use for interpreting the derived covariances estimated using an operational system.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Balgovind R, Dalcher A, Ghil M, Kalnay E. 1983. A stochastic-dynamic model for the spatial structure of forecast error statistics. Mon. Weather Rev. 111: 701-722.
    • Bannister RN. 2008. A review of forecast error covariance statistics in atmospheric variational data assimilation. I: Characteristics and measurements of forecast error covariances. Q. J. R. Meteorol. Soc. 134: 1951-1970.
    • Bormann N, Bauer P. 2010. Estimates of spatial and interchannel observationerror characteristics for current sounder radiances for numerical weather prediction. I: Methods and application to ATOVS data. Q. J. R. Meteorol. Soc. 136: 1036-1050.
    • Bormann N, Saarinen S, Kelly G, Thepaut J. 2003. The spatial structure of observation errors in atmospheric motion vectors from geostationary satellite data. Mon. Weather Rev. 131: 706-718.
    • Bormann N, Collard A, Bauer P. 2010. Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: Application to AIRS and IASI data. Q. J. R. Meteorol. Soc. 136: 1051-1063.
    • Dee DP, Da Silva AM. 1999. Maximum-likelihood estimation of forecast and observation error covariance parameters. Part I: Methodology. Mon. Weather Rev. 127: 1822-1843.
    • Desroziers G, Ivanov S. 2001. Diagnosis and adaptive tuning of observationerror parameters in a variational assimilation. Q. J. R. Meteorol. Soc. 127: 1433-1452, doi: 10.1002/qj.49712757417.
    • Desroziers G, Berre L, Chapnik B, Poli P. 2005. Diagnosis of observation, background and analysis-error statistics in observation space. Q. J. R. Meteorol. Soc. 131: 3385-3396.
    • Desroziers G, Berre L, Chapnik. 2009. 'Objective validation of data assimilation systems: Diagnosing sub-optimality'. In Proceedings of ECMWF Workshop on Diagnostics of Data Aassimilation System Performance, 15-17 June 2009, Reading, UK.
    • Golub GH, Van Loan CF. 2013. Matrix Computations, Chapter 7 (4th edn). The Johns Hopkins University Press: Baltimore, MD.
    • Gray RM. 2006. Toeplitz and Circulant Matrices: A Review. Foundations and Trends in Communications and Information Theory: 2: 155 - 239, doi: 10.1561/0100000006.
    • Healy SB, White AA. 2005. Use of discrete Fourier transforms in the 1D-Var retrieval problem. Q. J. R. Meteorol. Soc. 131: 63 - 72.
    • Hollingsworth A, Lo¨nnberg P. 1986. The statistical structure of short-range forecast errors as determined from radiosonde data. Part I: The wind field Tellus 38A: 111 - 136.
    • Ingleby NB. 2001. The statistical structure of forecast errors and its representation in the Met. Office global 3-D variational data assimilation scheme. Q. J. R. Meteorol. Soc. 127: 209, doi: 10.1002/qj.49712757112.
    • Janjic T, Cohn SE. 2006. Treatment of observation error due to unresolved scales in atmospheric data assimilation. Mon. Weather Rev. 134: 2900 - 2915.
    • Li H, Kalnay E, Miyoshi T. 2009. Simultaneous estimation of covariance inflation and observation errors within an ensemble Kalman filter. Q. J. R. Meteorol. Soc. 128: 1367 - 1386.
    • Lorenc AC. 1981. A global three-dimensional multivariate statistical interpolation scheme. Mon. Weather Rev. 109: 701 - 721.
    • Lorenc AC. 1986. Analysis methods for numerical weather prediction. Q. J. R. Meteorol. Soc. 112: 1177 - 1194.
    • Me´nard R, Yang Y, Rochon Y. 2009. 'Convergence and stability of estimated error variances derived from assimilation residuals in observation space'. In Proceedings of ECMWF Workshop on Diagnostics of Data Assimilation System Performance, 15-17 June 2009, Reading, UK.
    • Miyoshi T, Kalnay E, Li H. 2013. Estimating and including observation-error correlations in data assimilation. Inverse Prob. Sci. Eng. 21: 387 - 398.
    • Simonin D, Ballard SP, Li Z. 2014. Doppler radar radial wind assimilation using an hourly cycling 3D-Var with a 1.5 km resolution version of the Met Office Unified Model for nowcastings. Q. J. R. Meteorol. Soc. 140: 2298 - 2314, doi: 10.1002/qj.2298.
    • Stewart LM. 2010. 'Correlated observation errors in data assimilation', PhD thesis. University of Reading: http://www.reading.ac.uk/maths-andstats/research/theses/maths-phdtheses.aspx (accessed 1 October 2015).
    • Stewart LM, Dance SL, Nichols NK. 2008. Correlated observation errors in data assimilation. Int. J. Numer. Methods Fluids 56: 1521 - 1527.
    • Stewart LM, Dance SL, Nichols NK. 2013. Data assimilation with correlated observation errors: experiments with a 1-D shallow water model. Tellus A 65, doi: 10.3402/tellusa.v65i0.19546.
    • Stewart LM, Dance SL, Nichols NK, Eyre JR, Cameron J. 2014. Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system. Q. J. R. Meteorol. Soc. 140: 1236 - 1244, doi: 10.1002/qj.2211.
    • Todling R. 2015. A complementary note to 'A lag-1 smoother approach to system-error estimation': The intrinsic limitations of residual diagnostics. Q. J. R. Meteorol. Soc., doi: 10.1002/qj.2546.
    • Waller JA. 2013. 'Using observations at different spatial scales in data assimilation for environmental prediction', PhD thesis. University of Reading, Department of Mathematics and Statistics, http://www.reading.ac.uk/ maths-and-stats/research/theses/maths-phdtheses.aspx (accessed 1 October 2015).
    • Waller JA, Dance SL, Lawless AS, Nichols NK. 2014a. Estimating correlated observation error statistics using an ensemble transform Kalman filter. Tellus A 66: doi: 10.3402/tellusa.v66.23294.
    • Waller JA, Dance SL, Lawless AS, Nichols NK, Eyre JR. 2014b. Representativity error for temperature and humidity using the Met Office high-resolution model. Q. J. R. Meteorol. Soc. 140: 1189 - 1197, doi: 10.1002/qj.2207.
    • Waller JA, Simonin D, Dance SL, Nichols NK, Ballard SP. 2015. Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minusbackground and observation-minus-analysis statistics. University of Reading: http://www.reading.ac.uk/web/FILES/maths/Preprint MPS 15- 18 Waller.pdf (accessed 1 October 2015).
    • Weston PP, Bell W, Eyre JR. 2014. Accounting for correlated error in the assimilation of high-resolution sounder data. Q. J. R. Meteorol. Soc. 140: 2420 - 2429, doi: 10.1002/qj.2306.
    • Yaglom A. 1986. Correlation Theory of Stationary and Related Random Functions I. Basic Results. Springer-Verlag: New York, NY.
  • No related research data.
  • Discovered through pilot similarity algorithms. Send us your feedback.

Share - Bookmark

Cite this article