Remember Me
Or use your Academic/Social account:


Or use your Academic/Social account:


You have just completed your registration at OpenAire.

Before you can login to the site, you will need to activate your account. An e-mail will be sent to you with the proper instructions.


Please note that this site is currently undergoing Beta testing.
Any new content you create is not guaranteed to be present to the final version of the site upon release.

Thank you for your patience,
OpenAire Dev Team.

Close This Message


Verify Password:
Verify E-mail:
*All Fields Are Required.
Please Verify You Are Human:
fbtwitterlinkedinvimeoflicker grey 14rssslideshare1
Xu, Qin (2007)
Publisher: Co-Action Publishing
Journal: Tellus A
Languages: English
Types: Article
The relative entropy is compared with the previously used Shannon entropy difference as a measure of the amount of information extracted from observations by an optimal analysis in terms of the changes in the probability density function (pdf) produced by the analysis with respect to the background pdf. It is shown that the relative entropy measures both the signal and dispersion parts of the information content from observations, while the Shannon entropy difference measures only the dispersion part. When the pdfs are Gaussian or transformed to Gaussian, the signal part of the information content is given by a weighted inner-product of the analysis increment vector and the dispersion part is given by a non-negative definite function of the analysis and background covariance matrices. When the observation space is transformed based on the singular value decomposition of the scaled observation operator, the information content becomes separable between components associated with different singular values. Densely distributed observations can be then compressed with minimum information loss by truncating the components associate with the smallest singular values. The differences between the relative entropy and Shannon entropy difference in measuring information content and information loss are analysed in details and illustrated by examples.
  • The results below are discovered through our pilot algorithms. Let us know how we are doing!

    • Bennett, A. F. 1992. Inverse Method in Physical Oceanography. Cambridge University Press, New York, 346 pp.
    • Bernardo, J. M. and Smith, A. F. M. 1994. Bayesian Theory. John Wiley and Sons, New York, 586 pp.
    • Daley, R. 1991. Atmospheric Data Analysis. Cambridge University Press, New York, 457 pp.
    • Doviak, J. D. and Zrnic, D. S. 1993. Doppler Radar and Weather Observations. 2nd Edition. Academic Press, New York, 562 pp.
    • Eyre, J. R. 1990. The information content of data from satellite sounding systems. A simulation study. Q. J. R. Meteor. Soc. 116, 401-434.
    • Golub, G. H. and Van Loan, C. F. 1983. Matrix Computations. Johns Hopkins University Press, Baltimore, 476 pp.
    • Gong, J., Wang, L. and Xu, Q. 2003. A three-step dealiasing method for Doppler velocity data quality control. J. Atmos. Ocean. Technol. 20, 1738-1748.
    • Haven, K., Majda, A. J. and Abramov, R. 2005. Quantifying predictability through information theory. small sample estimation in a nonGaussian framework. J. Comp. Phys. 206, 334-362.
    • Hodur, R. M. 1997. The Naval Research Laboratory's coupled ocean/atmosphere mesoscale prediction system (COAMPS). Mon. Wea. Rev. 125, 1414-1430.
    • Houtekamer, P. L. and Mitchell, H. L. 2001. A sequential ensemble Kalman filter for atmospheric data assimilation. Mon. Wea. Rev. 129, 123-137.
    • Huang, H.-L. and Purser, R. J. 1996. Objective measures of the information density of satellite data. Meteorol. Atmos. Phys. 60, 105-117.
    • Jazwinski, A. H. 1970. Stochastic Processes and Filtering Theory. Academic Press, San Diego, 376 pp.
    • Kleeman, R. 2002. Measuring dynamical prediction utility using relative entropy. J. Atmos. Sci. 59, 2057-2072.
    • Kleeman, R. and Majda, A. J. 2005. Predictability in a model for geophysical turbulence. J. Atmos. Sci. 62, 2864-2879.
    • Liu, S., Xu, Q. and Zhang, P. 2005a. Quality control of Doppler velocities contaminated by migrating birds. Part II: Bayes identification and probability tests. J. Atmos. Oceanic Technol. 22, 1114-1121.
    • Liu, S., Xue, M., Gao, J. and Parrish, D. F. 2005b. Analysis and impact of super-obbed Doppler radial velocity in the NCEP grid-point statistical interpolation (GSI) analysis system. Extended abstract. 17th Conf. Num. Wea. Pred. Washington DC, Amer. Meteor. Soc. 13A.4.
    • Majda, A. J., Kleeman, R. and Cai, D. 2002. A mathematical framework for quantifying predictability through relative entropy. Methods Appl. Anal. 9, 425-444.
    • Majda, A. J. and Wang, X. 2006. Nonlinear Dynamics and Statistical Theories for Basic Geophysical Flows. Cambridge University Press, New York, 565 pp.
    • O'Hagan, A. 1994. Kendall's Advanced Theory of Statistics. Volume 2B. Bayesian Inference. Oxford University Press. New York, 332 pp.
    • Peckham, G. 1974. The information content of remote measurements of atmospheric temperature by satellite infra-red radiometry and optimum radiometer configurations. Q. J. R. Meteor. Soc. 100, 406-419.
    • Papoulis, A. 1991. Probability, Random Variables, and Stochastic Processes. 3rd Edition, McGraw-Hill, New York, 666 pp.
    • Purser, R. J., Parrish, D. F. and Masutani, M. 2000. Meteorological observational data compression; An alternative to conventional “super-Obbing”. Office Note 430, National Centers for Environmental Prediction, Camp Springs, MD, 12 pp. [available on line http://www.emc.ncep.noaa.gov/officenotes/FullTOC.html#2000]
    • Purser, R. J., Wu, W.-S., Parrish, D. F. and Roberts, N. M. 2003. Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev. 131, 1524-1535.
    • Shannon, C. E. 1949. Communication in the presence of noise. Proc. I.C.E. 37, 10-21.
    • Whitaker, J. S. and Hamill, T. M. 2002. Ensemble data assimilation without perturbed observations. Mon. Wea. Rev. 130, 1913-1924.
    • Wu, W.-S., Purser, R. J. and Parrish, D. F. 2002. Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev. 130, 2905-2916.
    • Xu, Q., Nai, K., Wei, L., Zhang, P., Wang, L. and co-authors. 2005. Progress in Doppler radar data assimilation. 32nd Conference on Radar Meteorology, 24-29 October 2005, Albuquerque, New Mexico, Amer. Meteor. Soc., CD-ROM, JP1J7.
    • Xu, Q., Nai, K. and Wei, L. 2007. An innovation method for estimating radar radial-velocity observation error and background wind error covariances. Q. J. R. Meteor. Soc., in press.
    • Xu, Q., Liu, S. and Xue, M. 2006. Background error covariance functions for vector wind analyses using Doppler radar radial-velocity observations. Q. J. R. Meteorol. Soc. in press.
    • Zhang, P., Liu, S. and Xu, Q. 2005. Quality control of Doppler velocities contaminated by migrating birds. Part I: Feature extraction and quality control parameters. J. Atmos. Oceanic Technol. 22, 1105-1113.
  • No related research data.
  • No similar publications.

Share - Bookmark

Cite this article

Collected from