On a New Csiszar’s f-Divergence Measure

P. K. Bhatia; Surender Singh

doi:10.2478/cait-2013-0013

.blurhash-client-img { display: none !important; }

On a New Csiszar’s f-Divergence Measure

Cybernetics and Information Technologies

Volume 13 (2013): Issue 2 (June 2013)

By: P. K. Bhatia and Surender Singh

Open Access

|May 2013

Abstract

A new parametric probabilistic measure of information and a corresponding symmetric divergence (distance) measure is proposed. The information measure is useful to study the uncertainty and the corresponding divergence measure is useful for comparing two probability distributions. The proposed parametric divergence measure belongs to the family of Csiszar’s f-divergence measures. The bounds of this divergence measures are obtained in terms of some well known divergence measures. Some properties of the proposed information and divergence measures are studied.

References

1. Basseville, M. Divergence Measures for Statistical Data Processing. Publications Internes de l’IRISA, November 2010.
Search in Google Scholar
2. Bregman, L. M. The Relaxation Method of Finding the Common Point of Convex Sets and Its Application to the Solution of Problems in Convex Programming. - USSR Computational Mathematics and Mathematical Physics, Vol. 7, 1967, No 3, 200-217.10.1016/0041-5553(67)90040-7
Search in Google Scholar
3. Cover, T. M., A. J. Thomas. Elements of Information Theory. Wiley India Pvt. Ltd. New Delhi, 2009.
Search in Google Scholar
4. Csiszar, I. Eine Informations Theoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizitat von marko_schen ketten. Magyar. - Tud. Akad. Mat. Kutato Int. Kozl, Vol. 8, 1963, 85-108.
Search in Google Scholar
5. Csiszar, I. On Topological Properties of f-Divergences. - Studia Math. Hungarica, Vol. 2, 1967, 329-339.
Search in Google Scholar
6. Dragomir, S. S. Some Inequalities for the Csiszar’s ∅-Divergence. Inequalities for the Csizar’s f-Divergence in Information Theory, 2000. http://rgmia.vu.edu.au/monographs/ciszar.htm
Search in Google Scholar
7. Dragomir, S. S. Upper and Lower Bounds for Csiszar’s f-Divergence in Terms of the Kullback-Leibder Distance and Applications. Inequalities for the Csiszar’s f-divergence in Information Theory, 2000. http://rgmia.vu.edu.au/monographs/ciszar.htm
Search in Google Scholar
8. Dragomir, S. S. Upper and Lower Bounds for Csiszar’s f-Divergence in Terms of the Hellinger Discrimination and Applications. Inequalities for the Csiszar’s f-Divergence in Information Theory, 2000. http://rgmia.vu.edu.au/monographs/csiszar.htm
Search in Google Scholar
9. Dragomir, S. S. Other Inequalities for the Csiszar’s Divergence and Applications. Inequalities for the Csizar’s f-Divergence in Information Theory, 2000. http://rgmia.vu.edu.au/monographs/ciszar.htm
Search in Google Scholar
10. Esteban, M. D., D. Morale s. A Summary on Entropy Statistic. - Kybernetika, Vol. 31, 1995, No 4, 337-346.
Search in Google Scholar
11. Hellinger, E. Neue Berunduring der Theorie der Quadratischen Formen von Un endlichen Vieden Veran derliehen. - J. Reine Aug. Math., Vol. 136, 1909, 210-271.10.1515/crll.1909.136.210
Search in Google Scholar
12. Kullback, S., R. A. Leible r. On Information and Sufficiency. - Ann. Math. Statist., Vol. 22, 1951, 79-86.10.1214/aoms/1177729694
Search in Google Scholar
13.Maji, P. f-Information Measures for Efficient Selection of Discriminative Genes From Microarray Data. - IEEE Transactions on Biomedical Engineering, Vol. 56, 2009, No 4, 1063-1069.10.1109/TBME.2008.200450219272938
Search in Google Scholar
14. Pearson, K. On the Criterion that a Given System of Deviations From the Probable in the Case of Correlated System of Variables is Such that it Can Be Reasonable Supposed to Have Arisen from Random Sampling. - Phil. Mag., Vol. 50, 1900, 157-172.10.1080/14786440009463897
Search in Google Scholar
15. Renyi, A. On Measures of Entropy and Information. - In: Proc. of 4th Berkeley Symp. Math. Stat. Probab., Vol. 1, 1961, 547-561.
Search in Google Scholar
16. Salicru, M. Measures of Information Associated with Csiszar’s Divergences. - Kybernetika, Vol. 50, 1994, No 5, 563-573.
Search in Google Scholar
17. Shannon, C. E. The Mathematical Theory of Communications. - Bell Syst. Tech. Journal, Vol. 27, 1948, 423-467.10.1002/j.1538-7305.1948.tb01338.x
Search in Google Scholar
18. Taneja, I. J. Generalized Information Measures and their Applications. - On-Line Book, 2001. http://www.mtm.ufsc.br/∼taneja/book/book.html
Search in Google Scholar
19. Taneja, I. J. Generalized Arithmetic and Geometric Mean Divergence Measure and Their Statistical Aspects. Available at: arXiv:math/0501297v1[math.ST] 19 Jan 2005.
Search in Google Scholar
20. Taneja, I. J. On Mean Divergence Measure. Available at: arXiv:math/0501298v2[math.ST] 13 June 2005.
Search in Google Scholar
21.Wang,Y. Generelized Information Theory: A Review and Outlook. - Information Technology Journal, Vol. 10, 2011, No 3, 461-469.10.3923/itj.2011.461.469
Search in Google Scholar

Articles in this issue

DOI: https://doi.org/10.2478/cait-2013-0013 | Journal eISSN: 1314-4081 | Journal ISSN: 1311-9702

Journal RSS Feed

Language: English

Page range: 43 - 57

Published on: May 21, 2013

Published by: Bulgarian Academy of Sciences, Institute of Information and Communication Technologies

In partnership with: Paradigm Publishing Services

Publication frequency: 4 issues per year

Keywords:

Csiszar’s f-Divergence,

distance measure information,

information bounds

Related subjects:

Computer sciences,

Information technology

© 2013 P. K. Bhatia, Surender Singh, published by Bulgarian Academy of Sciences, Institute of Information and Communication Technologies
This work is licensed under the Creative Commons License.

Volume 13 (2013): Issue 2 (June 2013)