Some New Information Inequalities Involving f-Divergences

Srivastava, Amit

doi:10.2478/cait-2012-0008

Abstract

New information inequalities involving f-divergences have been established using the convexity arguments and some well known inequalities such as the Jensen inequality and the Arithmetic-Geometric Mean (AGM) inequality. Some particular cases have also been discussed.

References

1. Ali, S. M., S. D. Silvey. A General Class of Coefficients of Divergence of One Distribution from Another. - Jour. Roy. Statist. Soc. Series B, Vol. 28, 1966, 131-142.10.1111/j.2517-6161.1966.tb00626.x
Search in Google Scholar
2. Bassat, M. B. f-Entropies, Probability of Error and Feature Selection. - Inform. Control, Vol. 39, 1978, 227-242.10.1016/S0019-9958(78)90587-9
Search in Google Scholar
3. Chen, H. C. Statistical Pattern Recognition. Hoyderc Book Co., Rocelle Park, New York, 1973.
Search in Google Scholar
4. Chow, C. K., C. N. Lin. Approximating Discrete Probability Distributions with Dependence Trees. - IEEE Trans. Inform. Theory, Vol. 14, 1968, No 3, 462-467.10.1109/TIT.1968.1054142
Search in Google Scholar
5. Csiszár, I. Information - Type Measures of Difference of Probability Distributions and Indirect Observations. - Studia Sci. Math. Hungar., Vol. 2, 1967, 299-318.
Search in Google Scholar
6. Csiszár, I. Information Measures: A Critical Survey. in Trans. - In: Seventh Prague Conf. on Information Theory, Academia, Prague, 1974, 73-86.
Search in Google Scholar
7. Gokhale, D. V., S. Kullback. Information in Contingency Tables. New York, Marcel Dekker, 1978.
Search in Google Scholar
8. Jones, L., C. Byrne. General Entropy Criteria for Inverse Problems, with Applications to Data Compression, Pattern Classification, and Cluster Analysis. - IEEE Trans. Inform. Theory, Vol. 36, 1990, 23-30.10.1109/18.50370
Search in Google Scholar
9. Kadota, T. T., L. A. Shepp. On the Best Finite Set of Linear Observables for Discriminating Two Gaussian Signals. - IEEE Trans. Inform. Theory, Vol. 13, 1967, 288-294.10.1109/TIT.1967.1054013
Search in Google Scholar
10. Kailath, T. The Divergence and Bhattacharyya Distance Measures in Signal Selection. - IEEE Trans. Comm. Technology, Vol. COM-15, 1967, 52-60.10.1109/TCOM.1967.1089532
Search in Google Scholar
11. Kazakos, D., T. Cotsidas. A Decision Theory Approach to the Approximation of Discrete Probability Densities. - IEEE Trans. Perform. Anal. Machine Intell, Vol. 1, 1980, 61-67.10.1109/TPAMI.1980.4766971
Search in Google Scholar
12. Kolmogorov, A. N. A New Invariant for Transitive Dynamical System. Dokl. Akad. Nauk. USSR, Vol. 119, 1957, 861-869.
Search in Google Scholar
13. Kullback, S., A. S. Leibler. On Information and Sufficiency. - Ann. Math. Stat., Vol. 22, 1951, 79-86.10.1214/aoms/1177729694
Search in Google Scholar
14. Österreicher, F. Csiszar’s f - Divergences - Basic Properties. Res. Report Collection, 2002. http://rgmia.vu.edu.au/monographs/csiszar.htm
Search in Google Scholar
15. Pearson, K. On the Criterion That a Given System of Deviations from the Probable in the Case of Correlated System of Variables in Such That It Can Be Reasonable Supposed to Have Arisen from Random Sampling. - Phil. Mag., 1900, 157-172.10.1080/14786440009463897
Search in Google Scholar
16. Pielou, E. C. Ecological Diversity. New York, Wiley, 1975.
Search in Google Scholar
17. Sibson, R. Information Radius. - Z. Wahrs. und Verw Geb., Vol. 14, 1969, 149-160.10.1007/BF00537520
Search in Google Scholar
18. Theil, H. Economics and Information Theory. Amsterdam, North-Holland, 1967.
Search in Google Scholar
19. Theil, H. Statistical Decomposition Analysis. Amsterdam, North-Holland, 1972.
Search in Google Scholar

Some New Information Inequalities Involving f-Divergences

Abstract

Paradigm

My account