Have a personal or library account? Click to login
Is more always better? Measuring the quality of ranking data through information entropy Cover

Is more always better? Measuring the quality of ranking data through information entropy

By: Yishan Liu,  Yu Xiao,  Xin Long and  Jun Wu  
Open Access
|Nov 2025

References

  1. Adade, S. Y.-S. S., Lin, H., Johnson, N. A. N., Nunekpeku, X., Aheto, J. H., Ekumah, J.-N., Kwadzokpui, B. A., Teye, E., Ahmad, W., & Chen, Q. (2024). Advanced food contaminant detection through multi-source data fusion: Strategies, applications, and future perspectives. Trends in Food Science & Technology, 104851.
  2. Aledo, J. A., Gámez, J. A., & Molina, D. (2016). Using extension sets to aggregate partial rankings in a flexible setting. Applied Mathematics and Computation, 290, 208–223. https://doi.org/10.1016/j.amc.2016.06.005
  3. Ali, A., & Meilă, M. (2012). Experiments with kemeny ranking: What works when? Mathematical Social Sciences, 64(1), 28–40.
  4. Ali, I., Cook, W. D., & Kress, M. (1986). On the minimum violations ranking of a tournament. Management Science, 32(6), 660–672.
  5. Almog, A., & Shmueli, E. (2019). Structural Entropy: Monitoring Correlation-Based Networks Over Time With Application To Financial Markets. Scientific Reports, 9(1), 10832. https://doi.org/10.1038/s41598-019- 47210-8
  6. Argentini, A., & Blanzieri, E. (2012). Ranking Aggregation Based on Belief Function. In S. Greco, B. Bouchon-Meunier, G. Coletti, M. Fedrizzi, B. Matarazzo, & R. R. Yager (Eds.), Advances in Computational Intelligence (Vol. 299, pp. 511–520). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-642-31718-7_53
  7. Caplin, A., & Nalebuff, B. (1991). Aggregation and Social Choice: A Mean Voter Theorem. Econometrica, 59(1), 1. https://doi.org/10.2307/2938238
  8. Chartier, T. P., Kreutzer, E., Langville, A. N., Pedings, K., & Yamamoto, Y. (2010). Minimum violations sports ranking using evolutionary optimization and binary integer linear program approaches. In A. Bedford & M. Ovens (Eds.), Proceedings of the Tenth Australian Conference on Mathematics and Computers in Sport (pp. 13–20).
  9. Chen, K., Bui, T., Chen, F., Wang, Z., & Nevatia, R. (2017). AMC: Attention guided Multi-modal Correlation Learning for Image Search (Version 1). arXiv. https://doi.org/10.48550/ARXIV.1704.00763
  10. Chen, X., Li, W., Lu, S., Zhou, Z., & Fu, X. (2018). Efficient resource allocation for on-demand mobile-edge cloud computing. IEEE Transactions on Vehicular Technology, 67(9), 8769–8780.
  11. de Borda, J. (1781). M’emoire sur les’ elections au scrutin. Histoire de l’Acad’emie Royale Des Sciences, 102, 657-665.
  12. de Caritat Mis, J. A. N. (1785). Essai sur I’application de I’analyse à la probabilité des décisions rendues à la pluralité des voix. Imprimerie royale.
  13. Dehmer, M., & Mowshowitz, A. (2011). A history of graph entropy measures. Information Sciences, 181(1), 57-78. https://doi.org/10.1016/j.ins.2010.08.041
  14. Deng, K., Han, S., Li, K. J., & Liu, J. S. (2014). Bayesian aggregation of order-based rank data. Journal of the American Statistical Association, 109(507), 1023–1039.
  15. Dourado, I. C., Pedronette, D. C. G., & Torres, R. da S. (2019). Unsupervised graph-based rank aggregation for improved retrieval. Information Processing & Management, 56(4), 1260–1279. https://doi.org/10.1016/j. ipm.2019.03.008
  16. Dwork, C., Kumar, R., Naor, M., & Sivakumar, D. (2001). Rank aggregation methods for the Web. In Proceedings of the 10th International Conference on World Wide Web (pp.613–622). Association for Computing Machinery. https://doi.org/10.1145/371920.372165
  17. Faghri, A., & Bergman, T. L. (2024). Ranking academic institutions based on the productivity, impact, and quality of institutional scholars. Journal of Data and Information Science, 9(3), 116–154. https://doi.org/10.2478/jdis-2024-0017
  18. Frank Hsu, D., & Taksa, I. (2005). Comparing rank and score combination methods for data fusion in information retrieval. Information Retrieval, 8(3), 449–480.
  19. Guo, X., Chen, Y., Du, J., & Dong, E. (2022). Extracting and Measuring Uncertain Biomedical Knowledge from Scientific Statements. Journal of Data and Information Science, 7(2), 6–30. https://doi.org/10.2478/jdis-2022-0008
  20. Hartley, R. V. (1928). Transmission of information 1. Bell System Technical Journal, 7(3), 535–563.
  21. Hong, Y., Zhang, M., & Meeker, W. Q. (2018). Big data and reliability applications: The complexity dimension. Journal of Quality Technology, 50(2), 135–149.
  22. Kayed, A., El-Qawasmeh, E., & Qawaqneh, Z. (2010). Ranking web sites using domain ontology concepts. Information & Management, 47(7–8), 350–355.
  23. Kendall, M. G. (1938). A new measure of rank correlation. Biometrika, 30(1–2), 81–93.
  24. Kendall, M. G. (1945). The treatment of ties in ranking problems. Biometrika, 33(3), 239–251.
  25. Keyhanipour, A. H. (2025). Graph-induced rank-aggregation using information fusion operators. The Journal of Supercomputing, 81(1), 43. https://doi.org/10.1007/s11227-024-06595-8
  26. Li, G., Gao, W., & Gao, W. (2024). Entropy coding. In Point Cloud Compression: Technologies and Standardization (pp. 113–133). Springer.
  27. Li, X., Wang, X., & Xiao, G. (2019). A comparative study of rank aggregation methods for partial and top ranked lists in genomic applications. Briefings in Bioinformatics, 20(1), 178–189. https://doi.org/10.1093/bib/bbx101
  28. Lillis, D. (2020). On the evaluation of data fusion for information retrieval. In Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation (pp. 54–57).
  29. Lin, S. (2010). Rank aggregation methods. Wiley Interdisciplinary Reviews: Computational Statistics, 2(5), 555–570.
  30. Liu, X., Wang, X., Lyu, L., & Wang, Y. (2022). Identifying disruptive technologies by integrating multi-source data. Scientometrics, 127(9), 5325–5351.
  31. Luce, R. D. (1959). Individual choice behavior (Vol. 4). Wiley New York.
  32. Lynch, C., Aryafar, K., & Attenberg, J. (2016). Images Don’t Lie: Transferring Deep Visual Semantic Features to Large-Scale Multimodal Learning to Rank. In Proceedings of the 22nd ACM SIGKDD International Conference on KnowledgeDiscovery andDataMining (pp. 541–548). https://doi.org/10.1145/2939672.2939728
  33. Maanijou, R., & Mirroshandel, S. A. (2019). Introducing an expert system for prediction of soccer player ranking using ensemble learning. Neural Computing and Applications, 31(12), 9157–9174.
  34. Mallows, C. L. (1957). Non-null ranking models. I. Biometrika, 44(1/2), 114–130.
  35. Neveling, M., & Rothe, J. (2021). Control complexity in Borda elections: Solving all open cases of offline control and some cases of online control. Artificial Intelligence, 298, 103508. https://doi.org/10.1016/j. artint.2021.103508
  36. Omar, Y. M., & Plapper, P. (2020). A Survey of Information Entropy Metrics for Complex Networks. Entropy, 22(12), 1417. https://doi.org/10.3390/e22121417
  37. Ossadnik, W., Schinke, S., & Kaspar, R. H. (2016). Group aggregation techniques for analytic hierarchy process and analytic network process: A comparative analysis. Group Decision and Negotiation, 25(2), 421–457.
  38. Pedings, K. E., Langville, A. N., & Yamamoto, Y. (2012). A minimum violations ranking method. Optimization and Engineering, 13(2), 349–370. https://doi.org/10.1007/s11081-011-9135-5
  39. Plackett, R. L. (1975). The analysis of permutations. Journal of the Royal Statistical Society Series C: Applied Statistics, 24(2), 193–202.
  40. Pujahari, A., & Sisodia, D. S. (2020). Aggregation of preference relations to enhance the ranking quality of collaborative filtering based group recommender system. Expert Systems with Applications, 156, 113476. https://doi.org/10.1016/j.eswa.2020.113476
  41. Qin, T., Geng, X., & Liu, T.-Y. (2010). A new probabilistic model for rank aggregation. In Proceedings of the 24th International Conference on Neural Information Processing Systems (pp. 1948–1956). Curran Associates Inc.
  42. Reilly, B. (2002). Social Choice in the South Seas: Electoral Innovation and the Borda Count in the Pacific Island Countries.InternationalPoliticalScienceReview,23(4), 355–372.https://doi.org/10.1177/0192512102023004002
  43. Rong, C., Wen, Z. H. U., & Jiqing, S. U. N. (2019). Multi-source data fusion method and its empirical study for journal evaluation. Chinese Journal of Scientific and Technical Periodicals, 30(6), 685.
  44. Schalekamp, F., & Zuylen, A. van. (2009). Rank aggregation: Together we’re strong. In 2009 Proceedings of the Eleventh Workshop on Algorithm Engineering and Experiments (ALENEX) (pp. 38–51).
  45. Shannon, C. E. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27(3), 379–423.
  46. Soneson, C., & Robinson, M. D. (2018). Bias, robustness and scalability in single-cell differential expression analysis. Nature Methods, 15(4), 255–261.
  47. Spearman, C. (1987). The proof and measurement of association between two things. The American Journal of Psychology, 100(3/4), 441–471.
  48. Tadbier, A. W., & Shoufan, A. (2021). Ranking educational channels on YouTube: Aspects and issues. Education and Information Technologies, 26(3), 3077–3096. https://doi.org/10.1007/s10639-020-10414-x
  49. Tian, Y., Zhang, Z., Xiong, J., Chen, L., Ma, J., & Peng, C. (2022). Achieving Graph Clustering Privacy Preservation Based on Structure Entropy in Social IoT. IEEE Internet of Things Journal, 9(4), 2761–2777. https://doi. org/10.1109/JIOT.2021.3092185
  50. Ursu, R. M. (2015). The power of rankings: Quantifying the effects of rankings on online consumer search and choice. SSRN Electronic Journal. https://dx.doi.org/10.2139/ssrn.2729325
  51. Wang, J.-N., Du, J., & Chiu, Y.-L. (2020). Can online user reviews be more helpful? Evaluating and improving ranking approaches. Information & Management, 57(8), 103281.
  52. Wang, X., Wang, Y., Yang, J., Jia, X., Li, L., Ding, W., & Wang, F.-Y. (2024). The survey on multi-source data fusion in cyber-physical-social systems: Foundational infrastructure for industrial metaverses and industries 5.0. Information Fusion, 107, 102321.
  53. Wimmer, L., Sale, Y., Hofman, P., Bischl, B., & Hüllermeier, E. (2023). Quantifying aleatoric and epistemic uncertainty in machine learning: Are conditional entropy and mutual information appropriate measures? Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence (UAI 2023), Proceedings of Machine Learning Research, 216, 2282–2292. Retrieved from https://proceedings.mlr.press/v216/wimmer23a.html
  54. Xiao, Y., Deng, H.-Z., Lu, X., & Wu, J. (2021). Graph-based rank aggregation method for high-dimensional and partial rankings. Journal of the Operational Research Society, 72(1), 227–236. https://doi.org/10.1080/0160 5682.2019.1657365
  55. Xiao, Y., Deng, Y., Wu, J., Deng, H.-Z., & Lu, X. (2017). Comparison of rank aggregation methods based on inherent ability. Naval Research Logistics, 64(7), 556–565.
  56. Xu, H., Luo, R., Winnink, J., Wang, C., & Elahi, E. (2022). A methodology for identifying breakthrough topics using structural entropy. Information Processing & Management, 59(2), 102862. https://doi.org/10.1016/j.ipm.2021.102862
  57. Xu, H.-Y., Yue, Z.-H., Wang, C., Dong, K., Pang, H.-S., & Han, Z. (2017). Multi-source data fusion study in scientometrics. Scientometrics, 111(2), 773–792.
  58. Zhao, Y., Shi, Z., Zhang, J., Chen, D., & Gu, L. (2019). A novel active learning framework for classification: Using weighted rank aggregation to achieve multiple query criteria. Pattern Recognition, 93, 581–602.
DOI: https://doi.org/10.2478/jdis-2025-0055 | Journal eISSN: 2543-683X | Journal ISSN: 2096-157X
Language: English
Page range: 105 - 131
Submitted on: Jul 1, 2025
|
Accepted on: Oct 13, 2025
|
Published on: Nov 7, 2025
In partnership with: Paradigm Publishing Services

© 2025 Yishan Liu, Yu Xiao, Xin Long, Jun Wu, published by Chinese Academy of Sciences, National Science Library
This work is licensed under the Creative Commons Attribution 4.0 License.