Uncertain research country rankings. Should we continue producing uncertain rankings?
References
- Abramo, G., Cicero, T., & D’Angelo, C. A. (2013). National peer-review research assessment execises for the hard sciences can be a complete waste of money: the Italian case. Scientometrics, 95, 311-324.
- Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019). Citations, citation indicators, and research quality: An overview of basic concepts and theories. SAGE Open, 9(1), 1-17.
- Bihari, A., Tripathi, S., & Deepak, A. (2021). A review on h-index and its alternative indices. Journal of Information Science, 49(3), 624-665.
- Bornmann, L., & Daniel, H.-D. (2007). What do we know about the h index. Journal of the American Society for information Science and Technology, 58, 1381-1385.
- Bornmann, L., Leydesdorff, L., & Mutz, R. (2013). The use of percentile and percentile rank classes in the analysis of bibliometric data: opportunities and limits. Journal of Informetrics, 7(1), 158-165.
- Bornmann, L., Ye, A., & Ye, F. (2018). Identifying landmark publications in the long run using field-normalized citation data. Journal of Documentation, 74(2), 278-288.
- Brito, R., & Rodríguez-Navarro, A. (2021). The incosistency of h-index: A mathematical analysis. Journal of Informetrics, 15(1), Article 101106.
- Conover, W. J., & Iman, R. (1981). Rank transformations as a bridge between parametric and nonparametric statitics. The American Statistician, 35(3), 124-129.
- Docampo, D., & Besoule, J.-J. (2019). A new appraoch to the analysis and evaluation of the research output of countries and institutions. Scientometrics, 119(2), 1207-1225.
- Drucker, P. F. (1954). The Practice of Management. Harper and Row Publisher, New York.
- European Commission. (2018). Science, Research and Innovation Performance of the EU 2018. Strengthening the foundations for Europe’s future. Publications Office of the European Union.
- European Commission. (2020). Science, Research and Innovation Performance of the EU 2020. A fair, green and digital Europe. Publication Office of the European Union.
- European Commission. (2022). Science, Reserach and Innovation Performance of the EU. Building a sustainable future in uncertain times. Publication Office of the European Union.
- European Commission. (2024). Science, Reserach and Innovation Performance of the EU. A competitive Europe for a sustainable future. Publication Office of the European Union.
- Gaida, J., Wong-Leung, J., Robin, S., & Cave, D. (2023). ASPI’s Critical Technology Tracker. The global race for future power. Policy Brief Report No.69/2023
- Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of ideas. Science, 122(3159), 108-111.
- Garfield, E. (1973). Citation frequency as a measure of research activity and performance. Essays of an Information Scientist, 1(2), 406-408.
- Garfield, E. (1999). Journal Impact Factor: a brief review. Canadian Medical Association journal, 161(8), 979-980.
- Garfield, E., & Welljams-Dorof, A. (1992). Citation data: their use as quantitative indicators for science and technology evaluations and policy-making. Science and Public Policy, 19(5), 321-327.
- Godin, B. (2003). The emergence of S&T indicators: why did governments supplement statistics with indicators? Research Policy, 32(4), 679-691.
- Godin, B. (2004). The new economy: what the concept owes to the OECD. Research Policy, 33(5), 679-690.
- Godin, B. (2006). On the origins of bibliometrics. Scientometrics, 68(1), 109-133.
- Golosovsky, M. (2021). Universality of citation distributions: A new understanding. Quantitative Science Studies, 2(2), 527-543.
- Hcéres. (2019). OST, Dynamics of scietific production in the world, in Europe and France, 2000-2016. Paris.
- Hirsch, J. E. (2005). An index to quantify an individual’s scientific reserach output. Proceedins of the National Academy of Sciences USA, 102(46), 16569-16572.
- Hu, X., & Rousseau, R. (2019). Do citation chimeras exist? The case of under-cited influential articles suffering delayed recognition. Journal of the Association for Information Science and Technology, 70(5), 499-508.
- Kuhn, T. (1970). The structure of scientific revolutions. University of Chicago Press.
- Leung, J. W., Robin, S., & Cave, D. (2024). Techology Tracker: The rewards of long-term reserach investments. The Australian Strategic Institute.
- Mcalister, P. R., Narin, F., & Corrigan, J. G. (1983). Programmatic evaluation and comparison based on standardized citatio scores. IEEE Transactions on Engineering Managament, EM-30, 205-2011.
- Merton, R. K. (1979). Foreword. In E. Garfield (Ed), Citation Indexing -Its Theory and Application in Science, Technology, and Humanities. John Wiley & Sons.
- Min, C., Bu, Y., Wu, D., Ding, Y., & Zhang, Y. (2021). Identifying citation patterns of scientific breakthroughs: A perspective o dynamic citation process. Information Processing & Management, 58(1), 102428.
- Mingers, J., & Leydesdorff, L. (2015). A review of theory and practice in scientometrics. European Journal of Operational Research, 246(1), 1-19.
- Narin, F. (1976). Evaluative bibliometrics: The Use of Publication and Citation Analysis in the Evaluation of Scientific Activity. Computer Horizon Inc.
- National Institute of Science and Technology, T. (2022). “Japanese Science and Technology Indicators 2022”, NISTEP RESEARCH MATERIAL No. 318.
- National Science Board. (2018). Science and Engineering Indicators 2018. National Science Foundation.
- National Science Board. (2020). Science and Engineering Indicators 2020: The State of U.S. Science and Engineering. NSB-2020-1. Alexandria, VA.
- National Science Board. (2022). Science and Engineering Indicators 2022: The State of U.S. Science and Engineering. NSB-2022-1.
- National Science Board. (2024). Science and Engineering Indicators 2024: The State of U.S. Science and Engineering. NSB-2024-3.
- Olensky, M., Schmidt, M., & van Eck, N. J. (2015). Evaluation of the citation matching algorithms of CWTS and iFQ in comparison to the Web of Science. Journal of the Association for Information Science and Technology, 67(10), 2550-2564.
- Pendlebury, D. A. (2020). When the data don’t mean what they say: Japan’s comparative underperformance in citation impact. In C. Daraio & W. Glanzel (Eds.), Evaluative Informetrics: The Art of Metrics-based Research Assessment. Springer.
- Perianes-Rodriguez, A., & Ruiz-Castillo, J. (2016). University citation distributions. Journal of the Association for Information Science and Technology, 67(11), 2790-2804.
- Poege, F., Harhoff, D., Gaessler, F., & Baruffaldi, S. (2019). Science quality and the value of inventions. Science Advances, 5(12), eaay7323.
- Prathap, G. (2010). An iCE map approach to evaluate performance and efficiency ofscientific production of countries. Scientometrics, 85(1), 185-191.
- Price, D. J. D. S. (1965). Networks of scientific papers: The pattern of bibliographic references indicates the nature of the scientific research front. Science, 149(3683), 510-515.
- Radicchi, F., Fortunato, S., & Castellano, C. (2008). Universality of citation distributions: toward an objective measure of scientific impact. Proceedins of the National Academy of Sciences USA, 105(45), 17268-17272.
- Rodríguez-Navarro, A. (2024). Citation distributions and reserach evaluations: The impossibility of formulating a universal indicator. Journal of Data and Information Science, 9(4), 24-48.
- Rodríguez-Navarro, A., & Brito, R. (2018a). Double rank analysis for research assessment. Journal of Informetrics, 12(1), 31-41.
- Rodríguez-Navarro, A., & Brito, R. (2018b). Technological research in the EU is less efficient than the world average. EU research policy risks Europeans’ future. Journal of Informetrics, 12(3), 718-731.
- Rodríguez-Navarro, A., & Brito, R. (2019). Probability and expected frequency of breakthroughs – basis and use of a robust method of research assessment. Scientometrics, 119(1), 213-235.
- Rodríguez-Navarro, A., & Brito, R. (2020). Like-for-like bibliometric substitutes for peer review: advantages and limits of indicators calculated from the ep index. Research Evaluation, 29(2), 215-230.
- Rodríguez-Navarro, A., & Brito, R. (2021b). Total number of papers and in a single percentile fully describes research impact – Revisiting concepts and applications. Quantitative Science Studies, 2(2), 544-559.
- Rodríguez-Navarro, A., & Brito, R. (2022). The link between countries’ economic and scientific wealth has a complex dependence on technological activity and research policy. Scientometrics, 127(5), 2871-2896.
- Rodríguez-Navarro, A., & Brito, R. (2024a). The extreme upper tail of Japan’s citation distribution reveals its research success. Quality & Quantity, 58(4), 3831-3844.
- Rodríguez-Navarro, A., & Brito, R. (2024b). Rank analysis of most cited publications, a new approach for research assessments. Journal of Informetrics, 18(2), 101503.
- Schlagberger, E. M., Bornmann, L., & Bauer, J. (2016). At what institutions did Nobel laureates do their prizewinning work? An analysis of bibliographical information on Nobel laureates from 1994 to 2014. Scientometrics, 109, 723-767.
- Schneider, J. W., & Costas, R. (2017). Identifying potential “breakthrough” publications using refined citation analyses: Three related explorative approaches. Journal of the Association for Information Science and Technology, 68(3), 709-723.
- Schubert, A., & Braun, T. (1996). Cross-field normalization of scientometric indicators. Scientometrics, 36(3), 311-324.
- Shahmandi, M., Wilson, P., & Thelwall, M. (2020). A new algorithm for zero-modified models applied to citation counts. Scientometrics, 125, 993-1010.
- Thelwall, M. (2016). Are there too many articles? Zero inflated variants of the discretised lognormal and hooked power law. Journal of Informetrics, 10(2), 622-633.
- Thelwall, M., Kousha, K., Stuart, E., Makita, M., Abdoli, M., Wilson, P., & Levitt, J. (2023). In which fields are citations indicators of reserach quality? Journal of the Association for Information Science and Technology, 74(8), 941-953.
- Traag, V. A., & Waltman, L. (2019). Systematic analysis of agreement between metrics and peer review in the UK REF. Palgrave Communications, 5(1), 29. https://doi.org/10.1057/s41599-019-0233-x
- Viiu, G.-A. (2018). The lognormal distribution explains the remarkable pattern documented by characteristic scores and scales in scientometrics. Journal of Informetrics, 12(2), 401-415.
- Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365-391.
- Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E. C. M., Tijssen, R. J. W., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. (2012). The Leiden ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for information Science and Technology, 63(12), 2419-2432.
- Waltman, L., & Schreiber, M. (2013). On the calculation of percentile-based bibliometric indicators. Journal of the American Society for information Science and Technology, 64, 372-379.
- Waltman, L., & van Eck, N. J. (2012). The inconsistency of the h-index. Journal of the American Society for information Science and Technology, 63(2), 406-415.
- Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2011). Towards a new crown indicator: Some theoretical considerations. Journal of Informetrics, 5(1), 37-47.
- Waltman, L., van Eck, N. J., & van Raan, A. F. J. (2012). Universality of citation distributions revisited. Journal of the American Society for information Science and Technology, 63(1), 72-77.
- Wildgaard, L., Schneider, J. W., & Larsen, B. (2014). A rewiew of the characteristics of 108 author-level bibliometric indicators. Scientometrics, 101, 125-158.
- Wuestman, M., Hoekman, J., & Frenken, K. (2020). A topology of scientific breakthroughs. Quantitative Science Studies, 1(3), 1203-1222.
Language: English
Page range: 161 - 182
Submitted on: Mar 21, 2025
Accepted on: Apr 24, 2025
Published on: May 13, 2025
Published by: Chinese Academy of Sciences, National Science Library
In partnership with: Paradigm Publishing Services
Publication frequency: 4 issues per year
Related subjects:
© 2025 Alonso Rodríguez-Navarro, published by Chinese Academy of Sciences, National Science Library
This work is licensed under the Creative Commons Attribution 4.0 License.