Exploring the effects of journal article features: Implications for automated prediction of scholarly impact
References
- Abramo, G. (2018). Revisiting the scientometric conceptualization of impact and its measurement. Journal of Informetrics, 12(3), 590-597.
- Abramo, G. (2024). The forced battle between peer-review and scientometric research assessment: Why the CoARA initiative is unsound. Research Evaluation, rvae021, DOI: 10.1093/reseval/rvae021.
- Abramo, G., Cicero, T., & D’Angelo, C.A. (2011). Assessing the varying level of impact measurement accuracy as a function of the citation window length. Journal of Informetrics, 5(4), 659-667.
- Abramo, G., & D’Angelo, C.A. (2015). The relationship between the number of authors of a publication, its citations and the impact factor of the publishing journal: Evidence from Italy. Journal of Informetrics, 9(4), 746-761.
- Abramo, G., D’Angelo, C.A., & Di Costa, F. (2016). The effect of a country’s name in the title of a publication on its visibility and citability. Scientometrics, 109(3), 1895-1909.
- Abramo, G., D’Angelo, C.A., & Di Costa, F. (2017a). Do interdisciplinary research teams deliver higher gains to science? Scientometrics, 111(1), 317-336.
- Abramo, G., D’Angelo, C.A., & Di Costa, F. (2017b). Specialization versus diversification in research activities: the extent, intensity and relatedness of field diversification by individual scientists. Scientometrics, 112(3), 1403-1418.
- Abramo, G., D’Angelo, C.A., & Felici, G. (2019). Predicting long-term publication impact through a combination of early citations and journal impact factor. Journal of Informetrics, 13(1), 32-49.
- Abramo, G., D’Angelo, C.A., & Murgia, G. (2013). Gender differences in research collaboration. Journal of Informetrics, 7(4), 811-822. DOI: 10.1016/j.joi.2013.07.002
- Abramo, G., D’Angelo, C.A., & Reale, E. (2019). Peer review vs bibliometrics: Which method better predicts the scholarly impact of publications? Scientometrics, 121(1), 537-554.
- Aczel, B., Szaszi, B., & Holcombe, A.O. (2021). A billion-dollar donation: Estimating the cost of researchers’ time spent on peer review’. Research Integrity and Peer Review, 6, 1-8.
- Adler, R., Ewing, J., & Taylor, P. (2008). Citation statistics. International Mathematical Union, in cooperation with the International Council of Industrial and Applied Mathematics and the Institute of Mathematical Statistics. https://www.mathunion.org/fileadmin/IMU/Report/CitationStatistics.pdf
- Aksnes, D.W., & Taxt, R.E. (2004). Peer reviews and bibliometric indicators: A comparative study at Norwegian University. Research Evaluation, 13 (1), 33-41.
- Alimohammadi, D., & Sajjadi, M. (2009). Correlation between references and citations. Webology, 6(2), a71.
- Allen, L., Jones, C., Dolby, K., Lynn, D., & Walport, M. (2009). Looking for landmarks: The role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLoS ONE, 4(6).
- Alohali, Y.A., Fayed, M.S, Mesallam, T., Abdelsamad, Y., Almuhawas, F., & Hagr, A. (2022). A machine learning model to predict citation counts of scientific papers in otology field. BioMed Research International. DOI: 10.1155/2022/2239152
- Ante, L. (2022). The relationship between readability and scientific impact: Evidence from emerging technology discourses. Journal of Informetrics, 16(1), 101252. DOI: 10.1016/j.joi.2022.101252
- Antelman, K. (2004). Do open-access articles have a greater research impact?. College & Research Libraries, 65(5), 372-382.
- Antoniou, G.A., Antoniou, S.A., Georgakarakos, E.I., Sfyroeras, G.S., & Georgiadis, G.S. (2015). Bibliometric analysis of factors predicting increased citations in the vascular and endovascular literature. Annals of Vascular Surgery, 29(2), 286-92.
- Archambault, É., Vignola-Gagné, É., Côté, G., Larivière, V., & Gingras, Y. (2006). Benchmarking scientific output in the social sciences and humanities: The limits of existing databases. Scientometrics, 68(3), 329-342.
- Baker, M. (2016). Stat-checking software stirs up psychology. Nature, 540(7631), 151-152.
- Ball, P. (2008). A longer paper gathers more citations. Nature, 455(7211), 274.
- Beranová, L., Joachimiak, M. P., Kliegr, T., Rabby, G., & Sklenák, V. (2022). Why was this cited? Explainable machine learning applied to COVID-19 research literature. Scientometrics, 127(5), 2313-2349.
- Bertocchi, G., Gambardella, A., Jappelli, T., Nappi, C. A., & Peracchi, F. (2015). Bibliometric evaluation vs informed peer review: Evidence from Italy. Research Policy, 44(2), 451-466.
- Bloor, D. (1976). Knowledge and Social Imagery. London: Routledge, Kegan and Paul.
- Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society of Information Science and Technology, 64(2), 217-233.
- Bornmann, L. (2017). Measuring impact in research evaluations: A thorough discussion of methods for, effects of and problems with impact measurements. Higher Education, 73(5), 775-787.
- Bornmann, L., & Leydesdorff, L. (2013). The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000. Journal of Informetrics, 7(2), 286-291.
- Budtz Pedersen, D., Grønvad, J. F., & Hvidtfeldt, R. (2020). Methods for mapping the impact of social sciences and humanities - A literature review. Research Evaluation, 29, 4-21.
- Calver, M.C., & Bradley, J.S. (2010). Patterns of citations of open access and non-open access conservation biology journal papers and book chapters. Conservation Biology, 24(3), 872-80.
- Caputo, A., Manesh, M.F., Farrukh, M., Farzipoor Saen, R., & Randolph-Seng, B. (2022). Editorial: Over a halfcentury of management decision: a bibliometric overview. Management Decision, 60(8), 2129-2147.
- Cárdenas, J. (2023). Inteligencia artificial, investigación y revisión por pares: escenarios futuros y estrategias de acción [Artificial intelligence, research, and peer review: Future scenarios and action strategies]. Revista Española De Sociología, 32(4), a184. DOI: 10.22325/fes/res.2023.184
- Caron, E., & van Eck, N. J. (2014). Large scale author name disambiguation using rule-based scoring and clustering. In E. Noyons (Ed.), 19th International Conference on Science and Technology Indicators. “Context counts: Pathways to master big data and little data” (pp. 79-86). Leiden: CWTS-Leiden University.
- Chen, S., Arsenault, C., & Larivière, V. (2015). Are top-cited papers more interdisciplinary? Journal of Informetrics, 9(4), 1034-1046.
- Cole, S., Cole, J.R., & Simon, G. A. (1981). Chance and consensus in peer review. Science, 214/4523, 881-886.
- D’Angelo, C.A., & Abramo, G. (2015). Publication rates in 192 research fields. In A. Salah, Y. Tonta, A.A.A. Salah, C. Sugimoto (Eds), Proceedings of the 15th International Society of Scientometrics and Informetrics Conference - (ISSI 2015) (pp. 909-919). Istanbul: Bogazici University Printhouse.
- de Winter, J. (2024). Can ChatGPT be used to predict citation counts, readership, and social media interaction? An exploration among 2222 scientific abstracts. Scientometrics, 129, 2469-2487.
- Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 4171-4186.
- Dickersin, K., Min, Y., & Meinert, C.L. (1992). Factors influencing publication of research results: Follow-up of applications submitted to two institutional review boards. JAMA, 267(3), 374-378.
- Didegah, F., & Thelwall, M. (2013). Which factors help authors produce the highest impact research? Collaboration, journal and document properties. Journal of Informetrics, 7(4), 861-873.
- Elgendi, M. (2019). Characteristics of a highly cited article: A machine learning perspective. IEEE Access, 7, 87977-87986.
- Fox, C. W., Paine, C. T., & Sauterey, B. (2016). Citations increase with manuscript length, author number, and references cited in ecology journals. Ecology and Evolution, 6(21), 7717-7726.
- Fu, L. D., & Aliferis, C. (2008). Models for predicting and explaining citation count of biomedical articles. In AMIA Annual symposium proceedings (Vol. 2008, p. 222). American Medical Informatics Association.
- Gargouri, Y., Hajjem, C., Larivière, V., Gingras, Y., Carr, L., Brody, T., & Harnad, S. (2010). Self-selected or mandated, open access increases citation impact for higher quality research. PloS ONE, 5(10), e13636.
- Glänzel, W., & de Lange, C. (2002). A distributional approach to multinationality measures of international scientific collaboration. Scientometrics, 54, 75-89.
- Glänzel, W., & Moed, H. F. (2002). Journal impact measures in bibliometric research. Scientometrics, 53(2), 171-193
- Grant, J., Brutscher, P. B., Kirk, S. E., Butler, L., & Wooding, S. (2010). Capturing Research Impacts: A Review of International Practice. Documented Briefing. Rand Corporation. www.rand.org/pubs/documented_ briefings/DB578.html
- Hanson, M.A., Gómez Barreiro, P., Crosetto, P., & Brockington, D. (2023). The strain on scientific publishing. arXiv. DOI: 10.48550/arXiv.2309.15884.
- Heßler, N., & Ziegler, A. (2022). Evidence-based recommendations for increasing the citation frequency of original articles. Scientometrics, 127, 3367-3381.
- Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44, 193-215.
- Himani, S., Kumar, M. H., Enduri, M. K., Begum, S. S., Rageswari, G., & Anamalamudi, S. (2022). A comparative study on machine learning based prediction of citations of articles. Proceedings of the 6th International Conference on Trends in Electronics and Informatics (2022), 1819-1824. DOI: 10.1109/ICOEI53556.2022.9777184.
- Hurley, L. A., Ogier, A. L., & Torvik, V. I. (2013). Deconstructing the collaborative impact: Article and author characteristics that influence citation count. Proceedings of the American Society for Information Science and Technology, 50(1), 1-10.
- Jiang, J., He, D., & Ni, C. (2013). The correlations between article citation and references’ impact measures: What can we learn? Proceedings of the American society for information science and technology, 50(1), 1-4. DOI: 10.1002/meet.14505001162
- Kirman, C.R., Simon, T.W., & Hays, S.M. (2019). Science peer review for the 21st century: Assessing scientific consensus for decision-making while managing conflict of interests, reviewer and process bias. Regulatory Toxicology and Pharmacology, 103, 73-85.
- Knorr-Cetina, K. D. (1981). The Manufacture of knowledge: An Essay on the Constructivist and Contextual Nature of Science. Oxford, UK: Pergamon Press.
- Knorr-Cetina, K. D. (1991). Merton sociology of science: the first and the last sociology of science. Contemporary Sociology, 20(4), 522-526.
- Kousha, K., & Thelwall, M. (2024a). Artificial intelligence to support publishing and peer review: A summary and review. Learned Publishing, 37(1), 4-12.
- Kousha, K., & Thelwall, M. (2024b). Factors associating with or predicting more cited or higher quality journal articles: An Annual Review of Information Science and Technology (ARIST) paper. Journal of the Association for Information Science and Technology, 75(3), 15-44.
- Langham-Putrow, A., Bakker, C., & Riegelman, A. (2021). Is the open access citation advantage real? A systematic review of the citation of open access and subscription-based articles. PLoS ONE, 16(6): e0253129. DOI: 10.1371/journal.pone.0253129
- Lansingh, V.C., & Carter, M.J. (2009). Does open access in ophthalmology affect how articles are subsequently cited in research?. Ophthalmology, 116(8), 1425-31.
- Larivière, V., & Gingras, Y. (2010). On the relationship between interdisciplinary and scientific impact. Journal of the American Society for Information Science and Technology, 61(1), 126-131.
- Larivière, V., Vignola-Gagné, E., Villeneuve, C., Gélinas, P., & Gingras, Y. (2011). Sex differences in research funding, productivity and impact: An analysis of Quebec university professors. Scientometrics, 87(3), 483-498. DOI: 10.1007/s11192-011-0369-y
- Latour, B., & Woolgar, S. (1979). Laboratory Life: The Social Construction of Scientific Facts. London:Sage.
- Lee, C.J., Sugimoto, C.R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2-17.
- Levitt, J. M., & Thelwall, M. (2008). Is multidisciplinary research more highly cited? A macro-level study. Journal of the American Society for Information Science and Technology, 59(12), 1973-1984.
- Liang, W., Zhang, Y., Cao, H., Wang, B., Ding, D., Yang, X., & Zou, J. (2023). Can large language models provide useful feedback on research papers? A large-scale empirical analysis. arXiv. https://arxiv.org/abs/2310.01783
- Liu, J., Chen, H., Liu, Z., Bu, Y., & Gu, W. (2022). Non-linearity between referencing behavior and citation impact: A large-scale, discipline-level analysis. Journal of Informetrics, 16(3), 101318.
- Lundberg, S.M., & Lee, S.I. (2017). A unified approach to interpreting model predictions. Advances in Neural Information Processing Systems, 30, 4765-4774.
- Mammola, S., Fontaneto, D., Martínez, A., & Chichorro, F. (2021). Impact of the reference list features on the number of citations. Scientometrics, 126(1), 785-799.
- Mammola, S., Piano, E., Doretto, A., Caprio, E., & Chamberlain, D. (2022). Measuring the influence of non-scientific features on citations. Scientometrics, 127(7), 4123-4137.
- Memon, A. R. (2020). Similarity and plagiarism in scholarly journal submissions: bringing clarity to the concept for authors, reviewers and editors. Journal of Korean medical science, 35(27), e217.
- Merton, R. K. (1973). Priorities in scientific discovery. In R. K. Merton (Ed.), The sociology of science: Theoretical and empirical investigations (pp. 286-324). Chicago: University of Chicago Press.
- Miettinen, R., Tuunainen, J., & Esko, T. (2015). Epistemological, artefactual and interactional-institutional foundations of social impact of academic research. Minerva, 53, 257-77.
- Milat, A.J., Bauman, A.E., & Redman, S. (2015). A narrative review of research impact assessment models and methods. Health Research Policy and Systems, 13, 18.
- Mulkay, M. (1976). Norms and ideology in science. Social Science Information, 15(4-5), 637-656.
- Narin, F., & Whitlow, E.S. (1990). Measurement of scientific cooperation and co-authorship in CEC-related areas of science (Vol. 1). Publications Office of the European Union.
- OECD/Eurostat (2018). Oslo manual 2018: Guidelines for collecting, reporting and using data on innovation (4th ed.). The measurement of scientific, technological and innovation activities. Luxembourg: OECD Publishing. DOI: 10.1787/9789264304604-en
- Özkent, Y. (2022). Social media usage to share information in communication journals: An analysis of social media activity and article citations. PLoS ONE, 17(2), e0263725. DOI: 10.1371/journal.pone.0263725.
- Penfield, T., Baker, M. J., Scoble, R., & Wykes, M. C. (2014). Assessment, evaluations, and definitions of research impact: A review. Research Evaluation, 23(1), 21-32.
- Rabe-Hesketh, S., & Skrondal, A. (2022). Multilevel and longitudinal modeling using stata (4th ed.). College Station, TX: Stata Press.
- Reale, E., Barbara, A., & Costantini, A. (2007). Peer review for the evaluation of academic research: Lessons from the Italian experience. Research Evaluation, 16(3), 216-228.
- Rhoten, D., & Pfirman, S. (2007). Women in interdisciplinary science: Exploring preferences and consequences. Research Policy, 36(1), 56-75. DOI: 10.1016/j.respol.2006.08.001
- Ribeiro, M. T., Singh, S., & Guestrin, C. (2016). “Why should I trust you?”: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1135-1144.
- Rinia, E.J., van Leeuwen, Th.N., van Vuren, H.G., & van Raan, A.F.J. (1998). Comparative analysis of a set of bibliometric indicators and central peer-review criteria, evaluation of condensed matter physics in the Netherlands. Research Policy, 27(1), 95-107.
- Rosenkrantz, A. B., Doshi, A. M., Ginocchio, L. A., & Aphinyanaphongs, Y. (2016). Use of a machine-learning method for predicting highly cited articles within general radiology journals. Academic Radiology, 23(12), 1573-1581.
- Rossi, M. J., & Brand, J. C. (2020). Journal article titles impact their citation rates. Arthroscopy, 36, 2025-2029.
- Ruan, X., Zhu, Y., Li, J., & Cheng, Y. (2020). Predicting the citation counts of individual papers via a BP neural network. Journal of Informetrics, 14(3), 101039.
- Sanfilippo, P., Hewitt, A. W., & Mackey, D. A. (2018). Plurality in multidisciplinary research: multiple institutional affiliations are associated with increased citations. PeerJ, 6, e5664. DOI: 10.7717/peerj.5664
- Schroter, S., Weber, W. E. J., Loder, E., Wilkinson, J., & Kirkham, J. J. (2022). Evaluation of editors’ abilities to predict the citation potential of research manuscripts submitted to the BMJ: A cohort study. British Medical Journal, 379. DOI: 10.1136/bmj-2022-073880.
- Schulz, R., Barnett, A., Bernard, R., Brown, N. J., Byrne, J. A., Eckmann, P., & Weissgerber, T. L. (2022). Is the future of peer review automated?. BMC Research Notes, 15(1), 203. DOI: 10.1186/s13104-022-06080-6
- Sivadas, E., & Johnson, M.S. (2015). Relationships between article references and subsequent citations of marketing journal articles. In Revolution in marketing: Market driving changes: Proceedings of the 2006 Academy of Marketing Science (AMS) Annual Conference (pp. 199-205). Cham: Springer International Publishing.
- Sivertsen, G. (2017). Unique, but still best practice? The Research Excellence Framework (REF) from an international perspective. Palgrave Communications, 3(1), 1-6.
- Smit, J. P., & Hessels, L. K. (2021). The production of scientific and societal value in research evaluation: A review of societal impact assessment methods. Research Evaluation, 30(3), 323-335.
- StataCorp. (2021). Stata: Release 17 [Statistical software]. College Station, TX: StataCorp LLC.
- Stremersch, S., Camacho, N., Vanneste, S., & Verniers, I. (2015). Unraveling scientific impact: Citation types in marketing journals. International Journal of Research in Marketing, 32(1), 64-77.
- Tahamtan, I., Afshar, A.S., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107(3), 1195-1225.
- Talaat, F.M., & Gamel, S.A. (2023). Predicting the impact of no. of authors on no. of citations of research publications based on neural networks. Journal of Ambient Intelligence and Humanized Computing, 14, 8499-8508. DOI: 10.1007/s12652-022-03882-1
- Thelwall, M. (2024). Can ChatGPT evaluate research quality? Journal of Data and Information Science, 9(2), 1-21. DOI: 10.2478/jdis-2024-0013
- Thelwall, M., Kousha, K, Abdoli, M., Stuart, E., Makita, M., Wilson, P., & Levitt, J. (2023). Why are co-authored academic articles more cited: Higher quality or larger audience? Journal of the Association for Information Science and Technology, 74(7), 791-810. DOI: 10.1002/asi.24755
- Thelwall, M., Kousha, K., Stuart, E., Makita, M., Abdoli, M., Wilson, P., & Levitt, J.M. (2023). Does the perceived quality of interdisciplinary research vary between fields? Journal of Documentation, 79(6), 1514-1531. DOI: 10.1108/JD-01-2023-0012
- Traag, V.A. (2021). Inferring the causal effect of journals on citations. Quantitative Science Studies, 2(2), 496-504.
- Uhly, K. M., Visser, L. M., & Zippel, K. S. (2015). Gendered patterns in international research collaborations in academia. Studies in Higher Education, 42(4), 760-782. DOI: 10.1080/03075079.2015.1072151
- van Lent, M., Overbeke, J., & Out, H.J. (2014). Role of editorial and peer review processes in publication bias: analysis of drug trials submitted to eight medical journals. PLoS ONE, 9(8), e104846.
- Waltman, L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365-391. DOI: 10.1016/j.joi.2016.02.007
- Waltman, L., Kaltenbrunner, W., Pinfield, S., & Woods, H.B. (2023). How to improve scientific peer review: Four schools of thought. Learned Publishing, 36(3), 334-347.
- Wang, J. (2013). Citation time window choice for research impact evaluation. Scientometrics, 94(3), 851-872. DOI: 10.1007/s11192-012-0775-9
- Wang, D., Song, C., & Barabási, A. (2013). Quantifying long-term scientific impact. Science, 342(6154), 127-132. DOI: 10.1126/science.1237825
- Wang, J., Thijs, B., & Glänzel, W. (2015). Interdisciplinarity and impact: Distinct effects of variety, balance, and disparity. PLoS ONE, 10(5), e0127298.
- Wang, X., Dworkin, J.D., Zhou, D., Stiso, J., Falk, E.B., Bassett, D.S., & Lydon-Staley, D.M. (2021). Gendered citation practices in the field of communication. Annals of the International Communication Association, 45(2), 134-153.
- Wang, X., Liu, C., Mao, W., & Fang, Z. (2015). The open access advantage considering citation, article usage and social media attention. Scientometrics, 103(2), 555-564. DOI: 10.1007/s11192-015-1547-0
- Wilsdon, J. (2016). The Metric Tide: Independent Review of the Role of Metrics in Research Assessment and Management. London: Sage Publications, Ltd.
- Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q. L., & Tang, Y. (2023). A brief overview of ChatGPT: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10(5), 1122-1136.
- Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of knowledge. Science, 316(5827), 1036-1039.
- Xie, J., Gong, K., Cheng, Y., & Ke, Q. (2019). The correlation between paper length and citations: A meta-analysis. Scientometrics, 118(3), 763-786.
- Yegros-Yegros, A., Rafols, I., & D’Este, P. (2015). Does interdisciplinary research lead to higher citation impact? The different effect of proximal and distal interdisciplinarity. PLoS ONE, 10(8). DOI: 10.1371/journal. pone.0135095
- Yu, X., Meng, Z., Qin, D., Shen, C., & Hua, F. (2022). The long-term influence of open access on the scientific and social impact of dental journal articles: An updated analysis. Journal of Dentistry, 119, 104067. DOI: 10.1016/j.jdent.2022.104067.
- Zhao, X., & Zhang, Y. (2022). Reviewer assignment algorithms for peer review automation: A survey. Information Processing & Management, 59(5), 103028.
- Zimmer, A., Krimmer, H., & Stallmann, F. (2006). Winners among losers: Zur feminisierung der Deutschen universitäten [Winners among losers: On the feminization of German universities]. Beiträge zur Hochschulforschung, 28(4), 30-56.
Language: English
Page range: 13 - 39
Submitted on: Sep 3, 2024
Accepted on: Feb 7, 2025
Published on: Feb 28, 2025
Published by: Chinese Academy of Sciences, National Science Library
In partnership with: Paradigm Publishing Services
Publication frequency: 4 times per year
Keywords:
Related subjects:
© 2025 Giovanni Abramo, Ciriaco Andrea D’Angelo, Leonardo Grilli, published by Chinese Academy of Sciences, National Science Library
This work is licensed under the Creative Commons Attribution 4.0 License.