Tracking direct and indirect impact on technology and policy of transformative research via ego citation network

Li, Xian; Hu, Xiaojun

doi:10.2478/jdis-2024-0018

Abstract

Purpose

The disseminating of academic knowledge to nonacademic audiences partly relies on the transition of subsequent citing papers. This study aims to investigate direct and indirect impact on technology and policy originating from transformative research based on ego citation network.

Design/methodology/approach

Key Nobel Prize-winning publications (NPs) in fields of gene engineering and astrophysics are regarded as a proxy for transformative research. In this contribution, we introduce a network-structural indicator of citing patents to measure technological impact of a target article and use policy citations as a preliminary tool for policy impact.

Findings

The results show that the impact on technology and policy of NPs are higher than that of their subsequent citation generations in gene engineering but not in astrophysics.

Research limitations

The selection of Nobel Prizes is not balanced and the database used in this study, Dimensions, suffers from incompleteness and inaccuracy of citation links.

Practical implications

Our findings provide useful clues to better understand the characteristics of transformative research in technological and policy impact.

Originality/value

This study proposes a new framework to explore the direct and indirect impact on technology and policy originating from transformative research.

References

Aristodemou, L., & Tietze, F. (2018). Citations as a measure of technological impact: A review of forward citation-based measures. World Patent Information, 53, 39–44. doi:10.1016/j.wpi.2018.05.001
Open DOI Search in Google Scholar Back to article
Atallah, G., & Rodriguez, G. (2006). Indirect patent citations. Scientometrics, 67(3), 437–465. doi:10.1556/Scient.67.2006.3.7
Open DOI Search in Google Scholar Back to article
Bartkowski, B., Theesfeld, I., Pirscher, F., & Timaeus, J. (2018). Snipping around for food: Economic, ethical and policy implications of CRISPR/Cas genome editing. Geoforum, 96, 172–180. doi:10.1016/j.geoforum.2018.07.017
Open DOI Search in Google Scholar Back to article
Belardo, M. B., & de Camargo, K. R. (2018). Biomedical knowledge and health policies: Hemolytic Uremic Syndrome and Fibromyalgia. Ciencia & Saude Coletiva, 23(9), 3085–3094. doi:10.1590/1413-81232018239.22742016
Open DOI Search in Google Scholar Back to article
Bornmann, L. (2013). What is societal impact of research and how can it be assessed? a literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217-233. doi:10.1002/asi.22803
Open DOI Search in Google Scholar Back to article
Bornmann, L., Haunschild, R., & Marx, W. (2016). Policy documents as sources for measuring societal impact: how often is climate change research mentioned in policy-related documents? Scientometrics, 109(3), 1477–1495. doi:10.1007/s11192-016-2115-y
Open DOI Search in Google Scholar Back to article
Campbell, D., Struck, B., Tippett, C., & Roberge, G. (2017, Oct 16-20). Impact of multidisciplinary research on innovation [Conference presentation]. 16th International Conference on Scientometrics and Informetrics (ISSI), Wuhan Univ, Wuhan, PEOPLES R CHINA.
Search in Google Scholar Back to article
Dance, A. (2013). Impact: Pack a punch. Nature, 502(7471), 398.
Search in Google Scholar Back to article
Didegah, F., Bowman, T. D., & Holmberg, K. (2018). On the Differences Between Citations and Altmetrics: An Investigation of Factors Driving Altmetrics Versus Citations for Finnish Articles. Journal of the Association for Information Science and Technology, 69(6), 832–843. doi:10.1002/asi.23934
Open DOI Search in Google Scholar Back to article
Dietz, J. S., & Rogers, J. D. (2012). Meanings and Policy Implications of “Transformative Research”: Frontiers, Hot Science, Evolution, and Investment Risk. Minerva, 50(1), 21–44. doi:10.1007/s11024-012-9190-x
Open DOI Search in Google Scholar Back to article
Economic and Social Research Council (ESRC). (2022). Defining impact. https://www.ukri.org/councils/esrc/impact-toolkit-for-economic-and-social-sciences/defining-impact/
Search in Google Scholar Back to article
Fragkiadaki, E., & Evangelidis, G. (2014). Review of the indirect citations paradigm: theory and practice of the assessment of papers, authors and journals. Scientometrics, 99(2), 261–288. doi:10.1007/s11192-013-1175-5
Open DOI Search in Google Scholar Back to article
Fragkiadaki, E., Evangelidis, G., Samaras, N., & Dervos, D. A. (2011). f-Value: measuring an article’s scientific impact. Scientometrics, 86(3), 671–686. doi:10.1007/s11192-010-0302-9
Open DOI Search in Google Scholar Back to article
Fujigaki, Y. (1998). Filling the gap between discussions on science and scientists’ everyday activities: applying the autopoiesis system theory to scientific knowledge. Social Science Information Sur Les Sciences Sociales, 37(1), 5–22. doi:10.1177/053901898037001001
Open DOI Search in Google Scholar Back to article
Herzog, C., Hook, D., & Konkiel, S. (2020). Dimensions: Bringing down barriers between scientometricians and data. Quantitative Science Studies, 1(1), 387–395. doi:10.1162/qss_a_00020
Open DOI Search in Google Scholar Back to article
Heydari, S., Shekofteh, M., & Kazerani, M. (2019). Relationship between Altmetrics and Citations: A Study on the Highly Cited Research Papers. Desidoc Journal of Library & Information Technology, 39(4), 169–174. doi:10.14429/djlit.39.4.14204
Open DOI Search in Google Scholar Back to article
Hu, X. J., & Rousseau, R. (2016). Scientific influence is not always visible: The phenomenon of under-cited influential publications. Journal of Infometrics, 10(4), 1079–1091. doi:10.1016/j. joi.2016.10.002
Open DOI Search in Google Scholar Back to article
Hu, X. J., & Rousseau, R. (2018). A new approach to explore the knowledge transition path in the evolution of science & technology: From the biology of restriction enzymes to their application in biotechnology. Journal of Informetrics, 12(3), 842–857. doi:10.1016/j.joi.2018.07.004
Open DOI Search in Google Scholar Back to article
Hu, X. J., Rousseau, R., & Chen, J. (2011). On the definition of forward and backward citation generations. Journal of Informetrics, 5(1), 27–36. doi:10.1016/j.joi.2010.07.004
Open DOI Search in Google Scholar Back to article
Hu, X. J., Rousseau, R., & Chen, J. (2012). A new approach for measuring the value of patents based on structural indicators for ego patent citation networks. Journal of the American Society for Information Science and Technology, 63(9), 1834–1842. doi:10.1002/asi.22632
Open DOI Search in Google Scholar Back to article
Hollingsworth, J. R. (2008). Scientific discoveries: An institutionalist and path-dependent perspective. Biomedical and Health Research-Commission of the European Communities Then IOS Press, 72, 317.
Search in Google Scholar Back to article
Huang, Z. H., Zong, Q. J., & Ji, X. R. (2022). The associations between scientific collaborations of LIS research and its policy impact. Scientometrics, 127(11), 6453–6470. doi:10.1007/s11192-022-04532-1
Open DOI Search in Google Scholar Back to article
Jones, T. H., & Hanney, S. (2016). Tracing the indirect societal impacts of biomedical research: development and piloting of a technique based on citations. Scientometrics, 107(3), 975–1003. doi:10.1007/s11192-016-1895-4
Open DOI Search in Google Scholar Back to article
Ke, Q. (2020). Technological impact of biomedical research: The role of basicness and novelty. Research Policy, 49(7), 15. doi:10.1016/j.respol.2020.104071
Open DOI Search in Google Scholar Back to article
Ke, Q. (2020). An analysis of the evolution of science-technology linkage in biomedicine. Journal of Informetrics, 14(4), 13. doi:10.1016/j.joi.2020.101074
Open DOI Search in Google Scholar Back to article
Ke, Q. (2023). Interdisciplinary research and technological impact: evidence from biomedicine. Scientometrics, 128(4), 2035–2077. doi:10.1007/s11192-023-04662-0
Open DOI Search in Google Scholar Back to article
Kim, G., & Bae, J. (2017). A novel approach to forecast promising technology through patent analysis. Technological Forecasting and Social Change, 117, 228–237. doi:10.1016/j.techfore.2016.11.023
Open DOI Search in Google Scholar Back to article
Koshland, D. E. (2007). The cha-cha-cha theory of scientific discovery. Science, 317(5839), 761-762. doi:10.1126/science.1147166
Open DOI Search in Google Scholar Back to article
Lavis, J. N., Ross, S. E., Hurley, J. E., Hohenadel, J. M., Stoddart, G. L., & Abelson, W. J. (2002). Examining the Role of Health Services Research in Public Policymaking. Milbank Quarterly, 80(1), 125–154.
Search in Google Scholar Back to article
Ledford, H. (2015). CRISPR, the disruptor. Nature, 522(7554), 20–24. doi:10.1038/522020a
Open DOI Search in Google Scholar Back to article
Lewison, G. (2004). James Bond and citations to his books. Scientometrics, 59(3), 311–320. doi:10.1023/B:SCIE.0000018536.84255.b1
Open DOI Search in Google Scholar Back to article
Li, D., Azoulay, P., & Sampat, B. N. (2017). The applied value of public investments in biomedical research. Science, 356(6333), 78–81. doi:10.1126/science.aal0010
Open DOI Search in Google Scholar Back to article
Li, X., Rousseau, R., Liang, L. M., Xi, F. J., Lu, Y. S., Yuan, Y. F., & Hu, X. J. (2022). Is low interdisciplinarity of references an unexpected characteristic of Nobel Prize winning research? Scientometrics, 127(4), 2105–2122. doi:10.1007/s11192-022-04290-0
Open DOI Search in Google Scholar Back to article
Liu, Y. X., & Rousseau, R. (2014). Citation analysis and the development of science: A case study using articles by some Nobel prize winners. Journal of the Association for Information Science and Technology, 65(2), 281–289. doi:10.1002/asi.22978
Open DOI Search in Google Scholar Back to article
Martin, B. R. (2011). The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster? Research Evaluation, 20(3), 247–254. doi:10.3152/095820211x13118583635693
Open DOI Search in Google Scholar Back to article
McMillan, G. S., Narin, F., & Deeds, D. L. (2000). An analysis of the critical role of public science in innovation: the case of biotechnology. Research Policy, 29(1), 1–8. doi:10.1016/s0048-7333(99)00030-x
Open DOI Search in Google Scholar Back to article
Mertens, D. M. (2021). Transformative Research Methods to Increase Social Impact for Vulnerable Groups and Cultural Minorities. International Journal of Qualitative Methods, 20, 9. doi:10.1177/16094069211051563
Open DOI Search in Google Scholar Back to article
Morton, S. (2015). Progressing research impact assessment: A ‘contributions’ approach. Research Evaluation, 24(4), 405–419. doi:10.1093/reseval/rvv016
Open DOI Search in Google Scholar Back to article
Mostert, S. P., Ellenbroek, S. P. H., Meijer, I., van Ark, G., & Klasen, E. C. (2010). Societal output and use of research performed by health research groups. Health Research Policy and Systems, 8, 10. doi:10.1186/1478-4505-8-30
Open DOI Search in Google Scholar Back to article
Narin, F., Hamilton, K. S., & Olivastro, D. (1997). The increasing linkage between US technology and public science. Research policy, 26(3), 317–330.
Search in Google Scholar Back to article
Narin, F., & Olivastro, D. (1992). Status report: linkage between technology and science. Research policy, 21(3), 237–249.
Search in Google Scholar Back to article
Narin, F., & Olivastro, D. (1998). Linkage between patents and papers: An interim EPO/US comparison. Scientometrics, 41(1-2), 51–59. doi:10.1007/bf02457966
Open DOI Search in Google Scholar Back to article
National Science Board. (2007). Enhancing Support of Transformative Research at the National Science Foundation. https://www.nsf.gov/nsb/documents/2007/tr_report.pdf.
Search in Google Scholar Back to article
Newson, R., Rychetnik, L., King, L., Milat, A., & Bauman, A. (2018). Does citation matter? Research citation in policy documents as an indicator of research impact - an Australian obesity policy case-study. Health Research Policy and Systems, 16, 12. doi:10.1186/s12961-018-0326-9
Open DOI Search in Google Scholar Back to article
Olmos-Penuela, J., Castro-Martinez, E., & D’Este, P. (2014). Knowledge transfer activities in social sciences and humanities: Explaining the interactions of research groups with nonacademic agents. Research Policy, 43(4), 696–706. doi:10.1016/j.respol.2013.12.004
Open DOI Search in Google Scholar Back to article
Petroni, G., & Venturini, K. (2023). Understanding technological spillovers: The case of main astrophysics European missions. Acta Astronautica, 204, 443–449. doi:10.1016/j.actaastro.2023.01.022
Open DOI Search in Google Scholar Back to article
Rafols, I., Porter, A. L., & Leydesdorff, L. (2010). Science Overlay Maps: A New Tool for Research Policy and Library Management. Journal of the American Society for Information Science and Technology, 61(9), 1871–1887. doi:10.1002/asi.21368
Open DOI Search in Google Scholar Back to article
Rousseau, R. (1987). The Gozinto theorem: Using citations to determine influences on a scientific publication. Scientometrics, 11(3-4), 217–229.
Search in Google Scholar Back to article
Rousseau, R. (2011, Jul 04-07). Algebraic structures in the ego article citation network [Conference presentation]. 13th Conference of the International-Society-for-Scientometrics-and-Informetrics (ISSI), Durban, South Africa.
Search in Google Scholar Back to article
Rousseau, R., Egghe, L., & Guns, R. (2018). Becoming metric-wise: A bibliometric guide for researchers. Chandos Publishing.
Search in Google Scholar Back to article
Strasser, T., de Kraker, J., & Kemp, R. (2020). Three Dimensions of Transformative Impact and Capacity: A Conceptual Framework Applied in Social Innovation Practice. Sustainability, 12(11), 40. doi:10.3390/su12114742
Open DOI Search in Google Scholar Back to article
Thelwall, M. (2016). Does astronomy research become too dated for the public? Wikipedia citations to astronomy and astrophysics journal articles 1996-2014. Profesional De La Informacion, 25(6), 893–900. doi:10.3145/epi.2016.nov.06
Open DOI Search in Google Scholar Back to article
Venturini, K., & Verbano, C. (2014). A systematic review of the Space technology transfer literature: Research synthesis and emerging gaps. Space Policy, 30(2), 98–114. doi:10.1016/j.spacepol.2014.04.003
Open DOI Search in Google Scholar Back to article
Veugelers, R., & Wang, J. (2019). Scientific novelty and technological impact. Research Policy, 48(6), 1362–1372. doi:10.1016/j.respol.2019.01.019
Open DOI Search in Google Scholar Back to article
Winnink, J. J., Tijssen, R. J. W., & van Raan, A. F. J. (2019). Searching for new breakthroughs in science: How effective are computerised detection algorithms? Technological Forecasting and Social Change, 146, 673–686. doi:10.1016/j.techfore.2018.05.018
Open DOI Search in Google Scholar Back to article
Wuestman, M., Hoekman, J., & Frenken, K. (2020). A typology of scientific breakthroughs. Quantitative Science Studies, 1(3), 1203–1222. doi:10.1162/qss_a_00079
Open DOI Search in Google Scholar Back to article
Yin, Y., Gao, J., Jones, B. F., & Wang, D. S. (2021). Coevolution of policy and science during the pandemic. Science, 371(6525), 128–130. doi:10.1126/science.abe3084
Open DOI Search in Google Scholar Back to article
Zhang, L., Sun, B. B., Shu, F., & Huang, Y. (2022). Comparing paper level classifications across different methods and systems: an investigation of Nature publications. Scientometrics, 127(12), 7633–7651. doi:10.1007/s11192-022-04352-3
Open DOI Search in Google Scholar Back to article