Open-Source Transformer-Based Information Retrieval System for Energy Efficient Robotics Related Literature

Bertoncel, Tine

doi:10.2478/orga-2025-0012

Abstract

Background and Purpose

This article employs the Hugging Face keyphrase-extraction-kbir-inspec machine learning model to analyze 654 abstracts on the topic of energy efficiency in systems and control, computer science and robotics.

Methods

This study targeted specific arXiv categories related to energy efficiency, scraping and processing abstracts with a state-of-the-art Transformer-based Hugging Face AI model to extract keyphrases, thereby enabling the creation of related keyphrase networks and the retrieval of relevant scientific preprints.

Results

The results demonstrate that state-of-the-art open-source machine learning models can extract valuable information from unstructured data, revealing prominent topics in the evolving field of energy-efficiency. Conclusion: This showcases the current landscape and highlights the capability of such information systems to pinpoint both well researched and less researched areas, potentially serving as an information retrieval system or early warning system for emerging technologies that promote environmental sustainability and cost efficiency.

References

Abbas, J., & Sağsan, M. (2019). Impact of knowledge management practices on green innovation and corporate sustainable development: A structural analysis. Journal of Cleaner Production, 229, 611-620. https://doi.org/10.1016/j.jclepro.2019.05.024
Search in Google Scholar Back to article
AlMahamid, F., & Grolinger, K. (2021). Reinforcement learning algorithms: An overview and classification. 2021 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE). https://doi.org/10.1109/ccece53047.2021.9569056
Search in Google Scholar Back to article
arXiv. (2024). arXiv Category Taxonomy. https://arxiv.org/category_taxonomy
Search in Google Scholar Back to article
Association for Computing Machinery. (2012). The 2012 acm computing classification system. https://www.acm.org/publications/class-2012
Search in Google Scholar Back to article
Babu, M. S., Ali, A. A., & Rao, A. S. (2014). A study on information retrieval methods in text mining. International Journal of Engineering Research & Technology (IJERT), 2(15). https://doi.org/10.17577/IJERTCONV2IS15028
Search in Google Scholar Back to article
Backlund, S., Broberg, S., Ottosson, M., & Thollander, P. (2012). Energy efficiency potentials and energy management practices in swedish firms. European Council for an Energy Efficient Economy, 11–14
Search in Google Scholar Back to article
Bagchi, C., Malmi, E., & Grabowicz, P. (2024). Effects of research paper promotion via arxiv and x. arXiv. https://doi.org/10.48550/arXiv.2401.11116
Search in Google Scholar Back to article
Bayani, A. (2024). Sélection automatisée d’informations crédibles sur la santé en ligne (Publication No. 33068) [Mémoire, Université de Montréal]. Papyrus: Institutional Repository. https://hdl.handle.net/1866/33068
Search in Google Scholar Back to article
Bengesi, S., El-Sayed, H., Sarker, M. K., Houkpati, Y., Irungu, J., & Oladunni, T. (2023). Advancements in generative ai: A comprehensive review of gans, gpt, autoencoders, diffusion model, and transformers. IEEE Access, 12. https://doi.org/10.48550/arXiv.2311.10242
Search in Google Scholar Back to article
Boas, T., Dunning, T., & Bussel, J. (2005). Will the digital revolution revolutionize development? drawing together the debate. Studies in Comparative International Development, 40, 95–110. https://doi.org/10.1007/BF02686296
Search in Google Scholar Back to article
Brugger, H., Eichhammer, W., Mikova, N., & Dönitz, E. (2021). Energy efficiency vision 2050: How will new societal trends influence future energy demand in the european countries? Energy Policy, 152, 112216. https://doi.org/10.1016/j.enpol.2021.112216
Search in Google Scholar Back to article
Carabin, G., Wehrle, E., & Vidoni, R. (2017). A review on energy-saving optimization methods for robotic and automatic systems. Robotics, 6(4), 39. https://doi.org/10.3390/robotics6040039
Search in Google Scholar Back to article
Chen, L., Chen, Z., Zhang, Y., Liu, Y., Osman, A. I., Farghali, M., Hua, J., Al-Fatesh, A., Ihara, I., Rooney, D. W., & Yap, P.-S. (2023). Artificial intelligence-based solutions for climate change: A review. Environmental Chemistry Letters, 21(5), 2525–2557. https://doi.org/10.1007/s10311-023-01617-y
Search in Google Scholar Back to article
Chen, M., Li, Y., & Li, R. (2019). Research on neural machine translation model. Journal of Physics: Conference Series, 1237, 052020. https://doi.org/10.1088/1742-6596/1237/5/052020
Search in Google Scholar Back to article
Clement, C., Bierbaum, M., O’Keeffe, K., & Alemi, A. (2019). On the use of arxiv as a dataset. arXiv. https://doi.org/10.48550/arXiv.1905.00075
Search in Google Scholar Back to article
Davis, P., & Fromerth, M. (2006). Does the arxiv lead to higher citations and reduced publisher downloads for mathematics articles? Scientometrics, 71, 203–215. https://doi.org/10.1007/s11192-007-1661-8
Search in Google Scholar Back to article
de la Rue du Can, S., Letschert, V., Agarwal, S., Park, W. Y., & Kaggwa, U. (2022). Energy efficiency improves energy access affordability. Energy for Sustainable Development, 70, 560–568. https://doi.org/10.1016/j.esd.2022.09.003
Search in Google Scholar Back to article
Debortoli, S., Müller, O., Junglas, I., & vom Brocke, J. (2016). Text mining for information systems researchers: An annotated topic modeling tutorial. Communications of the Association for Information Systems, 39. https://doi.org/10.17705/1CAIS.03907
Search in Google Scholar Back to article
Ding, C., Ke, J., Levine, M., & Zhou, N. (2024). Potential of artificial intelligence in reducing energy and carbon emissions of commercial buildings at scale. Nature Communications, 15(1), 5916. https://doi.org/10.1038/s41467-024-50088-4
Search in Google Scholar Back to article
Dinu, V., (2024). Innovative Application of Artificial Intelligence in Business Impacting Socio-Economic Progress. Amfiteatru Economic, 26(66), 398-401. https://doi.org/10.24818/EA/2024/66/398.
Search in Google Scholar Back to article
Dinu, V., Baciu, L. E., Mortan, M., & Veres, V. A. (2023). Effect of economic, institutional and cultural factors on the implementation of eu energy policies. Amfiteatru Economic Journal, 25(63), 306–306. https://doi.org/10.24818/EA/2023/63/306
Search in Google Scholar Back to article
Engelmann, B., Haak, F., Kreutz, C. K., Khasmakhi, N. N., & Schaer, P. (2023). Text simplification of scientific texts for non-expert readers. arXiv. https://doi.org/10.48550/arXiv.2307.03569
Search in Google Scholar Back to article
Fan, L., Pan, S., Liu, G., & Zhou, P. (2017). Does energy efficiency affect financial performance? evidence from chinese energy-intensive firms. Journal of Cleaner Production, 151, 53–59. https://doi.org/10.1016/j.jclepro.2017.03.044
Search in Google Scholar Back to article
Ferrer-Sapena, A., Aleixandre-Benavent, R., Peset, F., & Sánchez-Pérez, E. (2018). Citations to arxiv preprints by indexed journals and their impact on research evaluation. Journal of Information Science: Theory and Practice, 6, 6–16. https://doi.org/10.1633/JISTAP.2018.6.4.1
Search in Google Scholar Back to article
Firoozeh, N., Nazarenko, A., Alizon, F., & Daille, B. (2020). Keyword extraction: Issues and methods.Natural Language Engineering, 26(3), 259–291. https://doi.org/10.1017/S1351324919000457
Search in Google Scholar Back to article
Froelich, J., Ananyan, S. (2008). Decision Support via Text Mining. In: Handbook on Decision Support Systems 1. International Handbooks Information System (pp 609–635). Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-48713-5_28
Search in Google Scholar Back to article
Frontier Group. (2023, October 5). Fact file: Computing is using more energy than ever. https://frontiergroup.org/resources/fact-file-computing-is-using-more-energy-than-ever/
Search in Google Scholar Back to article
Gajzler, M. (2010). Hybrid advisory system for the process of industrial flooring repairs. Technological and Economic Development of Economy, 16, 219–232. https://doi.org/10.1016/j.proeng.2013.03.056
Search in Google Scholar Back to article
Gauci, J., Conti, E., Liang, Y., Virochsiri, K., He, Y., Kaden, Z., Narayanan, V., Ye, X., & Fujimoto, S. (2018). Horizon: Facebook’s open source applied reinforcement learning platform. arXiv. https://doi.org/10.48550/arXiv.1811.00260
Search in Google Scholar Back to article
Gennitsaris, S., Oliveira, M. C., Vris, G., Bofilios, A., Ntinou, T., Frutuoso, A. R., Queiroga, C., Giannatsis, J., Sofianopoulou, S., & Dedoussis, V. (2023). Energy efficiency management in small and medium-sized enterprises: Current situation, case studies and best practices. Sustainability, 15(4). https://doi.org/10.3390/su15043727
Search in Google Scholar Back to article
Ghobakhloo, M. (2020). Industry 4.0, digitization, and opportunities for sustainability. Journal of Cleaner Production, 252, 119869. https://doi.org/10.1016/j.jclepro.2019.119869
Search in Google Scholar Back to article
Ghungrad, S., Mohammed, A., & Haghighi, A. (2023). Energy-efficient and quality-aware part placement in robotic additive manufacturing. Journal of Manufacturing Systems, 68, 536–559. https://doi.org/10.1016/j.jmsy.2023.05.019
Search in Google Scholar Back to article
Gourdji, S., Sibley, A., & Lobell, D. (2013). Global crop exposure to critical high temperatures in the reproductive period: Historical trends and future projections. Environmental Research Letters, 8. https://doi.org/10.1088/1748-9326/8/2/024041
Search in Google Scholar Back to article
Hambarde, K., & Proença, H. (2023). Information retrieval: Recent advances and beyond. Journal of Information Retrieval, 15(3), 200–215. https://doi.org/10.1109/ACCESS.2023.3295776
Search in Google Scholar Back to article
Hansen, J., Sato, M., Ruedy, R., Lo, K., Lea, D., & Medina-Elizade, M. (2006). Global temperature change. Proceedings of the National Academy of Sciences, 103, 14288–14293. https://doi.org/10.1073/pnas.0606291103
Search in Google Scholar Back to article
Hassani, H., Beneki, C., Unger, S., Mazinani, M. T., & Yeganegi, M. R. (2020). Text mining in big data analytics. Big Data and Cognitive Computing, 4(1). https://doi.org/10.3390/bdcc4010001
Search in Google Scholar Back to article
Hughes, T., Kerry, J., Baird, A., Connolly, S., Dietzel, A., Eakin, C., Heron, S., Hoey, A., Hoogenboom, M., Liu, G., McWilliam, M., Pears, R., Pratchett, M., Skirving, W., Stella, J., & Torda, G. (2018). Global warming transforms coral reef assemblages. Nature, 556, 492–496. https://doi.org/10.1038/s41586-018-0041-2
Search in Google Scholar Back to article
Jazdi, N. (2014). Cyber physical systems in the context of industry 4.0. 2014 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), 1–4. https://doi.org/10.1109/AQTR.2014.6857843
Search in Google Scholar Back to article
Jena, M., Mishra, S., & Moharana, H. (2019). Application of industry 4.0 to enhance sustainable manufacturing. Environmental Progress & Sustainable Energy, 39, 1–11. https://doi.org/10.1002/ep.13360
Search in Google Scholar Back to article
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023). Active retrieval augmented generation. arXiv. https://doi.org/10.48550/arXiv.2305.06983
Search in Google Scholar Back to article
Khan, S. (2018). Text mining methodology for effective online marketing. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 3(8), 465–469. https://doi.org//10.32628/CSEIT12283129
Search in Google Scholar Back to article
Knuutila, M., Kosonen, A., Jaatinen-Värri, A., & Laaksonen, P. (2022). Profitability comparison of active and passive energy efficiency improvements in public buildings. Energy Efficiency, 15(6), 38. https://doi.org/10.1007/s12053-022-10046-9
Search in Google Scholar Back to article
Kochhar, P., Kalliamvakou, E., Nagappan, N., Zimmer-mann, T., & Bird, C. (2021). Moving from closed to open source: Observations from six transitioned projects to github. IEEE Transactions on Software Engineering, 47, 1838–1856. https://doi.org/10.1109/TSE.2019.2937025
Search in Google Scholar Back to article
Kulkarni, M., Mahata, D., Arora, R., & Bhowmik, R. (2022). Learning rich representation of keyphrases from text. arXiv. https://doi.org/10.48550/arX-iv.2112.08547
Search in Google Scholar Back to article
Kwon, H. (2022). Visualization methods of information regarding academic publications, research topics, and authors. Proceedings, 81(1). https://doi.org/10.3390/proceedings2022081154
Search in Google Scholar Back to article
Leonardi, N. (2009, January 1). Terminology as a system of knowledge representation: An overview. https://hdl.handle.net/11393/44305
Search in Google Scholar Back to article
Li, X., & Takano, T. (2022). Finding NLP papers by asking a multi-hop question. Proceedings of the Annual Conference of JSAI. https://doi.org/10.11517/jsaislud.95.0_59
Search in Google Scholar Back to article
López-Torres, G. C., Garza-Reyes, J. A., Maldonado-Guzmán, G., Kumar, V., Rocha-Lona, L., & Cherrafi, A. (2019). Knowledge management for sustainability in operations. Production Planning & Control, 30(10–12), 813–826. https://doi.org/10.1080/09537287.2019.1582091
Search in Google Scholar Back to article
MATLAB. (2024). Graph and network algorithms - matlab & simulink. https://nl.mathworks.com/help/matlab/graph-and-network-algorithms.html
Search in Google Scholar Back to article
Meinshausen, M., Meinshausen, N., Hare, W., Raper, S., Frieler, K., Knutti, R., Frame, D., & Allen, M. (2009). Greenhouse-gas emission targets for limiting global warming to 2°c. Nature, 458, 1158–1162. https://doi.org/10.1038/nature08017
Search in Google Scholar Back to article
Melnick, S. (2024). A Computational Journey Through Conspiracy Theories: A Genealogical Approach (Publication No. 1841) [Master’s thesis, University of Vermont]. ScholarWorks. https://scholarworks.uvm.edu/graddis/1841/
Search in Google Scholar Back to article
Melnik, A., & Ermolaev, K. (2020). Strategy context of decision making for improved energy efficiency in industrial energy systems. Energies, 13(7). https://doi.org/10.3390/en13071540
Search in Google Scholar Back to article
ML6Team. (2023). Keyphrase extraction model: Kbir-in-spec. https://huggingface.co/ml6team/keyphraseextraction-kbir-inspec
Search in Google Scholar Back to article
Moed, H. (2006). The effect of ’open access’ upon citation impact: An analysis of arxiv’s condensed matter section. arXiv. https://doi.org/10.48550/arXiv.cs/0611060
Search in Google Scholar Back to article
Monwar, M., Semiari, O., & Saad, W. (2018). Optimized path planning for inspection by unmanned aerial vehicles swarm with energy constraints. Proceedings of the IEEE Global Communications Conference and Ad Hoc and Sensor Networks Symposium. https://digitalcommons.georgiasouthern.edu/electrical-eng-facpubs/138/
Search in Google Scholar Back to article
Naveed, H., Khan, A. U., Qiu, S., Saqib, M., Anwar, S., Usman, M., Akhtar, N., Barnes, N., & Mian, A. (2024). A comprehensive overview of large language models. arXiv. https://doi.org/10.48550/arXiv.2307.06435
Search in Google Scholar Back to article
National Research Council (2011). The future of computing performance: Game over or next level. The National Academies Press. https://doi.org/10.17226/12980
Search in Google Scholar Back to article
Nomoto, T. (2022). Keyword extraction: A modern perspective. SN Computer Science, 4(1), 92. https://doi.org/10.1007/s42979-022-01481-7
Search in Google Scholar Back to article
ECA (2022). Energy efficiency in enterprises. https://www.eca.europa.eu/Lists/ECADocuments/SR22_02/SR_Energy-effic-enterpr_en.pdf
Search in Google Scholar Back to article
Papagiannopoulou, E., & Tsoumakas, G. (2019). A review of keyphrase extraction. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(2). https://doi.org/10.1002/widm.1339
Search in Google Scholar Back to article
Philbin, S., Viswanathan, R., & Telukdarie, A. (2022). Understanding how digital transformation can enable SMEs to achieve sustainable development: A systematic literature review. Small Business International Review, 6(2), 1–12. https://doi.org/10.26784/sbir.v6i1.473
Search in Google Scholar Back to article
Ricke, M., & Caldeira, K. (2014). Maximum warming occurs about one decade after a carbon dioxide emission. Environmental Research Letters, 9(12). https://doi.org/10.1088/1748-9326/9/12/124002
Search in Google Scholar Back to article
Ritchie, H., & Roser, M. (2020, September 18). Greenhouse gas emissions by sector. https://ourworldindata.org/ghg-emissions-by-sector
Search in Google Scholar Back to article
Roblek, V., Meško, M., Pušavec, F., & Likar, B. (2021). The role and meaning of the digital transformation as a disruptive innovation on small and medium manufacturing enterprises. Frontiers in Psychology, 12, 592528. https://doi.org/10.3389/fpsyg.2021.592528
Search in Google Scholar Back to article
Rocca, R., Rosa, P., Sassanelli, C., Fumagalli, L., & Terzi, S. (2020). Integrating virtual reality and digital twin in circular economy practices: A laboratory application case. Sustainability, (6), 1–8. https://doi.org/10.3390/su12062286
Search in Google Scholar Back to article
Rosenbloom, L. (2019). Arxiv.org. Charleston Advisor, 21(2), 8–10. https://doi.org/10.5260/chara.21.2.8
Search in Google Scholar Back to article
Samsi, S., Zhao, D., McDonald, J., Li, B., Michaleas, A., Jones, M., Bergeron, W., Kepner, J., Tiwari, D., & Gadepally, V. (2023). From words to watts: Benchmarking the energy costs of large language model inference. arXiv. https://doi.org/10.48550/arX-iv.2310.03003
Search in Google Scholar Back to article
Sarker, I. H. (2021). Deep learning: A comprehensive overview on techniques, taxonomy, applications and research directions. SN Computer Science, 2(6), 420. https://doi.org/10.1007/s42979-021-00815-1
Search in Google Scholar Back to article
Shahzad, M., Qu, Y., Zafar, A. U., Rehman, S. U., & Islam, T. (2020). Exploring the influence of knowledge management process on corporate sustainable performance through green innovation. Journal of Knowledge Management, 24(9), 2079-2106. https://doi.org/10.1108/JKM-11-2019-0624
Search in Google Scholar Back to article
Sharma, R., Jabbour, C. J. C., & Lopes de Sousa Jabbour, A. B. (2021). Sustainable manufacturing and industry 4.0: What we know and what we don’t. Journal of Enterprise Information Management, 34(1), 230-266. https://doi.org/10.1108/JEIM-01-2020-0024
Search in Google Scholar Back to article
Shin, A., & Narihira, T. (2021). Transformer-exclusive cross-modal representation for vision and language. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2719–2725. https://doi.org/10.18653/v1/2021.findings-acl.240
Search in Google Scholar Back to article
Smith, D., Scaife, A., Hawkins, E., Bilbao, R., Boer, G., Caian, M., Caron, L., Danabasoglu, G., Delworth, T., Doblas-Reyes, F., Doescher, R., Dunstone, N., Hermanson, R., Ishii, M., Kharin, V., Kimoto, M., Koenigk, T., Kushnir, Y., Matei, D., ... Yeager, S. (2018). Predicted chance that global warming will temporarily exceed 1.5 °c. Geophysical Research Letters, 45(21), 11, 895–11, 903. https://doi.org/10.1029/2018GL079362
Search in Google Scholar Back to article
Sutton, C., & Gong, L. (2017). Popularity of arxiv.org within computer science. arXiv. https://doi.org/10.48550/arXiv.1710.05225
Search in Google Scholar Back to article
The World Bank and ITU. (2024). Document of the world bank: Project appraisal document on a proposed loan. https://documents1.world-bank.org/curated/en/099121223165540890/pdf/P17859712a98880541a4b71d57876048abb.pdf
Search in Google Scholar Back to article
Tran, H. T. H., Martinc, M., Caporusso, J., Doucet, A., & Pollak, S. (2023). The recent advances in automatic term extraction: A survey. arXiv. https://doi.org/10.48550/arXiv.2301.06767
Search in Google Scholar Back to article
Vasiliev, A., & Goryachev, A. (2022). Application of text mining technology to solve project management problems. 2022 XXV International Conference on Soft Computing and Measurements (SCM), 202–205. https://doi.org/10.1109/SCM55405.2022.9794858
Search in Google Scholar Back to article
Virjan, D., Popescu, C. R., Pop, I., & Popescu, D. (2023). Energy transition and sustainable development at the level of the European Union. Amfiteatru Economic Journal, Bucharest University of Economic Studies, Bucharest, 25(63), 429–446. https://doi.org/http://dx.doi.org/10.24818/EA/2023/63/429
Search in Google Scholar Back to article
Wang, L., Mohammed, A., Wang, X., & Schmidt, B. (2017). Energy-efficient robot applications towards sustainable manufacturing. International Journal of Computer Integrated Manufacturing, 31(8), 692–700. https://doi.org/10.1080/0951192X.2017.1379099
Search in Google Scholar Back to article
Warner, K., & Wäger, M. (2019). Building dynamic capabilities for digital transformation: An ongoing process of strategic renewal. Long Range Planning, 52(3), 326–349. https://doi.org/10.1016/j.lrp.2018.12.001
Search in Google Scholar Back to article
Yang, G., Tang, H., Ding, M., Sebe, N., & Ricci, E. (2021). Transformer-based attention networks for continuous pixel-wise prediction. 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 16249–16259. https://doi.org/10.48550/arXiv.2103.12091
Search in Google Scholar Back to article
Zhu, Y., Zhang, X., Hu, S., Xue, Y., & Liang, J. (2024). Enhancing keyphrase extraction with data augmentation using large language models. EPJ Data Science, 13(1), 5. https://doi.org/10.1140/epjds/s13688-024-00470-5
Search in Google Scholar Back to article

Open-Source Transformer-Based Information Retrieval System for Energy Efficient Robotics Related Literature

Abstract

Background and Purpose

Methods

Results

Paradigm

My account