The Flemish Performance-based Research Funding System: A Unique Variant of the Norwegian Model

Engels, Tim C. E.; Guns, Raf

doi:10.2478/jdis-2018-0020

Abstract

The BOF-key is the performance-based research funding system that is used in Flanders, Belgium. In this paper we describe the historical background of the system, its current design and organization, as well as its effects on the Flemish higher education landscape. The BOF-key in its current form relies on three bibliometric parameters: publications in Web of Science, citations in Web of Science, and publications in a comprehensive regional database for SSH publications. Taken together, the BOF-key forms a unique variant of the Norwegian model: while the system to a large extent relies on a commercial database, it avoids the problem of inadequate coverage of the SSH. Because the bibliometric parameters of the BOF-key are reused in other funding allocation schemes, their overall importance to the Flemish universities is substantial.

References

BiGGAR Economics. (2017). The economic contribution of the Flemish Universities. Midlothian. Retrieved from http://www.vlir.be/media/docs/impact/VLIR%20Economic%20Impact%20Report%2011Dec17.pdf
Search in Google Scholar Back to article
Billiet, J., Bocken, H., Bruynseraede, Y., Clarysse, W., De Dijn, H., De Mey, M., … Witte, E. (2004). Bibliometrie in de humane wetenschappen. Brussel: KVAB.
Search in Google Scholar Back to article
Bonaccorsi, A. (2018). Towards an epistemic approach to evaluation in SSH. In A. Bonaccorsi (Ed.), The Evaluation of Research in Social Sciences and Humanities: Lessons from the Italian Experience (pp. 1–29). Cham: Springer International Publishing. Retrieved from https://doi.org/10.1007/978-3-319-68554-0_1
Search in Google Scholar Back to article
Cornelis, G. (2013). Eerlijke wetenschap. Waarom hebben wetenschapsmensen een geweten nodig? Lannoo Campus.
Search in Google Scholar Back to article
Dahler-Larsen, P. (2017). The new configuration of metrics, rules and guidelines creates a disturbing ambiguity in academia. Retrieved from http://blogs.lse.ac.uk/impactofsocialsciences/2017/07/13/the-new-configuration-of-metrics-rules-and-guidelines-creates-a-disturbing-ambiguity-in-academia/
Search in Google Scholar Back to article
De Wever, B. (2007). Van A1, A2, A3... Concurrentievervalsing in academia. In P. Loobuyck, G. Vanheeswijck, W. Van Herck, E. Grieten, & K. Vercauteren (Eds.), Welke universiteit willen wij (niet)? (pp. 199–203). Gent: Academia Press.
Search in Google Scholar Back to article
Debackere, K., Arnold, E., Sivertsen, G., Spaapen, J., & Sturn, D. (2018). Performance-based funding of university research: Summary Report. Luxembourg: Publications Office of the European Union. Retrieved from https://doi.org/10.2777/047514
Search in Google Scholar Back to article
Debackere, K., & Glänzel, W. (2004). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253–276.
Search in Google Scholar Back to article
Debackere, K., Glänzel, W., & Thijs, B. (2017). Bibliometrische analyse van levens, natuur en technische wetenschappen. Retrieved from https://www.vlaamsindicatorenboek.be/4.1/bibliometrische-analyse-levens-natuur-en-technische-wetenschappen
Search in Google Scholar Back to article
Delanote, J., Hoskens, M., Verheyden, L., & Viaene, P. (2017). Totale O&O-personeel. Retrieved from https://www.vlaamsindicatorenboek.be/3.4/totale-oo-personeel
Search in Google Scholar Back to article
Engels, T. C. E., Ossenblok, T. L. B., & Spruyt, E. H. J. (2012). Changing publication patterns in the Social Sciences and Humanities, 2000–2009. Scientometrics, 93(2), 373–390. Retrieved from https://doi.org/10.1007/s11192-012-0680-2
Search in Google Scholar Back to article
Engels, T. C. E., Spruyt, E. H. J., Glänzel, W., & Debackere, K. (2009). Het vlaams academisch bibliografisch bestand voor de Sociale en Humane Wetenschappen: Instrument ten dienste van een optimaal wetenschapsbeleid? Tijdschrift Voor Onderwijsrecht & Onderwijsbeleid, 2008–09(5), 395–403.
Search in Google Scholar Back to article
Engels, T. C. E., Starčič, A. I., Kulczycki, E., Pölönen, J., & Sivertsen, G. (2018). Are book publications disappearing from scholarly communication in the social sciences and humanities? In STI 2018 Conference Proceedings (pp. 774–780). Leiden: Centre for Science and Technology Studies (CWTS). Retrieved from https://openaccess.leidenuniv.nl/handle/1887/65259
Search in Google Scholar Back to article
Eykens, J., Guns, R., Rahman, A. I. M. J., Sīle, L., & Engels, T. C. E. (2018). Predatory Open Access journals: A review of past screenings within the Flemish performance based research funding system (2014–2018). In R. Costas, T. Franssen, & A. Yegros-Yegros (Eds.), STI 2018 Conference Proceedings (pp. 799–806). Leiden: Centre for Science and Technology Studies (CWTS). Retrieved from http://hdl.handle.net/1887/65285
Search in Google Scholar Back to article
Giménez-Toledo, E., Manana-Rodriguez, J., Engels, T. C. E., Ingwersen, P., Pölönen, J., Sivertsen, G., …, & Zuccala, A. A. (2016). Taking scholarly books into account. Current developments in five European countries. Scientometrics, 107(2), 685–699. Retrieved from https://doi.org/10.1007/s11192-016-1886-5
Search in Google Scholar Back to article
Glänzel, W., & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357–367.
Search in Google Scholar Back to article
Guns, R. (2016). Alphabetical co-authorship in the social sciences and humanities: Evidence from a comprehensive local database. In Proceedings of the 21st International Conference on Science and Technology Indicators (pp. 957–964). València: Editorial Universitat Politècnica de València.
Search in Google Scholar Back to article
Guns, R. (2018). Concentration of academic book publishers. In STI 2018 Conference Proceedings (pp. 518–525). Leiden: Centre for Science and Technology Studies (CWTS). Retrieved from https://openaccess.leidenuniv.nl/handle/1887/65268
Search in Google Scholar Back to article
Guns, R., & Engels, T. C. E. (2016). Effects of performance-based research funding on publication patterns in the social sciences and humanities. In I. Ràfols (Ed.), Proceedings of the 21^st International Conference on Science and Technology Indicators (pp. 893–900). València: Editorial Universitat Politècnica de València.
Search in Google Scholar Back to article
Guns, R., & Engels, T. C. E. (2017). Bibliometrische analyse van sociale en humane wetenschappen. Retrieved from https://www.vlaamsindicatorenboek.be/4.2/bibliometrische-analyse-van-sociale-en-humane-wetenschappen
Search in Google Scholar Back to article
Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41, 251–261.
Search in Google Scholar Back to article
Hicks, D. (2013). One size doesn’t fit all: On the co-evolution of national evaluation systems and social science publishing. Confero, 1(1), 67–90. Retrieved from https://doi.org/10.3384/confero13v1121207b
Search in Google Scholar Back to article
Korlaar, L., Bongers, F., & Groot Beumer, T. (2018). Evaluatie van de Bijzondere Onderzoeksfondsen (BOF) (2017.074-1814). Utrecht: Dialogic.
Search in Google Scholar Back to article
Korlaar, L., van Kerkhof, D., Steur, J., De Boer, P. J., & Groot Beumer, T. (2018). Evaluatie van het Vlaams Academisch Bibliografisch Bestand Sociale en Humane Wetenschappen (VABB-SHW) (No. 2017.074-1724). Utrecht: Dialogic.
Search in Google Scholar Back to article
Kulczycki, E., Engels, T. C. E., & Nowotniak, R. (2017). Publication patterns in the social sciences and humanities in Flanders and Poland. In Proceedings of the 16^th International Conference on Scientometrics and Informetrics (pp. 95–104). Wuhan.
Search in Google Scholar Back to article
Kulczycki, E., Engels, T. C. E., Pölönen, J., Bruun, K., Dušková, M., Guns, R., …, & Zuccala, A. A. (2018). Publication patterns in the social sciences and humanities: Evidence from eight European countries. Scientometrics, 116(1), 463–486. Retrieved from "https://doi.org/10.1007/s11192-018-2711-0">https://doi.org/10.1007/s11192-018-2711-0
Search in Google Scholar Back to article
Levecque, K., Anseel, F., De Beuckelaer, A., Van Der Heyden, J., & Gisle, L. (2017). Work organization and mental health problems in PhD students. Research Evaluation, 46(4), 868–879. Retrieved from https://doi.org/10.1016/j.respol.2017.02.008
Search in Google Scholar Back to article
Luwel, M., Moed, H. F., Nederhof, A. J., De Samblanx, V., Verbrugghen, K., & van der Wurff, L. J. (1999). Towards indicators of research performance in the social sciences and humanities. An exploratory study in the fields of law and linguistics at Flemish universities. Brussel.
Search in Google Scholar Back to article
Luwel, M., Noyons, E. C. M., & Moed, H. F. (1999). Bibliometric assessment of research performance in Flanders: Policy background and implications. R&D Management, 29(2), 133–141.
Search in Google Scholar Back to article
Moed, H. F., Luwel, M., Houben, J. A., Spruyt, E., & Van Den Berghe, H. (1998). The effects of changes in the funding structure of the Flemish universities on their research capacity, productivity and impact during the 1980’s and early 1990’s. Scientometrics, 43(2), 231–255.
Search in Google Scholar Back to article
Moed, H. F., Luwel, M., & Nederhof, J. (2002). Towards research performance in the humanities. Library Trends, 50(3), 498–520.
Search in Google Scholar Back to article
Ossenblok, T. L. B., & Engels, T. C. E. (2015). Edited books in the Social Sciences and Humanities: Characteristics and collaboration analysis. Scientometrics, 104(1), 219–237. Retrieved from https://doi.org/10.1007/s11192-015-1544-3
Search in Google Scholar Back to article
Ossenblok, T. L. B., Engels, T. C. E., & Sivertsen, G. (2012). The representation of the social sciences and humanities in the Web of Science—a comparison of publication patterns and incentive structures in Flanders and Norway (2005–9). Research Evaluation, 21(4), 280–290.
Search in Google Scholar Back to article
Ossenblok, T. L. B., Guns, R., & Thelwall, M. (2015). Book editors in the social sciences and humanities: An analysis of publication and collaboration patterns of established researchers in Flanders. Learned Publishing, 28(4), 261–273. Retrieved from https://doi.org/10.1087/20150405
Search in Google Scholar Back to article
Ossenblok, T. L. B., Verleysen, F. T., & Engels, T. C. E. (2014). Coauthorship of journal articles and book chapters in the social sciences and humanities (2000–2010). Journal of the Association for Information Science and Technology, 65(5), 882–897.
Search in Google Scholar Back to article
Pölönen, J., Auranen, O., Engels, T. C. E., & Kulczycki, E. (2018). Taking national language publications into account: the case of the Finnish performance-based research funding system. Presented at the STI 2018: 23^rd International Conference on Science and Technology Indicators, Leiden.
Search in Google Scholar Back to article
Pölönen, J., Engels, T. C. E., Guns, R., Sivertsen, G., & Verleysen, F. T. (2017). SSH journal publishing in Flanders and Finland. Presented at the 2^nd Conference on Research Evaluation in the Social Sciences and Humanities, Antwerp, Belgium.
Search in Google Scholar Back to article
Pölönen, J., Engels, T. C. E., Guns, R., & Verleysen, F. T. (2017). Is my publication peer reviewed? A comparison of top-down and bottom-up identification of peer review in the framework of the Finnish and Flemish performance-based research funding systems. Presented at the STI 2017: Conference on Science, Technology and Innovation Indicators, Paris.
Search in Google Scholar Back to article
Power, M. (1999). The Audit Society: Rituals of Verification. Oxford, New York: Oxford University Press.
Search in Google Scholar Back to article
Sīle, L., Guns, R., Sivertsen, G., & Engels, T. C. E. (2017). European Databases and Repositories for Social Sciences and Humanities Research Output. Retrieved from https://doi.org/10.6084/m9.figshare.5172322.v2
Search in Google Scholar Back to article
Sīle, L., Pölönen, J., Sivertsen, G., Guns, R., Engels, T. C. E., Arefiev, P., …, & Teitelbaum, R. (2018). Comprehensiveness of national bibliographic databases for social sciences and humanities: Findings from a European survey. Research Evaluation, forthcoming. Retrieved from "https://doi.org/10.1093/reseval/rvy016."https://doi.org/10.1093/reseval/rvy016
Search in Google Scholar Back to article
Sivertsen, G. (2016). A bibliometric indicator with a balanced representation of all fields. In Proceedings of the 21^st International Conference on Science and Technology Indicators (pp. 910–914). Editorial Universitat Politècnica de València. Retrieved from https://doi.org/10.4995/STI2016.2016.4543
Search in Google Scholar Back to article
Sivertsen, G. (2017). Problems and considerations in the design of bibliometric indicators for national performance based research funding systems. Presented at the STI 2017, Paris.
Search in Google Scholar Back to article
Spruyt, E. H. J., & Engels, T. C. E. (2013). Nieuwe sleutel verdeling van middelen Bijzonder Onderzoeksfonds. Thema: Tijdschrift voor Hoger Onderwijs en Management, 13(3), 56–61.
Search in Google Scholar Back to article
te Velde, R., Bongers, F., Hermanussen, L., De Boer, P. J., & Steur, J. (2018). Evaluatie van het Expertisecentrum Onderzoek en Ontwikkelingsmonitoring (ECOOM) (No. 2018.031-1817). Utrecht: Dialogic.
Search in Google Scholar Back to article
Zuijdam, F., Ploeg, M., Mostert, B., & Boekholt, P. (2013). Evaluatie van het Vlaams Academisch Bibliografisch bestand voor de Sociale en Humane Wetenschappen (VABB-SHW). Amsterdam: Technopolis Group.
Search in Google Scholar Back to article
Van Den Berghe, H., Houben, J. A., De Bruin, R. E., Moed, H. F., Kint, A., Luwel, M., & Spruyt, E. H. J. (1998). Bibliometric indicators of university research performance in Flanders. Journal of the American Society for Information Science, 49(1), 59–67.
Search in Google Scholar Back to article
Vandevelde, K., Toye, A., Vanden Berghe, J., & Lemahieu, I. (2009). UGent nota BOF-sleutel. Gent.
Search in Google Scholar Back to article
Verleysen, F. T., & Engels, T. C. E. (2012). Historical publications at Flemish universities, 2000–2009. An analysis on the basis of the Flemish Academic Database for the Social Sciences and Humanities. Journal of Belgian History, 42(4), 110–143.
Search in Google Scholar Back to article
Verleysen, F. T., & Engels, T. C. E. (2013). A label for peer-reviewed books. Journal of the American Society for Information Science and Technology, 64(2), 428–430.
Search in Google Scholar Back to article
Verleysen, F. T., & Engels, T. C. E. (2014a). Barycenter representation of book publishing internationalization in the social sciences and humanities. Journal of Informetrics, 8, 234–240.
Search in Google Scholar Back to article
Verleysen, F. T., & Engels, T. C. E. (2014b). Internationalization of peer reviewed and non-peer reviewed book publications in the Social Sciences and Humanities. Scientometrics, 101(2), 1431–1444.
Search in Google Scholar Back to article
Verleysen, F. T., Ghesquière, P., & Engels, T. C. E. (2014). The objectives, design and selection process of the Flemish Academic Bibliographic Database for the Social Sciences and Humanities (VABB-SHW). In W. Blockmans, L. Engwall, & D. Weaire (Eds.), Bibliometrics. Use and Abuse in the Review of Research Performance (pp. 117–127). London: Portland Press. Retrieved from http://www.portlandpress.com/pp/books/online/wg87/087/0117/0870117.pdf
Search in Google Scholar Back to article
Verleysen, F. T., & Ossenblok, T. L. B. (2017). Profiles of monograph authors in the social sciences and humanities: An analysis of productivity, career stage, co-authorship, disciplinary affiliation and gender, based on a regional bibliographic database. Scientometrics, 111(3), 1673–1686. Retrieved from https://doi.org/10.1007/s11192-017-2312-3
Search in Google Scholar Back to article
Verleysen, F. T., & Rousseau, R. (2017). How the existence of a regional bibliographic information system can help evaluators to conform to the principles of the Leiden Manifesto. Journal of Educational Media & Library Sciences, 54(1). Retrieved from "https://repository.uantwerpen.be/docman/irua/559ff0/142620.pdf">https://repository.uantwerpen.be/docman/irua/559ff0/142620.pdf
Search in Google Scholar Back to article
Verleysen, F. T., & Weeren, A. (2016a). Clustering by publication patterns of senior authors in the social sciences and humanities. Journal of Informetrics, 10, 254–272. Retrieved from https://doi.org/10.1016/j.joi.2016.01.004
Search in Google Scholar Back to article
Verleysen, F. T., & Weeren, A. (2016b). Mapping diversity of publication patterns in the social sciences and humanities. An approach making use of fuzzy cluster analysis. Journal of Data and Information Science, 3(1), 1–27.
Search in Google Scholar Back to article
Zacharewicz, T., Lepori, B., Reale, E., & Jonkers, K. (2018). Performance-based research funding in EU Member States—a comparative assessment. Science and Public Policy. Retrieved from "https://doi.org/10.1093/scipol/scy041."https://doi.org/10.1093/scipol/scy041
Search in Google Scholar Back to article

The Flemish Performance-based Research Funding System: A Unique Variant of the Norwegian Model

Abstract

Paradigm

My account