Approaches to creating ‘humane’ research evaluation metrics for the humanities

Stacy Konkiel

doi:10.1629/uksg.445

Abstract

There are many complexities and challenges associated with developing ‘humane’ research evaluation metrics in the humanities. This monumental task can only be addressed by reverse engineering evaluation metrics based upon the practices and values that funders, institutions, professional societies and individuals want to encourage in their disciplines. The work of the HuMetricsHSS initiative is described in this article as a framework for doing so.

References

1Butler L, Modifying publication practices in response to funding formulas, Research Evaluation, 2003, 12(1), 39–46; DOI: 10.3152/147154403781776780 (accessed 16 October 2018).
Back to article
2Hoffman A J, In Praise of ‘B’ Journals, Inside Higher Ed, 2017: https://www.insidehighered.com/views/2017/03/28/academics-shouldnt-focus-only-prestigious-journals-essay (accessed 16 October 2018).
Back to article
3Butler, ref. 1.
Back to article
4Hoffman, ref. 2.
Back to article
5Haustein S and Lariviere V, The Use of Bibliometrics for Assessing Research: Possibilities, Limitations and Adverse Effects. In: Incentives and Performance: Governance of Research Organizations, Ed Welpe I M et al., 2015, Springer International Publishing, London, pp. 121–139; DOI: 10.1007/978-3-319-09785-5_8 (accessed 17 October 2018).
Back to article
6Moore S, Neylon C, Eve M P, O’Donnell D P and Pattinson D, “Excellence R Us”: university research and the fetishisation of excellence, Palgrave Communications 3, 2017, 16105; DOI: 10.1057/palcomms.2016.105 (accessed 16 October 2018).
Back to article
7Sivertsen G, Patterns of internationalization and criteria for research assessment in the social sciences and humanities, Scientometrics, 2016, 107(2), 357–368; DOI: 10.1007/s11192-016-1845-1 (accessed 16 October 2018).
Back to article
8Modern Language Association, Guidelines for Evaluating Work in Digital Humanities and Digital Media, 2012: https://www.mla.org/About-Us/Governance/Committees/Committee-Listings/Professional-Issues/Committee-on-Information-Technology/Guidelines-for-Evaluating-Work-in-Digital-Humanities-and-Digital-Media (accessed 16 October 2018).
Back to article
9American Historical Association, Ad Hoc Committee on the Evaluation of Digital Scholarship by Historians, Guidelines for the Professional Evaluation of Digital Scholarship by Historians, 2015: https://www.historians.org/teaching-and-learning/digital-history-resources/evaluation-of-digital-scholarship-in-history/guidelines-for-the-professional-evaluation-of-digital-scholarship-by-historians (accessed 16 October 2018).
Back to article
10American Anthropological Association, AAA Guidelines for Tenure and Promotion Review: Communicating Public Scholarship in Anthropology, 2017: http://www.americananthro.org/AdvanceYourCareer/Content.aspx?ItemNumber=21713 (accessed 16 October 2018).
Back to article
11Curry S, Let’s move beyond the rhetoric: it’s time to change how we judge research, Nature, 2018, 554(147); DOI: 10.1038/d41586-018-01642-w (accessed 16 October 2018).
Back to article
12Schimanski L A & Alperin J P, The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future, F1000Research, 2018, 7, 1605; DOI: 10.12688/f1000research.16493.1 (accessed 16 October 2018).
Back to article
13Ball C, Barrett K, Berkery P, Clemons J, Crosby S, Falk-Krzesinski H J and Konkiel S, Promotion & Tenure Reform Workgroup Report, Open Scholarship Initiative Proceedings, 2017, 2(0): https://journals.gmu.edu/osi/article/view/1928 (accessed 16 October 2018).
Back to article
14De Rijcke S, Wouters P F, Rushforth A D, Franssen T P and Hammarfelt B, Evaluation practices and effects of indicator use – a literature review, Research Evaluation, 2016, 25(2), 161–169; DOI: 10.1093/reseval/rvv038 (accessed 16 October 2018).
Back to article
15Ochsner M, Hug S E, Daniel H D, Research assessment in the humanities: Towards criteria and procedures, 2016, Springer International Publishing, London; DOI: 10.1007/978-3-319-29016-4 (accessed 16 October 2018).
Back to article
16Guetzkow J, Lamont M and Mallard G, What is Originality in the Social Sciences and the Humanities?, American Sociological Review, 2004, 69(2), 190–212; DOI: 10.1177/000312240406900203 (accessed 24 October 2018).
Back to article
17Gogolin I, Åström and Hansen A, Approaches on Assessing Quality in European Educational Research Introduction to the volume. In: Assessing Quality in European Educational Research: Indicators and Approaches, Ed Gogolin I, Åström F and Hansen A, 2014, Springer Fachmedien Wiesbaden, Wiesbaden; DOI: 10.1007/978-3-658-05969-9_1 (accessed 16 October 2018).
Back to article
18Hammarfelt B, Following the Footnotes: A Bibliometric Analysis of Citation Patterns in Literary Studies, 2012, Uppsala: Uppsala University: http://urn.kb.se/resolve?urn=urn:nbn:se:hb:diva-280 (accessed 16 October 2018).
Back to article
19Ochsner M, Hug S and Galleron I, The future of research assessment in the humanities: bottom-up assessment procedures, Palgrave Communications, 2017: DOI: 10.1057/palcomms.2017.20 (accessed 16 October 2018).
Back to article
20De Rijcke S et al., ref. 14.
Back to article
21Ochsner M et al., ref. 15.
Back to article
22Hammarfelt B, An examination of the possibilities that altmetric methods offer in the case of the humanities (RIP), Proceedings of ISSI 2013 – 14th International Society of Scientometrics and Informetrics Conference, 2013, 1, 720–727.
Back to article
23Belfiore E, ‘Impact’, ‘value’ and ‘bad economics’: Making sense of the problem of value in the arts and humanities, Arts and Humanities in Higher Education, 2014; DOI: 10.1177/1474022214531503 (accessed 16 October 2018).
Back to article
24HuMetricsHSS Initiative: www.humetricshss.org (accessed 16 October 2018).
Back to article
25Triangle SCI: https://trianglesci.org/about/ (accessed 24 October 2018).
Back to article
26Ochsner M et al., ref. 15.
Back to article

Approaches to creating ‘humane’ research evaluation metrics for the humanities

Abstract

Paradigm

My account