Data journals: incentivizing data access and documentation within the scholarly communication system

William H. Walters

doi:10.1629/uksg.510

Abstract

Data journals provide strong incentives for data creators to verify, document and disseminate their data. They also bring data access and documentation into the mainstream of scholarly communication, rewarding data creators through existing mechanisms of peer-reviewed publication and citation tracking. These same advantages are not generally associated with data repositories, or with conventional journals’ data-sharing mandates. This article describes the unique advantages of data journals. It also examines the data journal landscape, presenting the characteristics of 13 data journals in the fields of biology, environmental science, chemistry, medicine and health sciences. These journals vary considerably in size, scope, publisher characteristics, length of data reports, data hosting policies, time from submission to first decision, article processing charges, bibliographic index coverage and citation impact. They are similar, however, in their peer review criteria, their open access license terms and the characteristics of their editorial boards.

References

1Leonardo Candela, Donatella Castelli, Paolo Manghi, and Alice Tani, “Data journals: a survey,” Journal of the Association for Information Science and Technology 66, 9 (2015): 1747–1762, DOI: 10.1002/asi.23358 (accessed 21 April 2020).
Back to article
2Patrick Andreoli-Versbach and Frank Mueller-Langer, “Open access to data: An ideal professed but not practised,” Research Policy 43, 9 (2014): 1621–1633, DOI: 10.1016/j.respol.2014.04.008 (accessed 21 April 2020); John Ernest Kratz and Carly Strasser, “Researcher Perspectives on Publication and Peer Review of Data,” PLOS ONE 10, 0117619 (2015): 1–21, DOI: https://doi.org/10.1371/journal.pone.0117619 (accessed 21 April 2020).
Back to article
3William G. Dewald, Jerry G. Thursby, and Richard G. Anderson, “Replication in Empirical Economics: The Journal of Money, Credit and Banking Project,” The American Economic Review 76, 4 (1986): 587–603, https://www.jstor.org/stable/1806061 (accessed 21 April 2020); Jelte M. Wicherts, Denny Borsboom, Judith Kats, and Dylan Molenaar, “The Poor Availability of Psychological Research Data for Reanalysis,” American Psychologist 61, 7 (2006): 726–728, DOI: 10.1037/0003-066X.61.7.726
Back to article
4James A. Mills et al., “Archiving Primary Data: Solutions for Long-Term Studies,” Trends in Ecology & Evolution 30, 10 (2015): 581–589, DOI: 10.1016/j.tree.2015.07.006 (accessed 21 April 2020).
Back to article
5“Sharing research data to improve public health: full joint statement by funders of health research,” 2019, Wellcome, https://wellcome.ac.uk/what-we-do/our-work/sharing-research-data-improve-public-health-full-joint-statement-funders-health (accessed 21 April 2020).
Back to article
6Christine L. Borgman, “The conundrum in sharing research data,” Journal of the American Society for Information Science and Technology 63, 6 (2012): 1059–1078, DOI: 10.1002/asi.22634 (accessed 21 April 2020); Eleni Castro et al., “Evaluating and Promoting Open Data Practices in Open Access Journals,” Journal of Scholarly Publishing 49, 1 (2017): 66–88, DOI: https://doi.org/10.3138/jsp.49.1.66 (accessed 21 April 2020).
Back to article
7Liz Silva, Theo Bloom, Emma Ganley, and Maggie Winker, “PLOS’ New Data Policy: Public Access to Data,” EveryONE (blog), PLOS ONE, February 24, 2014, https://blogs.plos.org/everyone/2014/02/24/plos-new-data-policy-public-access-data-2 (accessed 21 April 2020).
Back to article
8Giovanni Colavizza et al., The citation advantage of linking publications to research data, 2019, https://arxiv.org/abs/1907.02565 (accessed 21 April 2020).
Back to article
9Colavizza et al., The citation advantage; Tom E. Hardwicke et al., “Data availability, reusability, and analytic reproducibility: evaluating the impact of a mandatory open data policy at the journal Cognition,” Royal Society Open Science 5, 180448 (2018): 1–18, DOI: 10.1098/rsos.180448 (accessed 21 April 2020).
Back to article
10Rachel A. Spicer and Christoph Steinbeck, “A lost opportunity for science: journals promote data sharing in metabolomics but do not enforce it,” Metabolomics 14, 1 (2018): 16, DOI: 10.1007/s11306-017-1309-5 (accessed 21 April 2020).
Back to article
11William G. Dewald and Richard G. Anderson, “Replication and Reflection: A Decade at the Journal of Money, Credit, and Banking,” in Secrets of Economics Editors, ed. Michael Szenberg and Lall Ramrattan (Cambridge, MA: MIT Press, 2014), 199–212; Campbell R. Harvey, “Reflections on Editing the Journal of Finance, 2006 to 2012,” in Secrets of Economics Editors, ed. Michael Szenberg and Lall Ramrattan (Cambridge, MA: MIT Press, 2014), 67–81.
Back to article
12Frank Mueller-Langer and Patrick Andreoli-Versbach, “Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure,” Information Economics and Policy 42 (2018): 20–34, DOI: 10.1016/j.infoecopol.2017.05.004 (accessed 21 April 2020).
Back to article
13Candela et al., “Data journals”; Mark J. Costello, “Motivating Online Publication of Data,” BioScience 59, 5 (2009): 418–427, DOI: https://doi.org/10.1525/bio.2009.59.5.9 (accessed 21 April 2020); Melissa A. Haendel, Nicole A. Vasilevsky, and Jacqueline A. Wirz, “Dealing with Data: A Case Study on Information and Data Management Literacy,” PLOS Biology 19, 5 (2012): 1–4, DOI: https://doi.org/10.1371/journal.pbio.1001339 (accessed 21 April 2020); Mills et al., “Archiving Primary Data”; Sébastien Renaut et al., “Data Management, Archiving, and Sharing for Biologists and the Role of Research Institutions in the Technology-Oriented Age,” BioScience 68, 6 (2018): 400–411, DOI: https://doi.org/10.1093/biosci/biy038 (accessed 21 April 2020); Jeffrey N. Rouder, “The what, why, and how of born-open data,” Behavior Research Methods 48, 3 (2016): 1062–1069, DOI: https://doi.org/10.3758/s13428-015-0630-z (accessed 21 April 2020); Birgit Schmidt, Birgit Gemeinholzer, and Andrew Treloar, “Open Data in Global Environment Research: The Belmont Forum’s Open Data Survey,” PLOS ONE 11, 0146695 (2016): 1–29, DOI: https://doi.org/10.1371%2Fjournal.pone.0146695 (accessed 21 April 2020); Silva et al., “PLOS’ New Data Policy: Public Access to Data.”
Back to article
14Mills et al., “Archiving Primary Data.”
Back to article
15Mills et al., “Archiving Primary Data”, 585.
Back to article
16Mark D. Wilkinson et al., “The FAIR Guiding Principles for scientific data management and stewardship,” Scientific Data 3, 160018 (2016): 1–9, DOI: 10.1038/sdata.2016.18 (accessed 21 April 2020).
Back to article
17Borgman, “The conundrum in sharing research data,” 1067; Christine L. Borgman, Andrea Scharnhorst, and Milena S. Golshan, “Digital data archives as knowledge infrastructures: Mediating data sharing and reuse,” Journal of the Association for Information Science and Technology 70, 8 (2019): 888–904, DOI: https://doi.org/10.1002/asi.24172 (accessed 21 April 2020); Rouder, “The what, why, and how.”
Back to article
18Vishwas Chavan and Lyubomir Penev, “The data paper: a mechanism to incentivize data publishing in biodiversity science,” BMC Bioinformatics 12, 15 (2011): 1–12, DOI: 10.1186/1471-2105-12-S15-S2 (accessed 21 April 2020); John E. Kratz and Carly Strasser, “Data publication consensus and controversies,” F1000Research 3, 94 (2014): 1–21, DOI: https://doi.org/10.12688/f1000research.3979.3 (accessed 21 April 2020).
Back to article
19Mueller-Langer and Andreoli-Versbach, “Open access to research data.”
Back to article
20Mark J. Costello et al., “Biodiversity data should be published, cited, and peer reviewed,” Trends in Ecology & Evolution 28, 8 (2013): 454–461, DOI: 10.1016/j.tree.2013.05.002 (accessed 21 April 2020).
Back to article
21Candela et al., “Data journals”; Kratz and Strasser, “Data publication consensus”; Kratz and Strasser, “Researcher Perspectives.”
Back to article
22Kratz and Strasser, “Researcher Perspectives,” 14.
Back to article
23Sarah Callaghan et al., “Overlay Journals and Data Publishing in the Meteorological Sciences,” Ariadne 60 (2009), http://www.ariadne.ac.uk/issue60/callaghan-et-al/ (accessed 21 April 2020).
Back to article
24Candela et al., “Data journals.”
Back to article
25Candela et al., “Data journals.”
Back to article
26Beall’s List of Predatory Journals and Publishers, https://beallslist.net (accessed 21 April 2020); “Journal Citation Reports,” Clarivate; “Master Journal List,” Clarivate https://mjl.clarivate.com/home (accessed 21 April 2020); “About the Licenses”, Creative Commons, https://creativecommons.org/share-your-work/licensing-examples (accessed 21 April 2020); “Journal Finder,” Elsevier, https://journalfinder.elsevier.com (accessed 21 April 2020); Scopus “Sources,” Elsevier, https://www.scopus.com/sources (accessed 21 April 2020); “National Library of Medicine Catalog,” NCBI, https://www.ncbi.nlm.nih.gov/nlmcatalog (accessed 21 April 2020); Judith M. Nixon, “Core Journals in Library and Information Science: Developing a Methodology for Ranking LIS journals,” College & Research Libraries 75, 1 (2014): 66–90, DOI: https://doi.org/10.5860/crl12-387 (accessed 21 April 2020); “Journal suggester,” Springer Nature, https://journalsuggester.springer.com (accessed 21 April 2020); Wikipedia, https://www.wikipedia.org (accessed 21 April 2020).
Back to article
27Candela et al., “Data journals.”
Back to article
28Candela et al., “Data journals.”
Back to article
29Colavizza et al., The citation advantage; Hardwicke et al., “Data availability”; Spicer and Steinbeck, “A lost opportunity.”
Back to article
30This estimate is based on the ‘not found’ values presented in the table, and on the number of data reports published in each journal.
Back to article
31Candela et al., “Data journals”, 1755.
Back to article
32Gregory W. Lawrence et al., Risk Management of Digital Information: A File Format Investigation (Washington, DC: Council on Library and Information Resources, 2000), http://www.clir.org/pubs/reports/pub93 (accessed 21 April 2020); William H. Walters, “E-Books in Academic Libraries: Challenges for Acquisition and Collection Management,” portal: Libraries and the Academy 13, 2 (2013): 187–211, DOI: 10.1353/pla.2013.0012 (accessed 21 April 2020).
Back to article
33Peter Y. Chen and Paula M. Popovich, Correlation: Parametric and Nonparametric Measures (Thousand Oaks, CA: SAGE, 2002); David C. Howell, Statistical Methods for Psychology (Belmont, CA: Wadsworth, 2013); Laerd Statistics, Point-Biserial Correlation Using SPSS Statistics, https://statistics.laerd.com/spss-tutorials/point-biserial-correlation-using-spss-statistics.php (accessed 21 April 2020); Jason T. Newsom, Point-Biserial Correlation, Phi, & Cramer’s V, http://web.pdx.edu/~newsomj/pa551/lectur15.htm (accessed 21 April 2020).
Back to article
34“About Data,” Multidisciplinary Digital Publishing Institute, https://www.mdpi.com/journal/data/about (accessed 21 April 2020).
Back to article
35Candela et al., “Data journals.”
Back to article
36Kratz and Strasser, “Researcher Perspectives”; Limor Peer, Ann Green, and Elizabeth Stephenson, “Committing to Data Quality Review,” International Journal of Digital Curation 9, 1 (2014): 263–291, DOI: 10.2218/ijdc.v9i1.317 (accessed 21 April 2020).
Back to article
37Valerie Spezi et al., “’Let the community decide’? The vision and reality of soundness-only peer review in open-access mega-journals,” Journal of Documentation 74, 1 (2018): 137–161, DOI: 10.1108/JD-06-2017-0092 (accessed 21 April 2020).
Back to article
38These review and publication times were calculated from the dates shown on each website for the 50 most recent data reports. If the necessary dates were not available for individual reports, the values were estimated from statements on the journals’ websites or elsewhere (e.g., Elsevier Journal Finder and Springer Nature Journal suggester).
Back to article
39“Berlin Declaration on Open Access to Knowledge in the Sciences and Humanities,” Max-Planck-Gesellschaft zur Förderung der Wissenschaften, 2003, https://openaccess.mpg.de/Berlin-Declaration (accessed 21 April 2020).
Back to article
40Creative Commons Licenses; Iain Hrynaszkiewicz and Matthew J. Cockerill, “Open by default: a proposed copyright license and waiver agreement for open access research and data in peer-reviewed journals,” BMC Research Notes 5, 494 (2012): 1–12, DOI: 10.1186/1756-0500-5-494 (accessed 21 April 2020).
Back to article
41Candela et al., “Data journals.”
Back to article
42“Joint Declaration of Data Citation Principles,” Data Citation Synthesis Group, 2014, ed. M. Martone, https://www.force11.org/datacitationprinciples (accessed 21 April 2020); Bryan Lawrence et al., “Citation and Peer Review of Data: Moving Towards Formal Data Publication,” International Journal of Digital Curation 2, 6 (2011): 4–37, DOI: 10.2218/ijdc.v6i2.205 (accessed 21 April 2020); Gianmaria Silvello, “Learning to cite framework: How to automatically construct citations for hierarchical data,” Journal of the Association for Information Science and Technology 68, 6 (2017): 1505–1524, DOI: https://doi.org/10.1002/asi.23774 (accessed 21 April 2020); Gianmaria Silvello, “Theory and practice of data citation,” Journal of the Association for Information Science and Technology 69, 1 (2018): 6–20, DOI: https://doi.org/10.1002/asi.23917 (accessed 21 April 2020).
Back to article
43These estimates are based on data for journal size and index coverage. See Tables 3 and 7.
Back to article
44William H. Walters, “Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources,” IEEE Access 5 (2017): 22036–22053, DOI: 10.1109/ACCESS.2017.2761400 (accessed 21 April 2020).
Back to article
45David Stuart, “Data bibliometrics: metrics before norms,” Online Information Review 41, 3 (2017): 428–435, DOI: 10.1108/OIR-01-2017-0008 (accessed 21 April 2020).
Back to article
46Although Impact Factors, CiteScores, and related metrics do provide a rough guide to scholarly impact, it is important to recognize that all such measures have significant flaws, that impact may vary for reasons unrelated to “quality”, and that citation-based metrics and subjective ratings of journals are only modestly related. See, for example, Peter Haddawy, Saeed-Ul Hassan, Awais Asghar, and Sarah Amin, “A comprehensive examination of the relation of three citation-based journal metrics to expert judgment of journal quality,” Journal of Informetrics 10, 1 (2016): 162–173, DOI: 10.1016/j.joi.2015.12.005 (accessed 21 April 2020); San Francisco Declaration on Research Assessment (DORA), 2012, https://sfdora.org/read/ (accessed 21 April 2020); William H. Walters, “Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?” Journal of Informetrics 11, 3 (2017): 730–744, DOI: https://doi.org/10.1016/j.joi.2017.05.001 (accessed 21 April 2020); William H. Walters and Susanne Markgren, “Do faculty journal selections correspond to objective indicators of citation impact? Results for 20 academic departments at Manhattan College,” Scientometrics 118, 1 (2019): 321–337, DOI: https://doi.org/10.1007/s11192-018-2972-7 (accessed 21 April 2020).
Back to article
47Candela et al., “Data journals.”
Back to article
48Thu-Mai Christian, Sophia Lafferty-Hess, William G. Jacoby, and Thomas M. Carsey, “Operationalizing the Replication Standard: A Case Study of the Data Curation and Verification Workflow for Scholarly Journals,” 2018, https://osf.io/preprints/socarxiv/cfdba (accessed 21 April 2020); Dewald and Anderson, “Replication and Reflection”; Dewald et al., “Replication in Empirical Economics”; Jan H. Höffler, “Replication and Transparency in Economic Research,” Perspectives (blog), Institute for New Economic Thinking, 2015, https://www.ineteconomics.org/perspectives/blog/replication-and-transparency (accessed 21 April 2020); Arthur Lupia and Colin Elman, “Openness in Political Science: Data Access and Research Transparency,” PS: Political Science & Politics 47, 1 (2014): 19–42, DOI: https://doi.org/10.1017/S1049096513001716 (accessed 21 April 2020).
Back to article
49Although agencies such as Crossref and DataCite have worked to strengthen the mechanisms that underlie data citation, many (perhaps most) data citations conform to neither standard, and individual disciplines have established their own standards that are sometimes inconsistent with those of Crossref and DataCite. Moreover, there is currently no mechanism by which conventionally archived data files can be included in citation indexes such as Science Citation Index and Scopus.
Back to article
50Despite the emergence of new evaluation methods such as post-publication review, conventional peer review remains the only mechanism of quality control that is widely accepted by scholars in all disciplines. See, for example, Melissa Blankstein and Christine Wolff-Eisenberg, Ithaka S+ R US Faculty Survey 2018 (New York: Ithaka S+R, 2019), 10.18665/sr.311199 (accessed 21 April 2020); Christine Wolff-Eisenberg, Alisa B. Rod, and Roger C. Schonfeld, UK Survey of Academics 2015 (New York: Ithaka S+R, 2016), https://doi.org/10.18665/sr.282736 (accessed 21 April 2020); and the many discipline-specific surveys conducted by Ithaka S+R.
Back to article
51Walters, “E-Books in Academic Libraries.”
Back to article
52Blankstein and Wolff-Eisenberg, Ithaka S+ R US Faculty Survey; Kratz and Strasser, “Researcher Perspectives”; Wolff-Eisenberg, Rod, and Schonfeld, UK Survey of Academics.
Back to article
53Barbara E. Bierer, Mercè Crosas, and Heather H. Pierce, “Data Authorship as an Incentive to Data Sharing,” New England Journal of Medicine 376, 17 (2017): 1684–1687, DOI: 10.1056/NEJMsb1616595 (accessed 30 April 2020).
Back to article
54Ana Sofia Figueiredo, “Data Sharing: Convert Challenges into Opportunities,” Frontiers in Public Health 5, 327 (2017): 1–6, DOI: 10.3389/fpubh.2017.00327 (accessed 21 April 2020); Lawrence et al., “Citation and Peer Review.”
Back to article
55Courtney R. Butler and Brett D. Currier, “You Can’t Replicate What You Can’t Find: Data Preservation Policies in Economic Journals” (paper presented at the 2017 Annual Conference of the International Association for Social Science Information Services & Technology, Lawrence, Kansas, May 2017), https://osf.io/preprints/lissa/hf3ds.
Back to article
56Mills et al., “Archiving Primary Data.”
Back to article

Data journals: incentivizing data access and documentation within the scholarly communication system

Abstract

Paradigm

My account