Developments in Open Data Norms

Thomas Rhys Evans

doi:10.5334/jopd.60

Abstract

Open data has been transformative for the scientific and public understanding of the recent COVID-19 pandemic, bringing into sharp focus the clear benefits of increasing transparency and accountability within psychological research. Despite the knowledge that individual gatekeeping of data is antithetical to the goals of the scientific community, research norms across Psychology are changing too slowly. This editorial reviews the recent developments made in open data practices and norms, the upcoming demands of openness, and marks the changes made to the Journal of Open Psychology Data to further support progress towards a culture of transparency and collaboration.

References

Alsheikh-Ali, A. A., Qureshi, W., Al-Mallah, M. H., & Ioannidis, J. P. (2011). Public availability of published research data in high-impact journals. PloS one, 6(9), e24357. DOI: 10.1371/journal.pone.0024357
Open DOI Search in Google Scholar Back to article
Azevedo, F., Parsons, S., Micheli, L., Strand, J. F., Rinke, E., Guay, S., … & FORRT. (2019, December 13). Introducing a Framework for Open and Reproducible Research Training (FORRT). DOI: 10.31219/osf.io/bnh7p
Open DOI Search in Google Scholar Back to article
Chambers, C. D. (2020). Verification Reports: A new article type at Cortex. Cortex, 129, A1–A3. DOI: 10.1016/j.cortex.2020.04.020
Open DOI Search in Google Scholar Back to article
Chivers, T. (2019). Does psychology have a conflict-of-interest problem?. Nature, 571(7763), 20–24. DOI: 10.1038/d41586-019-02041-5
Open DOI Search in Google Scholar Back to article
Colavizza, G., Hrynaszkiewicz, I., Staden, I., Whitaker, K., & McGillivray, B. (2020). The citation advantage of linking publications to research data. PloS one, 15(4), e0230416. DOI: 10.1371/journal.pone.0230416
Open DOI Search in Google Scholar Back to article
Dos Reis, N. D., Ferreira, C. M., Silva, M. T., & Galvão, T. F. (2021). Frequency of receiving requested data for a systematic review and associated factors: a cross-sectional study. Accountability in Research. DOI: 10.1080/08989621.2021.1910029
Open DOI Search in Google Scholar Back to article
Evans, T. R., Branney, P., Clements, A., & Hatton, E. (2021). Improving evidence-based practice through preregistration of applied research: barriers and recommendations. Accountability in Research. DOI: 10.1080/08989621.2021.1969233
Open DOI Search in Google Scholar Back to article
Evans, T. R., Pownall, M., Collins, E., Henderson, E., Pickering, J., O’Mahony, A., Zaneva, M., Jaquiery, M., & Dumbalska, T. (Under review). A Network of Change: United Action on Research Integrity.
Search in Google Scholar Back to article
Farrow, R. (2017). Open education and critical pedagogy. Learning, Media and Technology, 42(2), 130–146. DOI: 10.1080/17439884.2016.1113991
Open DOI Search in Google Scholar Back to article
Fleming, J. I., Wilson, S. E., Hart, S. A., Therrien, W. J., & Cook, B. G. (2021). Open accessibility in education research: Enhancing the credibility, equity, impact, and efficiency of research. Educational Psychologist, 56(2), 110–121. DOI: 10.1080/00461520.2021.1897593
Open DOI Search in Google Scholar Back to article
Gardner, L., Ratcliff, J., Dong, E., & Katz, A. (2021). A need for open public data standards and sharing in light of COVID-19. The Lancet Infectious Diseases, 21(4), e80. DOI: 10.1016/S1473-3099(20)30635-6
Open DOI Search in Google Scholar Back to article
Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University, 348.
Search in Google Scholar Back to article
Hardwicke, T. E., & Ioannidis, J. P. (2018). Populating the Data Ark: An attempt to retrieve, preserve, and liberate data from the most highly-cited psychology and psychiatry articles. PloS one, 13(8), e0201856. DOI: 10.1371/journal.pone.0201856
Open DOI Search in Google Scholar Back to article
Hardwicke, T. E., Mathur, M. B., MacDonald, K., Nilsonne, G., Banks, G. C., Kidwell, M. C., … & Frank, M. C. (2018). Data availability, reusability, and analytic reproducibility: Evaluating the impact of a mandatory open data policy at the journal Cognition. Royal Society Open Science, 5(8), 180448. DOI: 10.1098/rsos.180448
Open DOI Search in Google Scholar Back to article
Hardwicke, T. E., Thibault, R. T., Kosie, J. E., Wallach, J. D., Kidwell, M. C., & Ioannidis, J. P. (2021). Estimating the prevalence of transparency and reproducibility-related research practices in psychology (2014–2017). Perspectives on Psychological Science, 17(1), 239–251. DOI: 10.1177/1745691620979806
Open DOI Search in Google Scholar Back to article
Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V., Nichols, T. E., & Wagenmakers, E. J. (2018). Data sharing in psychology: A survey on barriers and preconditions. Advances in Methods and Practices in Psychological Science, 1(1), 70–85. DOI: 10.1177/2515245917751886
Open DOI Search in Google Scholar Back to article
Hrynaszkiewicz, I., Harney, J., & Cadwallader, L. (2021). A survey of researchers’ needs and priorities for data sharing. Data Science Journal, 20(1), 31. DOI: 10.5334/dsj-2021-031
Open DOI Search in Google Scholar Back to article
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L. S., … & Nosek, B. A. (2016). Badges to acknowledge open practices: A simple, low-cost, effective method for increasing transparency. PLoS biology, 14(5), e1002456. DOI: 10.1371/journal.pbio.1002456
Open DOI Search in Google Scholar Back to article
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., … & Vazire, S. (2018). A practical guide for transparency in psychological science. Collabra: Psychology, 4(1), 20. DOI: 10.1525/collabra.158
Open DOI Search in Google Scholar Back to article
Levenstein, M. C., & Lyle, J. A. (2018). Data: Sharing is caring. Advances in Methods and Practices in Psychological Science, 1(1), 95–103. DOI: 10.1177/2515245918758319
Open DOI Search in Google Scholar Back to article
Moshontz, H., Campbell, L., Ebersole, C. R., IJzerman, H., Urry, H. L., Forscher, P. S., … & Chartier, C. R. (2018). The Psychological Science Accelerator: Advancing psychology through a distributed collaborative network. Advances in Methods and Practices in Psychological Science, 1(4), 501–515. DOI: 10.1177/2515245918797607
Open DOI Search in Google Scholar Back to article
Nosek, B. A., Spies, J., & Motyl, M. (2012). Scientific Utopia: II - Restructuring Incentives and Practices to Promote Truth Over Publishability. Perspectives on Psychological Science, 7, 615–631. DOI: 10.1177/1745691612459058
Open DOI Search in Google Scholar Back to article
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., … & Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422–1425. DOI: 10.1126/science.aab2374
Open DOI Search in Google Scholar Back to article
Nuijten, M. B., Borghuis, J., Veldkamp, C. L., Dominguez-Alvarez, L., Van Assen, M. A., Wicherts, J. M., … & Chambers, C. (2017). Journal data sharing policies and statistical reporting inconsistencies in psychology. Collabra: Psychology, 3(1): 31. DOI: 10.1525/collabra.102
Open DOI Search in Google Scholar Back to article
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007). Sharing detailed research data is associated with increased citation rate. PloS one, 2(3), e308. DOI: 10.1371/journal.pone.0000308
Open DOI Search in Google Scholar Back to article
Quintana, D. S. (2020). A synthetic dataset primer for the biobehavioural sciences to promote reproducibility and hypothesis generation. Elife, 9, e53275. DOI: 10.7554/eLife.53275.sa2
Open DOI Search in Google Scholar Back to article
Rouder, J. N. (2016). The what, why, and how of born-open data. Behavior Research Methods, 48(3), 1062–1069. DOI: 10.3758/s13428-015-0630-z
Open DOI Search in Google Scholar Back to article
Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., … & Nosek, B. A. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1(3), 337–356. DOI: 10.1177/2515245917747646
Open DOI Search in Google Scholar Back to article
Srivastava, S. [@hardsci]. (2018, December 18). “In the same spirit as the Pottery Barn Rule. Call them Verification Reports. One-pager online reports linked from the original, where you can report you reran the analyses to verify the numbers in the paper, + possibly ran other analyses to verify strength of conclusions” [Tweet]. Retrieved from https://twitter.com/hardsci/status/107506585851329740
Search in Google Scholar Back to article
Tedersoo, L., Küngas, R., Oras, E., Köster, K., Eenmaa, H., Leijen, Ä., … & Sepp, T. (2021). Data sharing practices and data availability upon request differ across scientific disciplines. Scientific Data, 8(1), 1–11. DOI: 10.1038/s41597-021-00981-0
Open DOI Search in Google Scholar Back to article
Tenopir, C., Dalton, E. D., Allard, S., Frame, M., Pjesivac, I., Birch, B., … & Dorsett, K. (2015). Changes in data sharing and data reuse practices and perceptions among scientists worldwide. PloS one, 10(8), e0134826. DOI: 10.1371/journal.pone.0134826
Open DOI Search in Google Scholar Back to article
Towse, J. N., Ellis, D. A., & Towse, A. S. (2021). Opening Pandora’s Box: Peeking inside Psychology’s data sharing practices, and seven recommendations for change. Behavior Research Methods, 53(4), 1455–1468. DOI: 10.3758/s13428-020-01486-1
Open DOI Search in Google Scholar Back to article
Vines, T. H., Albert, A. Y., Andrew, R. L., Débarre, F., Bock, D. G., Franklin, M. T., … & Rennison, D. J. (2014). The availability of research data declines rapidly with article age. Current Biology, 24(1), 94–97. DOI: 10.1016/j.cub.2013.11.014
Open DOI Search in Google Scholar Back to article
Vines, T. H., Andrew, R. L., Bock, D. G., Franklin, M. T., Gilbert, K. J., Kane, N. C., … & Yeaman, S. (2013). Mandated data archiving greatly improves access to research data. The FASEB Journal, 27(4), 1304–1308. DOI: 10.1096/fj.12-218164
Open DOI Search in Google Scholar Back to article
Wicherts, J. (2013). Science revolves around the data. Journal of Open Psychology Data, 1(1). e1. DOI: 10.5334/jopd.e1
Open DOI Search in Google Scholar Back to article
Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PloS one, 6(11), e26828. DOI: 10.1371/journal.pone.0026828
Open DOI Search in Google Scholar Back to article
Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61(7), 726–728. DOI: 10.1037/0003-066X.61.7.726
Open DOI Search in Google Scholar Back to article
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., Bouwman, J., Brookes, A. J., Clark, T., Crosas, M., Dillo, I., Dumon, O., Edmunds, S., Evelo, C. T., Finkers, R., … & Mons, B. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Scientific Data, 3(1), 160018. DOI: 10.1038/sdata.2016.18
Open DOI Search in Google Scholar Back to article

Developments in Open Data Norms

Abstract

Paradigm

My account