References
- Andreessen, M “Why Software Is Eating The World,” The Wall Street Journal, August 2011. Available online:
http://online.wsj.com/news/articles/SB10001424053111903480904576512250915629460 . - Royal Society 2012 “Science as an open enterprise,” Available from:
https://royalsociety.org/policy/projects/science-public-enterprise/report/ . - Editorial 2011 “Devil in the details,” Nature, 470(7334): 305–306, DOI: 10.1038/470305b
- Alberts, B, Cicerone, R J, Fienberg, S E, Kamb, A, McNutt, M, Nerem, R M, Schekman, R, Shiffrin, R, Stodden, V, Suresh, S, Zuber, M T, Kline Pope, B and Jamieson, K 2015 “Self-correction in science at work,” Science, 348(6242): 1420–1422. DOI: 10.1126/science.aab3847
- Collberg, C and Proebsting, T A 2016 “Repeatability in Computer Systems Research,” Communications of the ACM, 59(3): 62–69. DOI: 10.1145/2812803
- Crick,T, De Vos, M, Brain, M and Fitch, J 2009 “Generating Optimal Code using Answer Set Programming.” In: Proceedings of 10th International Conference on Logic Programming and Nonmonotonic Reasoning (LPNMR’09), Lecture Notes in Computer Science, 5753: 554–559,
Springer . DOI: 10.1007/978-3-642-04238-6_57 - Berdine, J, Cook, B and Ishtiaq, S 2011 “SLAyer: Memory Safety for Systems-Level Code,” In: Proceedings of the 23rd International Conference on Computer Aided Verification (CAV 2011), of Lecture Notes in Computer Science, 6806: 178–183,
Springer . DOI: 10.1007/978-3-642-22110-1_15 - De Roure, D. “Replacing the Paper: The Twelve Rs of the e-Research Record.” Available from:
http://www.scilogs.com/eresearch/replacing-the-paper-the-twelve-rs-of-the-e-research-record/ , November 2011. - Stodden, V, Guo, P and Ma, Z 2013 “Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals,” PLoS ONE, 8(6). DOI: 10.1371/journal.pone.0067111
- Fursin, G and Dubach, C 2014 “Community-Driven Reviewing and Validation of Publications,” In: Proceedings of the 1st ACM SIGPLAN Workshop on Reproducible Research Methodologies and New Publication Models in Computer Engineering (TRUST’14), pp. 1–4,
ACM Press . DOI: 10.1145/2618137.2618142 - National Academies of Sciences, Engineering, and Medicine 2016 Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results: Summary of a Workshop. The National Academies Press.
- Galiani, S, Gertler, P and Romero, M.
“Incentives for Replication in Economics,” Tech. rep., National Bureau of Economic Research, July 2017. NBER Working Paper No. 23576. - Barnes, N 2010 “Publish your computer code: it is good enough,” Nature, 467(753). DOI: 10.1038/467753a
- Morin, A, Urban, J, Adams, P D, Foster, I, Sali, A, Baker, D and Sliz, P 2012 “Shining Light into Black Boxes,” Science, 336(6078): 159–160. DOI: 10.1126/science.1218263
- Joppa, L N, McInerny, G, Harper, R, Salido, L, Takeda, K, O’Hara, K, Gavaghan, D and Emmott, S 2013 “Troubling Trends in Scientific Software Use,” Science, 340(6134): 814–815. DOI: 10.1126/science.1231535
- Bechhofer, S, Buchan, I, De Roure, D, Missier, P, Ainsworth, J, Bhagata, J, Couch, P, Cruickshank, D, Delderfield, M, Dunlop, I, Gamble, M, Michaelides, D, Owen, S, Newman, D, Sufi, S and Goble, C 2013 “Why linked data is not enough for scientists,” Future Generation Computer Systems, 29(2): 599–611. DOI: 10.1016/j.future.2011.08.004
- Osherovich, L 2011 “Hedging against academic risk,” Science-Business eXchange, 4(15).
- Hesman Saey, T 2015 “Repeat Performance: Too many studies, when replicated, fail to pass muster,” Science News, 187(2): 21–26. DOI: 10.1002/scin.2015.187002014
- Goble, C 2014 “Better Software, Better Research,” IEEE Internet Computing, 18(5): 4–8. DOI: 10.1109/MIC.2014.88
- Chirigati, F, Troyer, M, Shasha, D and Freire, J 2013 “A Computational Reproducibility Benchmark,” IEEE Data Engineering Bulletin, 36(4): 54–59.
- Stodden, V and Miguez, S 2014 “Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research,” Journal of Open Research Software, 2(1): 1–6. DOI: 10.5334/jors.ay
- Stodden, V, Miguez, S and Seiler, J 2015 “ResearchCompendia.org: Cyberinfrastructure for Reproducibility and Collaboration in Computational Science,” Computing in Science & Engineering, 17(12). DOI: 10.1109/MCSE.2015.18
- Stodden, V, McNutt, M, Bailey, D H, Deelman, E, Gil, Y, Hanson, B, Heroux, M A, Ioannidis, J P and Taufer, M 2016 “Enhancing reproducibility for computational methods,” Science, 354(6317): 1240–1241. DOI: 10.1126/science.aah6168
- Fomel, S and Claerbout, J F 2008 “Reproducible Research,” Computing in Science & Engineering, 11(1).
- “Reproducible Research” 2010 Computing in Science & Engineering, 12(5): 8–13. DOI: 10.1109/MCSE.2010.113
- Gent, I P “The Recomputation Manifesto.” Available from:
http://arxiv.org/abs/1304.3674 , April 2013. - Fursin, G, Miceli, R, Lokhmotov, A, Gerndt, M, Baboulin, M, Malony, A D, Chamski, Z, Novillo, D and Del Vento, D 2014 “Collective mind: Towards practical and collaborative auto-tuning,” Scientific Programming, 22(4): 309–329. DOI: 10.1155/2014/797348
- Bailey, D, Borwein, J and Stodden, V 2013 “Set the Default to “Open”,” Notices of the AMS.
- James, D, Wilkins-Diehr, N, Stodden, V, Colbry, D, Rosales, C, Fahey, M R, Shi, J, da Silva, R F, Lee, K, Roskies, R, Loewe, L, Lindsey, S, Kooper, R, Barba, L, Bailey, D H, Borwein, J M, Corcho, Ó, Deelman, E, Dietze, M C, Gilbert, B, Harkes, J, Keele, S, Kumar, P, Lee, J, Linke, E, Marciano, R, Marini, L, Mattmann, C, Mattson, D, McHenry, K, McLay, R T, Miguez, S, Minsker, B S, Pérez-Hernández, M S, Ryan, D, Rynge, M, Pérez, I S, Satyanarayanan, M, Clair, G S, Webster, K, Hovig, E, Katz, D S, Kay, S, Sandve, G K, Skinner, D, Allen, G, Cazes, J, Cho, K W, Fonseca, J, Hwang, L, Koesterke, L, Patel, P, Pouchard, L, Seidel, E and Suriarachchi, I 2014
“Standing Together for Reproducibility in Large-Scale Computing: Report on reproducibility@XSEDE,” Tech. rep., XSEDE. - Prlić, A and Procter, J B 2012 “Ten Simple Rules for the Open Development of Scientific Software,” PLoS Computational Biology, 8(12): e1002802. DOI: 10.1371/journal.pcbi.1002802
- Masum, H, Rao, A, Good, B M, Todd, M H, Edwards, A M, Chan, L, Bunin, B A, Su, A I, Thomas, Z and Bourne, P E 2013 “Ten Simple Rules for Cultivating Open Science and Collaborative R&D,” PLoS Computational Biology, 9(9): e1003244. DOI: 10.1371/journal.pcbi.1003244
- Sandve, G, Nekrutenko, A, Taylor, J and Hovig, E 2013 “Ten Simple Rules for Reproducible Computational Research,” PLoS Computational Biology, 9(10): e1003285. DOI: 10.1371/journal.pcbi.1003285
- Osborne, J M, Bernabeu, M O, Bruna, M, Calderhead, B, Cooper, J, Dalchau, N, Dunn, S-J, Fletcher, A G, Freeman, R, Groen, D, Knapp, B, McInerny, G J, Mirams, G R, Pitt-Francis, J, Sengupta, B, Wright, D W, Yates, C A, Gavaghan, D J, Emmott, S and Deane, C 2013 “Ten Simple Rules for Effective Computational Research,” PLoS Computational Biology, 10(3): e1003506. DOI: 10.1371/journal.pcbi.1003506
- Goodman, A, Pepe, A, Blocker, A W, Borgman, C L, Cranmer, K, Crosas, M, Di Stefano, R, Gil, Y, Groth, P, Hedstrom, M, Hogg, D W, Kashyap, V, Mahabal, A, Siemiginowska, A and Slavkovic, A 2014 “Ten Simple Rules for the Care and Feeding of Scientific Data,” PLoS Computational Biology, 10(4): e1003542. DOI: 10.1371/journal.pcbi.1003542
- Chue Hong, N P, Crick, T, Gent, I P, Kotthoff, L and Takeda, K 2015 “Top Tips to Make Your Research Irreproducible.” Available from:
http://arxiv.org/abs/1504.00062 . - List, M, Ebert, P and Albrecht, F 2017 “Ten Simple Rules for Developing Usable Software in Computational Biology,” PLoS Computational Biology, 13(1): e1005265. DOI: 10.1371/journal.pcbi.1005265
- Stodden, V 2008 “The Legal Framework for Reproducible Scientific Research: Licensing and Copyright,” Computing in Science & Engineering, 11(1).
- Haas, C N 2016 “Reproducible Risk Assessment,” Risk Analysis, 6(10): 1829–1833. DOI: 10.1111/risa.12730
- Crick, T, Hall, B A and Ishtiaq, S 2014 ““Can I Implement Your Algorithm?”: A Model for Reproducible Research Software,” In: 2nd International Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2).
- Crick, T, Hall, B A, Ishtiaq, S and Takeda, K 2014 ““Share and Enjoy”: Publishing Useful (and Usable) Scientific Models,” In: Proceedings of the 7th IEEE/ACM International Conference on Utility and Cloud Computing, pp. 957–961.
- Stallman, R M 2010 Free Software Free Society: Selected Essays of Richard M. Stallman. Free Software Foundation.
- Giles, J 2004 “Software company bans competitive users,” Nature, 429(6989). DOI: 10.1038/429231a
- Hess, B, Kutzner, C, van der Spoel, D and Lindahl, E 2008 “GROMACS 4: Algorithms for Highly Efficient, Load-Balanced, and Scalable Molecular Simulation,” Journal of Chemical Theory and Computation, 4(3): 435–447. DOI: 10.1021/ct700301q
- Brooks, B R, Brooks, C L, Mackerell, A D, Nilsson, L, Petrella, R J, Roux, B, Won, Y, Archontis, G, Bartels, C, Boresch, S, Caflisch, A, Caves, L, Cui, Q, Dinner, A R, Feig, M, Fischer, S, Gao, J, Hodoscek, M, Im, W, Kuczera, K, Lazaridis, T, Ma, J, Ovchinnikov, V, Paci, E, Pastor, R W, Post, C B, Pu, J Z, Schaefer, M, Tidor, B, Venable, R M, Woodcock, H L, Wu, X, Yang, W, York, D M and Karplus, M 2009 “CHARMM: The biomolecular simulation program,” Journal of Computational Chemistry, 30(10): 1545–1614. DOI: 10.1002/jcc.21287
- Bowers, K J, Chow, E, Xu, H, Dror, R O, Eastwood, M P, Gregersen, B A, Klepeis, J L, Kolossvary, I, Moraes, M A, Sacerdoti, F D, Salmon, J K, Shan, Y and Shaw, D E 2006 “Scalable algorithms for molecular dynamics simulations on commodity clusters,” In: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing,
IEEE Press . DOI: 10.1109/SC.2006.54 - de Moura, L 2012 “Releasing the Z3 source code.” Available online:
http://leodemoura.github.io/blog/2012/10/02/open-z3.html . - “Open Source Licenses”
http://opensource.org/licenses . - Wilson, G 2006 “Software carpentry: Getting scientists to write better code by making them more productive,” Computing in Science & Engineering, 8(6). DOI: 10.1109/MCSE.2006.122
- Gonthier, G, Ziliani, B, Nanevski, A and Dreyer, D 2013 “How to make ad hoc proof automation less adhoc,” Journal of Functional Programming, 23(4): 357–401. DOI: 10.1017/S0956796813000051
- Miller, G 2006 “A Scientist’s Nightmare: Software Problem Leads to Five Retractions,” Science, 314(5807): 1856–1857. DOI: 10.1126/science.314.5807.1856
- Smith, A M, Katz, D S and Niemeyer, K E and the FORCE11 Software Citation Working Group 2016, “Software Citation Principles,” PeerJ Computer Science, 2(e86).
- Boettiger, C 2015 “An introduction to Docker for reproducible research,” ACM SIGOPS Operating Systems Review, 49(1): 71–79. Special Issue on Repeatability and Sharing of Experimental Artifacts. DOI: 10.1145/2723872.2723882
- Kauffman, S A 1969 “Metabolic stability and epigenesis in randomly constructed genetic nets,” Journal of Theoretical Biology, 22(3): 437–67. DOI: 10.1016/0022-5193(69)90015-0
- Schaub, M A, Henzinger, T A and Fisher, J 2007 “Qualitative networks: a symbolic approach to analyze biological signaling networks,” BMC Systems Biology, 1: 4. DOI: 10.1186/1752-0509-1-4
- Benque, D, Bourton, S, Cockerton, C, Cook, B, Fisher, J, Ishtiaq, S, Piterman, N, Taylor, A and Vardi, M Y 2012 “BMA: visual tool for modeling and analyzing biological networks,” In: Proceedings of the 24th International Conference on Computer Aided Verification (CAV 2012), of Lecture Notes in Computer Science, 7358: 686–692,
Springer . DOI: 10.1007/978-3-642-31424-7_50 - Chaouiya, C, Berenguier, D, Keating, S M, Naldi, A, van Iersel, M P, Rodriguez, N, Drager, A, Buchel, F, Cokelaer, T, Kowal, B, Wicks, B, Goncalves, E, Dorier, J, Page, M, Monteiro, P T, von Kamp, A, Xenarios, I, de Jong, H, Hucka, M, Klamt, S, Thieffry, D, Le Novere, N, Saez-Rodriguez, J and Helikar, T 2013 “SBML qualitative models: a model representation format and infrastructure to foster interactions between qualitative modelling formalisms and tools,” BMC Systems Biology, 7.
- Moult, J, Fidelis, K, Kryshtafovych, A, Schwede, T and Tramontano, A 2014 “Critical assessment of methods of protein structure prediction (CASP) — round x,” Proteins: Structure, Function, and Bioinformatics, 82: 1–6. DOI: 10.1002/prot.24452
- Lensink, M F, Velankar, S and Wodak, S J 2017 “Modeling proteinprotein and proteinpeptide complexes: Capri 6th edition,” Proteins: Structure, Function, and Bioinformatics, 85(3): 359–377.
- Hall, B A, Halim, K B A, Buyan, A, Emmanouil, B and Sansom, M S P 2014 “Sidekick for membrane simulations: Automated ensemble molecular dynamics simulations of transmembrane helices,” Journal of Chemical Theory and Computation, 10(5): 2165–2175. DOI: 10.1021/ct500003g
- Ofria, C and Wilke, C O 2004 “Avida: A Software Platform for Research in Computational Evolutionary Biology,” Artificial Life, 10(2): 191–229. DOI: 10.1162/106454604773563612
- Crick, T, Dunning, P, Kim, H and Padget, J 2009 “Engineering Design Optimization using Services and Workflows,” Philosophical Transactions of the Royal Society A, 367(1898): 2741–2751.
- Olabarriaga, S, Pierantoni, G, Taffoni, G, Sciacca, E, Jaghoori, M, Korkhov, V, Castelli, G, Vuerli, C, Becciani, U, Carley, E and Bentley, B 2014 “Scientific Workflow Management – For Whom?,” in Proceedings of 10th IEEE International Conference on e-Science (e-Science 2014), 298–305,
IEEE Press . DOI: 10.1109/eScience.2014.8 - Hall, B A, Jackson, E, Hajnal, A and Fisher, J 2014 “Logic programming to predict cell fate patterns and retrodict genotypes in organogenesis,” Journal of The Royal Society Interface, 11(98). DOI: 10.1098/rsif.2014.0245
- Rollins, N D, Barton, C M, Bergin, S, Janssen, M A and Lee, A 2014 “A Computational Model Library for publishing model documentation and code,” Environmental Modelling & Software, 61: 59–64. DOI: 10.1016/j.envsoft.2014.06.022
- Vardi, M Y 2014 “Openism, IPism, Fundamentalism, and Pragmatism,” Communications of the ACM, 57(8). DOI: 10.1145/2632265
- Hey, T, Tansley, S and Tolle, K (eds.) 2009 The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research.