A Framework for the Assessment of Research and Its Impacts
References
- Abramovitz, M. (1956). Resource and output trends in the United States since 1870. The American Economic Review, 46(2), 5–23.
- Acemoglu, D., Aghion, P., & Zilibotti, F. (2003). Vertical integration and distance to frontier. Journal of the European Economic Association, 1(2–3), 630–638.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1162/154247603322391260" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1162/154247603322391260</a></pub-id>
- Acemoglu, D., Aghion, P., & Zilibotti, F. (2006). Distance to frontier, selection, and economic growth. Journal of the European Economic Association, 4(1), 37–74.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1162/jeea.2006.4.1.37" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1162/jeea.2006.4.1.37</a></pub-id>
- Adams, J.D. (1990). Fundamental stocks of knowledge and productivity growth. Journal of Political Economy, 98(4), 673–702.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1086/261702" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1086/261702</a></pub-id>
- Agarwal, R., & Dhar, V. (2014). Editorial big data, data science, and analytics: The opportunity and challenge for IS research. Information Systems Research, 25(3), 443–448.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1287/isre.2014.0546" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1287/isre.2014.0546</a></pub-id>
- Aghion, P. (2009). Growth and education. Commission on growth and development working paper, no. 56. Washington, DC: World Bank Publications.
- Aghion, P., & Howitt, P. (2009). The economics of growth. Cambridge, MA: The MIT Press.
- Aghion, P., David, P.A., & Foray, D. (2009). Science, technology and innovation for economic growth: Linking policy research and practice in ‘STIG Systems’. Research Policy, 38(4), 681–693.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.respol.2009.01.016" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.respol.2009.01.016</a></pub-id>
- Anderson C. (2008). The end of theory: The data deluge makes the scientific method obsolete. Wired Magazine. Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.wired.com/2008/06/pb-theory/">https://www.wired.com/2008/06/pb-theory/</ext-link>.
- Antonelli, C., & Link, A.N. (2014). Routledge handbook of the economics of knowledge. London: Routledge.
- Audretsch, D.B., Bozeman, B., Combs, K.L., Feldman, M., Link, A.N., Siegel, D.S., ... & Wessner, C. (2002). The economics of science and technology. The Journal of Technology Transfer, 27(2), 155–203.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1023/A:1014382532639" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1023/A:1014382532639</a></pub-id>
- Bădin, L., Daraio, C., & Simar, L. (2012). How to measure the impact of environmental factors in a nonparametric production model. European Journal of Operational Research, 223(3), 818–833.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.ejor.2012.06.028" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.ejor.2012.06.028</a></pub-id>
- Bădin, L., Daraio, C., & Simar, L. (2014). Explaining inefficiency in nonparametric production models: The state of the art. Annals of Operations Research, 214(1), 5–30.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s10479-012-1173-7" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s10479-012-1173-7</a></pub-id>
- Ballard, B.W. (2000). Understanding MacIntyre. Lanham, MD: University Press of America.
- Bammer, G. (2016). What constitutes appropriate peer review for interdisciplinary research? Palgrave Communications, 2 (palcomms201617). Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.nature.com/articles/palcomms201617">http://www.nature.com/articles/palcomms201617</ext-link>.
- Barré, R. (2004). S&T indicators for policy making in a changing science–society relationship. In H. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research (pp. 115–132). Dordrecht: Springer Netherlands.
- Batini, C., & Scannapieco, M. (2016). Data and information quality. Cham, Switzerland: Springer International Publishing.
- Benessia, A., Funtowicz, S., Giampietro, M., Pereira, Â.G., Ravetz, J., Saltelli, A., ... & van der Sluijs, J.P. (2016). Science on the verge. Tempe, AZ: Consortium for Science, Policy, & Outcomes at Arizona State University.
- Blaug, M. (1966). Economics of education; a selected annotated bibliography (No. 370.193 B5). Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.sciencedirect.com/science/article/pii/B9780080206271500025">http://www.sciencedirect.com/science/article/pii/B9780080206271500025</ext-link>.
- Bogetoft, P., Fried, H.O., & Eeckaut, P.V. (2007). The university benchmarker: An interactive computer approach. In A. Bonaccorsi, & C. Daraio (Eds.), Universities and Strategic Knowledge Creation: Specialization and Performance in Europe (pp. 443–462). Cheltenham: Edward Elgar Publishing.
- Bonaccorsi, A., & Daraio, C. (2004). Econometric approaches to the analysis of productivity of RD systems. Production functions and production frontiers. In H.F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of Quantitative Science and Technology Research (pp. 51–74). Dordrecht: Springer Netherlands.
- Borgman, C.L. (2015). Big data, little data, no data: Scholarship in the networked world. Cambridge, MA: The MIT Press.
- Bornmann, L. (2013). What is societal impact of research and how can it be assessed? A literature survey. Journal of the American Society for Information Science and Technology, 64(2), 217–233.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1002/asi.22803" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1002/asi.22803</a></pub-id>
- Borrás, S., & Edquist, C. (2013). The choice of innovation policy instruments. Technological Forecasting and Social Change, 80(8), 1513–1522.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.techfore.2013.03.002" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.techfore.2013.03.002</a></pub-id>
- Bozeman, B., & Melkers, J. (Eds). (1993). Evaluating R&D impacts: Methods and practice. New York: Springer.
- Brunsson, N., & Jacobsson, B. (Eds.). (2002a). A world of standards. Oxford: Oxford University Press.
- Brunsson, N., & Jacobsson, B. (2002b). The pros and cons of standardization—An epilogue. In A World of Standards (pp. 169–173). Oxford: Oxford University Press.
- Capano, G., Howlett, M., & Ramesh, M. (Eds.). (2015). Varieties of governance. Hampshire, UK: Palgrave Macmillan.
- Carayannis, E.G., & Campbell, D.F. (2009). ‘Mode 3’ and ‘Quadruple Helix’: Toward a 21<sup>st</sup>century fractal innovation ecosystem. International Journal of Technology Management, 46(3–4), 201–234.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1504/IJTM.2009.023374" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1504/IJTM.2009.023374</a></pub-id>
- Checchi, D. (2006). The economics of education: Human capital, family background and inequality. Cambridge: Cambridge University Press.
- Chen, C.M., Delmas, M.A., & Lieberman, M.B. (2015). Production frontier methodologies and efficiency as a performance measure in strategic management research. Strategic Management Journal, 36(1), 19–36.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1002/smj.2199" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1002/smj.2199</a></pub-id>
- Chesbrough, H. (2012). Open innovation: Where we’ve been and where we’re going. Research-Technology Management, 55(4), 20–27.
- Cohen, W.M., & Levinthal, D.A. (1990). Absorptive capacity: A new perspective on learning and innovation. Administrative Science Quarterly, 35(1), 128–152.<pub-id pub-id-type="doi"><a href="https://doi.org/10.2307/2393553" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.2307/2393553</a></pub-id>
- Creswell, J.W., & Clark, V.L.P. (Eds.). (2011). 2<sup>nd</sup> Ed. Designing and conducting mixed methods research. Thousand Oaks, CA: SAGE Publications.
- Cronin, B. (2013). Thinking about data. Journal of the American Society for Information Science and Technology, 64(3), 435–436.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1002/asi.22928" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1002/asi.22928</a></pub-id>
- Cronin, B., & Sugimoto, C.R. (Eds.). (2014). Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact. Cambridge, MA: The MIT Press.
- Cronin, B., & Sugimoto, C.R. (Eds.). (2015). Scholarly metrics under the microscope: From citation analysis to academic auditing. Medford, NJ: Information Today.
- D’Ariano, G.M., & Perinotti, P. (2016). Quantum theory is an information theory. Foundations of Physics, 46(3), 269–281.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s10701-015-9935-0" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s10701-015-9935-0</a></pub-id>
- Dahler-Larsen, P. (2012). The evaluation society. California: Stanford University Press.
- Daraio, C. (2015). Assessing the efficiency, effectiveness and impact of education in the age of big data: Challenges and a way forward. Keynote presentation at Leuven LEER Workshop ‘Efficiency in Education and the Use of Big Data’, November 19–20, 2015, Leuven (Belgium).
- Daraio, C. (2017a). Assessing research and its impacts: The generalized implementation problem and a doubly-conditional performance evaluation model, paper presented at the ISSI 2017 Conference, October 2017, Wuhan (China).
- Daraio, C. (2017b). A doubly conditional performance evaluation model, the democratization of evaluation and altmetrics, paper presented at the STI 2017 Conference, September 2017, Paris.
- Daraio, C. (in press). Econometric approaches to the measurement of research productivity. In W. Glänzel, H.F. Moed, H. Schmoch, & M. Thelwall (Eds.), Springer Handbook of Science and Technology Indicators.
- Daraio, C., & Simar, L. (2014). Directional distances and their robust versions: Computational and testing issues. European Journal of Operational Research, 237(1), 358–369.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.ejor.2014.01.064" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.ejor.2014.01.064</a></pub-id>
- Daraio, C., & Simar, L. (2007). Advanced robust and nonparametric methods in efficiency analysis. Methodology and applications. New York: Springer.
- Daraio, C., Simar, L., & Wilson, P.W. (2017). Central limit theorems for conditional efficiency measures and tests of the “separability” condition in nonparametric two-stage models of production. The Econometrics Journal. Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="<a href="https://doi.org/10.1111/ectj.12103" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">https://doi.org/10.1111/ectj.12103</a>">https://doi.org/10.1111/ectj.12103</ext-link>.
- Daraio, C., & Glänzel, W. (2016). Grand challenges in data integration—state of the art and future perspectives: An introduction. Scientometrics, 108 (1), 391–400.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s11192-016-1914-5" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s11192-016-1914-5</a></pub-id>
- Daraio, C., Lenzerini, M., Leporelli, C., Moed, H.F., Naggar, P., Bonaccorsi, A., & Bartolucci, A. (2016a). Data integration for research and innovation policy: An ontology-based data management approach. Scientometrics, 106(2), 857–871.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s11192-015-1814-0" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s11192-015-1814-0</a></pub-id>
- Daraio, C., Lenzerini, M., Leporelli, C., Naggar, P., Bonaccorsi, A., & Bartolucci, A. (2016b). The advantages of an Ontology-Based Data Management approach: Openness, interoperability and data quality. Scientometrics, 108 (1), 441–455.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s11192-016-1913-6" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s11192-016-1913-6</a></pub-id>
- Debackere, K. (2016). Let the data speak for themselves: Opportunities and caveats. Journal of Data and Information Science, 1(1), 3–5.
- DeMillo, R.A., & Young, A.J. (2015). Revolution in higher education: How a small band of innovators will make college accessible and affordable. Cambridge, MA: The MIT Press.
- Ding, Y., Rousseau, R., & Wolfram, D. (Eds). (2014). Measuring scholarly impact. Cham, Switzerland: Springer International Publishing.
- Ding, Y., & Stirling, K. (2016). Data-driven discovery: A new era of exploiting the literature and data. Journal of Data and Information Science, 1(4), 1–9.<pub-id pub-id-type="doi"><a href="https://doi.org/10.20309/jdis.201622" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.20309/jdis.201622</a></pub-id>
- Drucker, P.F. (1967). The effective executive. New York: Harper and Row.
- Ebrahim, A., & Rangan, V.K. (2014). What impact? California Management Review, 56(3), 118–141.
- Edquist, C. (2001). The systems of innovation approach and innovation policy: An account of the state of the art. In Druid Nelson and Winter Conference 2001 (pp. 12–15). Denmark: Aalborg University.
- Edwards, P.N., Jackson, S.J., Chalmers, M.K., Bowker, G.C., Borgman, C.L., Ribes, D., … & Calvert, S. (2013). Knowledge infrastructures: Intellectual frameworks and research challenges (p. 40). Ann Arbor, MI: University of Michigan. Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://deepblue.lib.umich.edu/handle/2027.42/97552">http://deepblue.lib.umich.edu/handle/2027.42/97552</ext-link>.
- Egghe, L., & Rousseau, R. (1990). Introduction to informetrics: Quantitative methods in library, documentation and information science. Amsterdam: Elsevier.
- Egghe, L. (2005). Power laws in the information production process: Lotkaian informetrics. Amsterdam: Elsevier.
- Ekbia, H., Mattioli, M., Kouper, I., Arave, G., Ghazinejad, A., Bowman, T., … & Sugimoto, C.R. (2015). Big data, bigger dilemmas: A critical review. Journal of the Association for Information Science and Technology, 66(8), 1523–1545<pub-id pub-id-type="doi"><a href="https://doi.org/10.1002/asi.23294" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1002/asi.23294</a></pub-id>
- Etzkowitz, H., & Leydesdorff, L. (2000). The dynamics of innovation: From national systems and mode 2 to a triple helix of university-industry-government relations. Research Policy, 29(2), 109–123.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/S0048-7333(99)00055-4" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/S0048-7333(99)00055-4</a></pub-id>
- European Commission (2014). Expert Group to support the development of tailor-made impact assessment methodologies for ERA (European Research Area), Brussels, Belgium.
- Fagerberg, J., Martin, B.R., & Andersen, E.S. (Eds.). (2013). Innovation studies: Evolution and future challenges. Oxford: Oxford University Press.
- Fealing, K.H., Lane, J.I., Marburger, J.H. JIII, & Shipp, S.S. (Eds.) (2011). The science of science policy, a handbook. Stanford: Stanford University Press.
- Floridi, L. (Ed.). (2008). The Blackwell guide to the philosophy of computing and information. Hoboken, NJ: John Wiley & Sons.
- Floridi, L. (Ed.). (2010). The Cambridge handbook of information and computer ethics. Cambridge: Cambridge University Press.
- Floridi, L. (2012). The road to the philosophy of information. In H. Demir (Eds.), Luciano Floridi’s Philosophy of Technology (pp. 245–271). Dordrecht: Springer Netherlands.
- Floridi, L. (2014). The fourth revolution: How the infosphere is reshaping human reality. Oxford: Oxford University Press.
- Floridi, L., & Illari, P. (Eds.). (2014). The philosophy of information quality (Vol. 358). Cham, Switzerland: Springer International Publishing.
- Frické, M. (2015). Big data and its epistemology. Journal of the Association for Information Science and Technology, 66 (4), 651–661.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1002/asi.23212" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1002/asi.23212</a></pub-id>
- Frischmann, B.M. (2012). Infrastructure: The social value of shared resources. Oxford: Oxford University Press.
- Funtowicz, S.O., & Ravetz, J.R. (1990). Science for policy: Uncertainty and quality. In Uncertainty and Quality in Science for Policy (pp. 7–16). Dordrecht: Springer Netherlands.
- Furner, J. (2014). The ethics of evaluative bibliometrics. In B. Cronin, & C. Sugimoto (Eds.), Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact (pp. 85–107). Cambridge, MA: MIT Press.
- Galán, J.E., Veiga, H., & Wiper, M.P. (2014). Bayesian estimation of inefficiency heterogeneity in stochastic frontier models. Journal of Productivity Analysis, 42(1), 85–101.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s11123-013-0377-4" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s11123-013-0377-4</a></pub-id>
- Georgescu-Roegen, N. (1971). The entropy law and the economic process. Cambridge, MA: Harvard University Press.
- Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: The dynamics of science and research in contemporary societies. London: Sage.
- Gingras, Y. (2016). Bibliometrics and research evaluation: Uses and abuses. Cambridge, MA: The MIT Press.
- Glänzel, W. (1996). The need for standards in bibliometric research and technology. Scientometrics, 35(2), 167–176.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/BF02018475" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/BF02018475</a></pub-id>
- Glänzel, W., & Schoepflin, U. (1994). Little scientometrics, big scientometrics… and beyond? Scientometrics, 30(2–3), 375–384.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/BF02018107" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/BF02018107</a></pub-id>
- Glänzel, W. (2010). On reliability and robustness of scientometrics indicators based on stochastic models. An evidence-based opinion paper. Journal of Informetrics, 4(3), 313–319.
- Glänzel, W., & Moed, H.F. (2013). Opinion paper: Thoughts and facts on bibliometric indicators. Scientometrics, 96(1), 381–394.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s11192-012-0898-z" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s11192-012-0898-z</a></pub-id>
- Glänzel, W., Moed, H.F., Schmoch, H., & Thelwall, M. (Eds). (in press). Springer Handbook of Science and Technology Indicators.
- Godin, B. (2002). Outline for a history of science measurement. Science, Technology, & Human Values, 27(1), 3–27.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1177/016224390202700101" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1177/016224390202700101</a></pub-id>
- Godin, B. (2004). Measurement and statistics on science and technology: 1920 to the present. London: Routledge.
- Griliches, Z. (1986). Economic data issues. In Z. Griliches, & M.D. Intriligator (Eds.), Handbook of Econometrics Volume III (pp. 1465–1514). Amsterdam: Elsevier.
- Griliches, Z. (1994). Productivity, R&D, and the data constraint. American Economic Review, 84(1), 1–23.
- Griliches, Z. (1998). R&D and productivity: The econometric evidence. Chicago: University of Chicago Press.
- Hall, B.H., & Rosenberg, N. (Eds.). (2010). Handbook of the economics of innovation. Amsterdam: Elsevier.
- Hanson, B., Sugden, A., & Alberts, B. (2011). Making data maximally available. Science, 331(6018), 649–649.<pub-id pub-id-type="pmid">21310971</pub-id><pub-id pub-id-type="doi"><a href="https://doi.org/10.1126/science.1203354" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1126/science.1203354</a></pub-id>
- Hanushek, E.A., & Woessmann, L. (2007). The role of education quality for economic growth. World Bank Policy Research Working Paper, No. 4122. Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://ssrn.com/abstract=960379">https://ssrn.com/abstract=960379</ext-link>.
- Hanushek, E.A., Machin, S.J., & Woessmann, L. (Eds.). (2016). Handbook of the economics of education. Amsterdam: Elsevier.
- Hanushek, E.A., Woessmann, L., Jamison, E.A., & Jamison, D.T. (2008). Education and economic growth. Education Next, 8(2). Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.freepatentsonline.com/article/Education-Next/177556508.html">http://www.freepatentsonline.com/article/Education-Next/177556508.html</ext-link>.
- Helbing, D., & Carbone, A.F. (Eds.). (2012). Participatory science and computing for our complex world. The European Physical Journal. Special Topics, Vol. 214. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://epjst.epj.org/index.php?option=com_toc&url=/articles/epjst/abs/2012/14/contents/contents.html">https://epjst.epj.org/index.php?option=com_toc&url=/articles/epjst/abs/2012/14/contents/contents.html</ext-link>.
- Hemlin, S. (1996). Research on research evaluation. Social Epistemology, 10(2), 209–250.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1080/02691729608578815" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1080/02691729608578815</a></pub-id>
- Henderson, R., Jaffe, A.B., & Trajtenberg, M. (1998). Universities as a source of commercial technology: A detailed analysis of university patenting, 196588. Review of Economics and Statistics, 80(1), 119–127.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1162/003465398557221" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1162/003465398557221</a></pub-id>
- Hicks, D., Wouters, P., Waltman, L., De Rijcke, S., & Rafols, I. (2015). Bibliometrics: The Leiden Manifesto for research metrics. Nature, 520, 429–431.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1038/520429a" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1038/520429a</a></pub-id><pub-id pub-id-type="pmid">25903611</pub-id>
- Hill, S. (2016). Assessing (for) impact: Future assessment of the societal impact of research. Palgrave Communications. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.nature.com/articles/palcomms201673">http://www.nature.com/articles/palcomms201673</ext-link>.
- Hinrichs-Krapels, S., & Grant, J. (2016). Exploring the effectiveness, efficiency and equity (3es) of research and research impact assessment. Palgrave Communications. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.nature.com/articles/palcomms201690#t2">https://www.nature.com/articles/palcomms201690#t2</ext-link>.
- Horstemeyer, M.F. (2009). Multiscale modeling: A review. In J. Leszczynski, & M. Shukla (Eds.), Practical Aspects of Computational Chemistry (pp. 87–135). Dordrecht: Springer Netherlands.
- ISCED. (2011), International Standard Classification of Education, UNESCO Montreal, Canada. Retrieved on December 20, 2016, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.uis.unesco.org/Education/Documents/isced-2011-en.pdf">http://www.uis.unesco.org/Education/Documents/isced-2011-en.pdf</ext-link>.
- Johnes, G., & Johnes, J. (Eds.). (2004). International handbook on the economics of education. Cheltenham: Edward Elgar.
- Khandker, S.R., Koolwal, G.B., & Samad, H.A. (2010). Handbook on impact evaluation: Quantitative methods and practices. Washington, DC: World Bank Publications.
- Kitchin, R. (2014). Big data, new epistemologies and paradigm shifts. Big Data & Society, 1(1), 1–12.
- Kuhn, T.S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press.
- Kuhn, T.S. (1969). The structure of scientific revolutions. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://projektintegracija.pravo.hr/_download/repository/Kuhn_Structure_of_Scientific_Revolutions.pdf/">http://projektintegracija.pravo.hr/_download/repository/Kuhn_Structure_of_Scientific_Revolutions.pdf/</ext-link>.
- Largent, M.A., & Lane, J.I. (2012). STAR METRICS and the science of science policy. Review of Policy Research, 29(3), 431–438.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1111/j.1541-1338.2012.00567.x" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1111/j.1541-1338.2012.00567.x</a></pub-id>
- Lenzerini, M. (2011). Ontology-based data management. CIKM, Proceedings of the 20<sup>th</sup> ACM International Conference on Information and Knowledge Management (pp. 5–6). New York: ACM.
- Leydesdorff, L. (2012). The triple helix, quadruple helix, and an N-tuple of helices: Explanatory models for analysing the knowledge-based economy? Journal of the Knowledge Economy, 3(1), 25–35.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s13132-011-0049-4" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s13132-011-0049-4</a></pub-id>
- Lutz, C.S. (2017). Alasdair Chalmers MacIntyre. In The Internet Encyclopedia of Philosophy. Retrieved on January 7, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.iep.utm.edu/mac-over/">http://www.iep.utm.edu/mac-over/</ext-link>.
- MacIntyre, A. (2006). The end of education: The fragmentation of the American university. Commonweal, 133(18), 10–14.
- MacIntyre, A. (2007). After virtue: A study in moral theory. 3<sup>rd</sup> ed. Notre Dame, Indiana: University of Notre Dame Press.
- MacIntyre, A. (1988). Whose justice? Which rationality? Notre Dame, Indiana: University of Notre Dame Press.
- Mansfield, E. (1991). Academic research and industrial innovation. Research Policy, 20(1), 1–12.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/0048-7333(91)90080-A" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/0048-7333(91)90080-A</a></pub-id>
- Mansfield, E. (1995). Academic research underlying industrial innovations: Sources, characteristics, and financing. The Review of Economics and Statistics, 77(1), 55–65.<pub-id pub-id-type="doi"><a href="https://doi.org/10.2307/2109992" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.2307/2109992</a></pub-id>
- Martin, B.R. (2016). Introduction to discussion paper on the sciences are different and the differences are important. Research Policy, 9(45), 1691.
- Mingers, J. (2006). Realising systems thinking: Knowledge and action in management science. Boston, MA: Springer.
- Mirowski, P., & Sent, E.M. (2002). Science bought and sold: Essays in the economics of science. Chicago: University of Chicago Press.
- Moed, H.F. (2016). Altmetrics as traces of the computerization of the research process. In C.R. Sugimoto (Ed.), Theories of Informetrics and Scholarly Communication. A Festschrift in Honor of Blaise Cronin (pp. 360–371). Berlin: De Gruyter.
- Moed, H.F., & van Leeuwen, T.N. (1996). Impact factors can mislead. Nature, 381 (6579), 186.<pub-id pub-id-type="pmid">8622752</pub-id><pub-id pub-id-type="doi"><a href="https://doi.org/10.1038/381186a0" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1038/381186a0</a></pub-id>
- Moed, H.F., Glänzel, W., & Schmoch, U. (2004). Handbook of quantitative science and technology research. Dordrecht: Springer Netherlands.
- Moore, S., Neylon, C., Eve, M.P., O’Donnell, D., & Pattinson, D. (2017). “Excellence R Us”: University research and the fetishisation of excellence. Palgrave Communications. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://www.nature.com/articles/palcomms2016105">https://www.nature.com/articles/palcomms2016105</ext-link>.
- Moreau, L., Freire, J., Futrelle, J., McGrath, R.E., Myers, J., & Paulson, P. (2008). The open provenance model: An overview. In International Provenance and Annotation Workshop (pp. 323–326). Berlin: Springer.
- Morris, W.T. (1967). On the art of modeling. Management Science, 13(12), B707 – B717.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1287/mnsc.13.12.B707" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1287/mnsc.13.12.B707</a></pub-id>
- Munafò, M.R., Nosek, B.A., Bishop, D.V., Button, K.S., Chambers, C.D., du Sert, N.P., … & Ioannidis, J.P. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021.
- Myung, I.J. (2000). The importance of complexity in model selection. Journal of mathematical psychology, 44(1), 190–204.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1006/jmps.1999.1283" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1006/jmps.1999.1283</a></pub-id><pub-id pub-id-type="pmid">10733864</pub-id>
- National Research Council. (2014). Science of science and innovation policy: Principal investigators’ conference summary. Washington, DC: The National Academies Press.
- Nelson, R.R., & Phelps, E.S. (1966). Investment in humans, technological diffusion, and economic growth. The American Economic Review, 56(1/2), 69–75.
- Nielsen, M. (2012). Reinventing discovery: The new era of networked science. Princeton: Princeton University Press.
- Nielsen, M.A., & Chuang, I.L. (2010). Quantum computation and quantum information. 10<sup>th</sup> Anniversary Edition. Cambridge: Cambridge University Press.
- Nowotny, H., Scott, P., & Gibbons, M. (2001). Re-thinking science: Knowledge and the public in an age of uncertainty. Cambridge: Polity.
- O’Donnell, C.J. (2016). Using information about technologies, markets and firm behaviour to decompose a proper productivity index. Journal of Econometrics, 190(2), 328–340.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.jeconom.2015.06.009" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.jeconom.2015.06.009</a></pub-id>
- OECD. (2002). Frascati Manual: Proposed standard practice for surveys on research and experimental development. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.tubitak.gov.tr/tubitak_content_files/BTYPD/kilavuzlar/Frascati.pdf">http://www.tubitak.gov.tr/tubitak_content_files/BTYPD/kilavuzlar/Frascati.pdf</ext-link>.
- OECD. (2005). Oslo Manual: Guidelines for collecting and interpreting innovation data. 3<sup>rd</sup> edition. Paris: OECD Publishing.
- OECD. (2011). Quality framework and guidelines for OECD statistical activities. Paris: OECD Publishing.
- OECD (2015a). Making open science a reality. OECD Science, Technology and Industry Policy Papers No. 25. Paris: OECD Publishing.
- OECD. (2015b). Frascati Manual 2015: Guidelines for collecting and reporting data on research and experimental development. Retrieved on July 30, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.oecd.org/science/frascati-manual-2015-9789264239012-en.htm">http://www.oecd.org/science/frascati-manual-2015-9789264239012-en.htm</ext-link>.
- Owen, R., Macnaghten, P., & Stilgoe, J. (2012). Responsible research and innovation: From science in society to science for society, with society. Science and Public Policy, 39(6), 751–760.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1093/scipol/scs093" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1093/scipol/scs093</a></pub-id>
- Parent, C., & Spaccapietra, S. (2000) Database integration: The key to data interoperability. In M.P. Papazoglou, & Z. Zari (Eds.), Advances in Object-Oriented Data Modeling (pp. 221–253). Cambridge, MA: The MIT Press.
- Parmeter, C.F., Wan, A.T., & Zhang, X. (2016). A model averaging stochastic frontier estimator, paper presented at the NAPW 2016 Quebec City, Canada, June 2016.
- Perkmann, M., Tartari, V., McKelvey, M., Autio, E., Brostrom, A., D’Este, P., … & Krabel, S. (2013). Academic engagement and commercialisation: A review of the literature on university-industry relations. Research Policy, 42(2), 423–442.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.respol.2012.09.007" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.respol.2012.09.007</a></pub-id>
- Pollock, S.M. (1976). Mathematical modeling: Applying the principles of the art studio. Engineering Education, 67(2), 167–171.
- Romer, P.M. (1986). Increasing returns and long-run growth. The Journal of Political Economy, 94(5), 1002–1037.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1086/261420" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1086/261420</a></pub-id>
- Romer, P.M. (1994). The origins of endogenous growth. The Journal of Economic Perspectives, 8(1), 3–22.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1257/jep.8.1.3" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1257/jep.8.1.3</a></pub-id>
- Roper, C.D., & Hirth, M.A. (2005). A history of change in the third mission of higher education: The evolution of one-way service to interactive engagement. Journal of Higher Education Outreach and Engagement, 10(3), 3–21.
- Rosenberg, N., & Nelson, R.R., (1994). American universities and technical advance in industry. Research Policy, 23(3), 323–348.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/0048-7333(94)90042-6" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/0048-7333(94)90042-6</a></pub-id>
- Saltelli, A., & Funtowicz, S. (2014). When all models are wrong. Issues in Science and Technology, 30(2), 79–85.
- Saltelli, A., & Funtowicz, S. (2015). Evidence-based policy at the end of the Cartesian dream: The case of mathematical modelling. In G. Pereira, & S. Funtowicz (Eds.), Science, Philosophy and Sustainability: The End of the Cartesian Dream. Beyond the Techno–Scientific Worldview. Routledge’s Series: Explorations in Sustainability and Governance (pp. 147–162). London: Routledge.
- Saltelli, A., Guimaraes Pereira, A., van der Sluijs, J.P., & Funtowicz, S. (2013). What do I make of your latinorum? Sensitivity auditing of mathematical modelling. International Journal of Foresight and Innovation Policy, 9(2–3–4), 213–234.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1504/IJFIP.2013.058610" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1504/IJFIP.2013.058610</a></pub-id>
- Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., … & Tarantola, S. (2008). Global sensitivity analysis: The primer. Chichester, UK: John Wiley & Sons.
- Salter, A.J., & Martin, B.R. (2001). The economic benefits of publicly funded basic research: A critical review. Research Policy, 30(3), 509–532.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/S0048-7333(00)00091-3" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/S0048-7333(00)00091-3</a></pub-id>
- Scharnhorst, A., Borner, K., van den Besselaar, P. (Eds.). (2012). Models of science dynamics: Encounters between complexity theory and information sciences. Berlin: Springer.
- Simar, L., & Wilson, P.W. (2015). Statistical approaches for nonparametric frontier models: A guided tour. International Statistical Review, 83(1), 77–110.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1111/insr.12056" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1111/insr.12056</a></pub-id>
- Solow, R.M. (1957). Technical change and the aggregate production function. The Review of Economics and Statistics, 39(3): 554–562.
- Stephan, P.E. (2012). How economics shapes science. Cambridge, MA: Harvard University Press.
- Stilgoe, J., Owen, R., & Macnaghten, P. (2013). Developing a framework for responsible innovation. Research Policy, 42(9), 1568–1580.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.respol.2013.05.008" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.respol.2013.05.008</a></pub-id>
- Teixeira, P.N., & Dill, D.D. (Eds.). (2011). Public vices, private virtues? Assessing the effects of marketization in higher education (Vol. 2). Rotterdam: Sense Publishers.
- Teixeira, P., Jongbloed, B., Dill, D., & Amaral, A. (2004). Markets in higher education: Rhetoric or reality? Dordrecht: Kluwer.
- Vandenbussche, J., Aghion, P., & Meghir, C. (2006). Growth, distance to frontier and composition of human capital. Journal of Economic Growth, 11(2), 97–127.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1007/s10887-006-9002-y" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1007/s10887-006-9002-y</a></pub-id>
- Veugelers, R., & Del Rey, E. (2014). The contribution of universities to innovation, (regional) growth and employment. EENEE Analytical Report. Munich, Germany: EENEE. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://www.voced.edu.au/node/82516">http://www.voced.edu.au/node/82516</ext-link>.
- Vinkler, P. (2010). The evaluation of research by scientometric indicators. Cambridge: Chandos Publishing.
- von Hippel, E. (2005). Democratizing innovation. Cambridge, MA: The MIT Press.
- von Hippel, E. (2016). Free innovation. Cambridge, MA: The MIT Press.
- Weinan, E. (2011). Principles of multiscale modeling. Cambridge: Cambridge University Press.
- West, J., Salter, A., Vanhaverbeke, W., & Chesbrough, H. (2014). Open innovation: The next decade. Research Policy, 43(5), 805–811.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1016/j.respol.2014.03.001" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1016/j.respol.2014.03.001</a></pub-id>
- Whitley, R., & Gläser, J. (Eds.). (2007). The changing governance of the sciences: The advent of research evaluation systems. Dordrecht: Springer Netherlands.
- Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., … & Johnson, B. (2015). The metric tide: Report of the independent review of the role of metrics in research assessment and management. Retrieved on July 31, 2017, from <ext-link ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="http://dera.ioe.ac.uk/23424/">http://dera.ioe.ac.uk/23424/</ext-link>.
- Willemain, T.R. (1994). Insights on modeling from a dozen experts. Operations Research, 42(2), 213–222.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1287/opre.42.2.213" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1287/opre.42.2.213</a></pub-id>
- Zucchini, W. (2000). An introduction to model selection. Journal of Mathematical Psychology, 44(1), 41–61.<pub-id pub-id-type="doi"><a href="https://doi.org/10.1006/jmps.1999.1276" target="_blank" rel="noopener noreferrer" class="text-signal-blue hover:underline">10.1006/jmps.1999.1276</a></pub-id><pub-id pub-id-type="pmid">10733857</pub-id>
Language: English
Page range: 7 - 42
Submitted on: Jul 30, 2017
Accepted on: Sep 6, 2017
Published on: Dec 29, 2017
Published by: Chinese Academy of Sciences, National Science Library
In partnership with: Paradigm Publishing Services
Publication frequency: 4 times per year
Keywords:
Related subjects:
© 2017 Cinzia Daraio, published by Chinese Academy of Sciences, National Science Library
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License.