A Potential Framework for Integration of Architecture and Methodology to Improve Statistical Production Systems

John L. Eltinge; Paul P. Biemer; Anders Holmberg

doi:10.2478/jos-2013-0007

.blurhash-client-img { display: none !important; }

A Potential Framework for Integration of Architecture and Methodology to Improve Statistical Production Systems

Journal of Official Statistics

Volume 29 (2013): Issue 1 (March 2013)

By: John L. Eltinge, Paul P. Biemer and Anders Holmberg

Open Access

|Oct 2013

Abstract

This article outlines a framework for formal description, justification and evaluation in development of architectures for large-scale statistical production systems. Following an introduction of the main components of the framework, we consider four related issues: (1) Use of some simple schematic models for survey quality, cost, risk, and stakeholder utility to outline several groups of questions that may inform decisions on system design and architecture. (2) Integration of system architecture with models for total survey quality (TSQ) and adaptive total design (ATD). (3) Possible use of concepts from the Generic Statistical Business Process Model (GSBPM) and the Generic Statistical Information Model (GSIM). (4) The role of governance processes in the practical implementation of these ideas.

References

Australian Bureau of Statistics (2012). International Collaboration Effort - Statistical Network Progress Report. Paper presented by Brian Pink at the meeting of Directors General of participating countries Australia, New Zealand, Norway, Sweden, Canada, UK in New York, February 2012.
Search in Google Scholar
Biemer, P.P. (2010). Total Survey Error: Design, Implementation and Evaluation. Public Opinion Quarterly, 74, 817-848.10.1093/poq/nfq058
Open DOI Search in Google Scholar
Bisgaard, S. and Pinho, A. (2003). Follow-Up Experiments to Verify Dispersion Effects: Taguchi’s Welding Experiment. Quality Engineering, 16, 335-343.10.1081/QEN-120024023
Open DOI Search in Google Scholar
Borowik, J., Henden, M., and Fraser, B. (2012). Riding the Big Data Wave to Streamline Acquiring and Managing Data. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
Search in Google Scholar
Box, George E.P. (1957). Evolutionary Operation: A Method for Increasing Industrial Productivity, Journal of the Royal Statistical Society. Series C (Applied Statistics). Vol. 6, No. 2 (June 1957), 81-101.
Search in Google Scholar
Box, G.E.P. and Draper, N.R. (1969). Evolutionary Operation. New York: Wiley.
Search in Google Scholar
Braaksma, B. (2009). Redesigning a Statistical Institution: the Dutch Case. In Proceedings of Modernisation of Statistics Production. International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/H_2_business_models_Braaksma_final.pdf. (accessed February 12, 2013).
Search in Google Scholar
Brackstone, G. (1999). Managing Data Quality in a Statistical Agency. Survey Methodology, 25, 139-149.
Search in Google Scholar
Brenneman, W.A. and Nair, V.N. (2001). Methods for Identifying Dispersion Effects in Unreplicated Factorial Experiments: A Critical Analysis and Proposed Strategies. Technometrics, 43, 388-405.10.1198/00401700152672483
Open DOI Search in Google Scholar
Calinescua, M., Bhulaia, S., and Schouten, B. (2013). Optimal Resource Allocation in Survey Designs. European Journal of Operational Research, 21, 115-121.10.1016/j.ejor.2012.10.046
Open DOI Search in Google Scholar
Camstra, A. and Renssen, R. (2011). Standard Process Steps Based on Standard Methods as a Part of the Business Architecture. 58th Congress of the International Statistical Institute. Dublin, 21-26 August 2011.
Search in Google Scholar
Carly-Baxter, L., Mitchell, S., and Peytchev, A. (2011). Developing and Implementing Adaptive Total Design. Presented at the 2011 International Field Directors and Technologies Conference (IFD&TC), Scottsdale, AZ. Available at: http://www.ifdtc.org/PC2011/presentation_2011_files/3A-Lisa%20Carley-Baxter.pdf. (accessed February 12, 2013).
Search in Google Scholar
Chief Information Officers Council CIOC (1999). Federal Enterprise Architecture Framework. Version 1.1, Chief Information Officers Council, September 1999.
Search in Google Scholar
Chief Information Officers Council CIOC (2001). A Practical Guide to Federal Enterprise Architecture. Version 1.0, Chief Information Officers Council, February 2001.
Search in Google Scholar
Couper, M. (1998). Measuring Survey Quality in a CASIC Environment. In: Proceedings of the Section on Survey Research Methods, American Statistical Association.
Search in Google Scholar
Cunningham-Hunter, D., Mitchell, S., Carley-Baxter, L., and Keating, M. (2012). Using Paradata to Monitor Survey Production, Cost, and Quality within an Adaptive Total Design Framework. Paper presented at the 2012 Federal Conference on Survey Methodology, Washington, DC. Available at: http://www.fcsm.gov/12papers/Hunter_2012FCSM_V-C.pdf. (accessed February 12, 2013).
Search in Google Scholar
Dunnet, G. (2007). The BmTS: Creating a new business model for a national statistical office of the 21st century. Invited paper at the UNECE/Eurostat /OECD Meeting on the Management of Statistical Information Systems (MSIS). Geneva, 8-10 May 2007. Available at: http://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.50/2007/mtg1/wp.11.e.pdf. (accessed February 13, 2013).
Search in Google Scholar
Eltinge, J.L. (2012). Doing More with Less: Balancing Survey Cost, Quality, Risk and Stakeholder Utility. Presentation at the Fourth International Conference on Establishment Surveys, Montreal, Canada.
Search in Google Scholar
Eltinge, J.L. and Phipps, P.A. (2009). Characterization, Evaluation and Management of Prospective Benefits, Costs and Risks in the Development of New Statistical Programs for Energy. Proceedings of the Section on Survey Research Methods, Joint Statistical Meetings.
Search in Google Scholar
European Commission (2009). Communication from the Commission to the European Parliament and the Council on the production method of EU statistics: a vision for the next decade, Document COM(2009)404. Available at: http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uriCOM:2009:0404:FIN:EN:PDF. (accessed February 12, 2013).
Search in Google Scholar
Field, S. (2009). From Red to Green: The role of Enterprise Architecture in the ONS Corporate IT Strategy. UNECE/Eurostat/OECD Meeting on the Management of Statistical Information Systems (MSIS 2009). Oslo, Norway, 18-20 May 2009.
Search in Google Scholar
Finselbach, H. (2011). Implementing Standardised Systems, Processes, Tools and Methods with the ONS Design Charter. 58th Congress of the International Statistical Institute. Dublin, 21-26 August 2011.
Search in Google Scholar
Gloersen, R. and Saeboe, V. (2009). Standardisation for Improvements in Statistics Norway. In Proceedings of Modernisation of Statistics Production International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/H_1_business_models_Gloersen.pdf. (accessed February 12, 2013).
Search in Google Scholar
Groves, R.M. (1989). Survey Errors and Survey Costs. New York: John Wiley & Sons.10.1002/0471725277
Open DOI Search in Google Scholar
Groves, R.M. and Heeringa, S. (2006). Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs. Journal of the Royal Statistical Society Series A: Statistics in Society, 169(Part 3), 439-457.10.1111/j.1467-985X.2006.00423.x
Open DOI Search in Google Scholar
Groves, R.M. and Lyberg, L. (2010). Total Survey Error: Past, Present and Future. Public Opinion Quarterly, 74, 849-879.10.1093/poq/nfq065
Open DOI Search in Google Scholar
Hahn, G.J. (1982). Statistical Assessment of a Process Change. Journal of Quality Technology, 14, 1-9.10.1080/00224065.1982.11978778
Open DOI Search in Google Scholar
Hall, J.M. and Johnson, M.E. (2009). When Should a Process be Art, not Science. Harvard Business Review, March 2009 issue, 58-64. Hamilton, A. and Tam, S.-M. (2012a). Toward GSIM V1.0 as a Cornerstone for Common Reference Architecture. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
Search in Google Scholar
Hamilton, A. and Tam, S.-M. (2012b). Platform for International Collaboration - Preliminary Thoughts From the ABS. Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
Search in Google Scholar
Jug, M. (2009). Uptake of Service Oriented Architecture in Statistics - Are We Really So Different? In Proceedings of Modernisation of Statistics Production International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/C_1_SOA_Jug_final.pdf. (accessed February 12, 2013).
Search in Google Scholar
Kreuter, F., Olson, K., Wagner, J., Yan, T., Ezzati-Rice, T.M., Casas-Cordero, C., Lemay, M., Peytchev, A., Groves, R.M., and Raghunathan, T.E. (2010). Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Nonresponse: Examples from Multiple Surveys. Journal of the Royal Statistical Society, Series A, 173, 389-407.10.1111/j.1467-985X.2009.00621.x
Open DOI Search in Google Scholar
Laflamme, F., Pasture, T., Talon, J., Maydan, M., and Miller, A. (2008). Using Paradata to Actively Manage Data Collection, Proceedings of the ASA Survey Methods Research Section, Denver, Colorado.
Search in Google Scholar
Linacre, S. (2011). Standardising Methods and Tools: A Look Back Over 30-Plus Years of Experience. Paper presented at Statistics Canada Symposium 2011.
Search in Google Scholar
Lyberg, L. (2012). Survey Quality. Waksberg Memorial Lecture delivered at the Fourth International Conference on Establishment Surveys, 14 June 2012.
Search in Google Scholar
MIT (2004). Engineering Systems Division Architecture Committee: The Influence of Architecture in Engineering Systems. Paper presented at the Engineering Systems Symposium, MIT, Cambridge MA, USA, 29-31 March 2004. Available at: http://esd.mit.edu/symposium/monograph/default.htm. (accessed February 12, 2013).
Search in Google Scholar
Munoz-Lopez, J. (2012). Value of Systems Architecture for Statistical Organizations.
Search in Google Scholar
Meeting on the Management of Statistical Information Systems (MSIS 2012). Washington, DC, 21-23 May 2012.
Search in Google Scholar
OMB (2009). Improving Agency Performance Using Information and Information Technology EA Assessment Framework v3.1, Office of Management and Budget. June 2009.
Search in Google Scholar
Penneck, S. (2009). The Office for National Statistics (ONS) Statistical Modernisation Programme: What Went Right? What Went Wrong? In Proceedings of Modernisation of Statistics Production, International conference organised by Statistics Sweden, Stockholm, 2-4 November 2009. Available at: http://www.scb.se/Grupp/Produkter_Tjanster/Kurser/ModernisationWorkshop/final_papers/D_1_management_ Penneck_final.pdf. (accessed February 12, 2013).
Search in Google Scholar
Pink, B., Borowik, J., and Lee, G. (2010). The Case for an International Statistical Innovation Program -Transforming National and International Statistics Systems. Statistical Journal of the International Association for Official Statistics (IAOS), 26, Number 3-4. Available at: http://iospress.metapress.com/content/2h5764574t6318r4/fulltext.html. (accessed February 12, 2013).
Search in Google Scholar
Renssen, R. and Van Delden, A. (2008). Standardisation of Design and Production of Statistics; A Service Oriented Approach at Statistics Netherlands. Paper presented at the IAOS Congress Reshaping Official Statistics, Shanghai, October 2008.
Search in Google Scholar
Renssen, R., Wings, J., and Paulussen, R. (2009). Processes, Methods and Tools. Statistics Netherlands internal report. Available at: http://www.cbs.nl/NR/rdonlyres/8EFDD386-0FD9-4567-90BC-6ADAB5C67B3D/0/2009Processesmethodsandtoolsart.pdf. (accessed February 12, 2013.).
Search in Google Scholar
Rozanski, N. and Woods, E. (2005). Software Systems Architecture. New Jersey, United States of America: Addison Wesley.
Search in Google Scholar
Schouten, B., Calinescu, M., and Luiten, A. (2011). Optimizing Quality of Response Through Adaptive Survey Designs, Statistics Netherlands, Available at: http://www.cbs.nl/NR/rdonlyres/2D62BF4A-6783-4AC4-8E45-12EF20C6675C/0/2011x1018.pdf. (accessed February 12, 2013).
Search in Google Scholar
Sowa, J.F. and Zachman, J. (1992). Extending and Formalizing the Framework for Information Systems Architecture, IBM Systems Journal, 31.10.1147/sj.313.0590
Open DOI Search in Google Scholar
Statistics Canada (2012). Corporate Business Plan - Statistics Canada - 2012-13 to 2014-15. Available at: http://www.statcan.gc.ca/about-apercu/business-plan-affaires_2012-2015-eng.htm. (accessed February 12, 2013).
Search in Google Scholar
Studman, B. (2010). A Collaborative Development Approach to Agile Statistical Processing Architecture - Australian Bureau of Statistics (ABS) Experience and Aspirations. UNECE/Eurostat/OECD Meeting on the Management of Statistical Information Systems (MSIS 2010). Daejeon, Republic of Korea, 26-29 April 2010.
Search in Google Scholar
Sundgren, B. (2007). Process Reengineering at Statistics Sweden. From UNECE/Eurostat /OECD Meeting on the Management of Statistical Information Systems (MSIS). Geneva, 8-10 May 2007. Available at: http://www.unece.org/fileadmin/DAM/stats/documents/ece/ces/ge.50/2007/mtg1/wp.22.e.pdf. (accessed February 13, 2013).
Search in Google Scholar
Sundgren, B. (2010). The Systems Approach to Official Statistics. In Official Statistics Methodology and Applications in Honour of Daniel Thorburn. Stockholm University, Stockholm. 225-260. Available at: http://officialstatistics.files.wordpress.com/2010/05/bok18.pdf. (accessed February 12, 2013).
Search in Google Scholar
Sundgren, B. (2011). Towards a System of Official Statistics Based on a Coherent Combination of Data Sources, Including Surveys and Administrative Data. Available at: https://sites.google.com/site/bosundgren/my-life/Towardsasystemofofficialstatisticsbasedonacoherentcombinationofdatasources.doc?attredirects¼0. (accessed February 12, 2013).
Search in Google Scholar
The Open Group (2009). The Open Group Architecture Framework (TOGAF), Version 9.
Search in Google Scholar
The Open Group: San Francisco, CA. Available from: http://www.togaf.info/. (accessed February 12, 2013).
Search in Google Scholar
Todorov, V. (2011). Cooperation Models for Software Development. Meeting on the Management of Statistical Information Systems (MSIS 2011). Luxembourg, 23-25 May 2011.
Search in Google Scholar
United Nations Economic Commission for Europe, UNECE (2011). Strategic Vision of the High-level Group for Strategic Developments in Business Architecture in Statistics. ECE/CES/2011/1. Economic Commission for Europe, Conference for European Statisticians, Geneva, 14-16 June 2011. United Nations Economic Commission for Europe, UNECE (2012a). Strategy to implement the vision of the High-level Group for Strategic Developments in Business Architecture in Statistics. ECE/CES/2012/10. Economic Commission for Europe, Conference for European Statisticians, Paris 6-8 June 2012.
Search in Google Scholar
United Nations Economic Commission for Europe, UNECE (2012b). The Generic Statistical Information Model (GSIM): Version 0.8. Available at: http://www1.unece.org/stat/platform/display/metis/GSIMþv0.8. (accessed February 12, 2013).
Search in Google Scholar
Vaccari, C. (2009). Common Reference Architecture (CORA) ESSnet: history and next activities, paper presented at the Meeting on the Management of Statistical Information Systems (MSIS 2009). Oslo, Norway, 18-20 May 2009. Available at: http://www.unece.org/stats/documents/2009.05.msis.htm. (accessed February 12, 2013).
Search in Google Scholar
Vale, S. (2009). The Generic Statistical Business Process Model, Version 4.0. Available at: http://www1.unece.org/stat/platform/display/metis/TheþGenericþStatisticalþBusinessþProcessþModel. (accessed February 12, 2013).
Search in Google Scholar
Wagner, J. (2008). Adaptive Survey Design to Reduce Nonresponse Bias. Unpublished Ph.D. dissertation, University of Michigan, Ann Arbor, MI.
Search in Google Scholar