Protocol Analysis in Engineering Design Education Research: Observations, Limitations, and Opportunities
By: Gregory Litster and Ada Hurst
References
- Adams, R., Punnakanta, P., Atman, C. J., & Lewis, C. D. (2002). Comparing design team self-reports with actual performance: Cross-validating assessment instruments. In American Society of Engineering Education Annual Conference Proceedings, Montreal, QC. 5783–5798. DOI: 10.18260/1-2--10043
- Adams, R. S., Turns, J., & Atman, C. J. (2003). Educating effective engineering designers: The role of reflective practice. Design Studies, 24(3), 275–294. DOI: 10.1016/S0142-694X(02)00056-X
- Akinci-Ceylan, S., Cetin, K. S., Fleming, R., Ahn, B., Surovek, A. E., Cetin, B., & Taylor, P. (2018). Bridging the gap between academia and industry in approaches for solving ill-structured problems: Problem formulation and protocol development. In American Society of Engineering Education Annual Conference Proceedings, Tampa, FL. DOI: 10.18260/1-2--30157
- Atman, C. J. (2019). Design timelines: Concrete and sticky representations of design process expertise. Design Studies, 65, 125–151. DOI: 10.1016/j.destud.2019.10.004
- Atman, C. J., & Bursic, K. M. (1996). Teaching engineering design: Can reading a textbook make a difference? Research in Engineering Design, 8(4), 240–250. DOI: 10.1007/BF01597230
- Atman, C. J., Bursic, K. M., & Lozito, S. L. (1996). An application of protocol analysis to the engineering design process. In American Society of Engineering Education Annual Conference Proceedings, Washington, DC. 443–452. DOI: 10.18260/1-2--5888
- Atman, C. J., & Bursic, K. M. (1998). Verbal Protocol Analysis as a Method to Document Engineering Student Design Processes. Journal of Engineering Education, 87(2), 121–132. DOI: 10.1002/j.2168-9830.1998.tb00332.x
- Atman, C. J., Cardella, M. E., Turns, J., & Adams, R. (2005). Comparing freshman and senior engineering design processes: An in-depth follow-up study. Design Studies, 26(4), 325–357. DOI: 10.1016/j.destud.2004.09.005
- Bailey, R., & McFarland, M. E. (2018). Prototyping and the engineer of 2020. International Journal of Engineering Education, 34(2), 567–573.
- Ball, L. J., Ormerod, T. C., & Morley, N. J. (2004). Spontaneous analogising in engineering design: A comparative analysis of experts and novices. Design Studies, 25(5), 495–508. DOI: 10.1016/j.destud.2004.05.004
- Carberry, A., Lemons, G., Swan, C., Jarvin, L., & Rogers, C. (2009). Investigating engineering design through model-building. Research in Engineering Education Symposium.
- Cardella, M. E., Atman, C. J., & Adams, R. S. (2006). Mapping between design activities and external representations for engineering student designers. Design Studies, 27(1), 5–24. DOI: 10.1016/j.destud.2005.05.001
- Cardella, M. E., Atman, C. J., Adams, R. S., & Turns, J. (2002). Engineering student design processes: Looking at evaluation practices across problems. In American Society of Engineering Education Annual Conference Proceedings, Montreal, QC. 6015–6023. DOI: 10.18260/1-2--10066
- Cardella, M. E., Atman, C. J., Turns, J., & Adams, R. S. (2008). Students with differing design processes as freshmen: Case studies on change. International Journal of Engineering Education, 24(2), 246–259.
- Cardella, M. E., & Tolbert, D. (2014). “Problem solving” in engineering: Research on students’ engineering design practices and mathematical modeling practices. Proceedings – Frontiers in Education Conference, FIE, 1–3. DOI: 10.1109/FIE.2014.7044345
- Cash, P. (2018). Developing theory-driven design research. Design Studies, 56, 84–119. DOI: 10.1016/j.destud.2018.03.002
- Cash, P., & Maier, A. (2016). Prototyping with your hands: The many roles of gesture in the communication of design concepts. Journal of Engineering Design, 27(1–3), 118–145. DOI: 10.1080/09544828.2015.1126702
- Cheong, H., Hallihan, G. M., & Shu, L. H. (2014). Design problem solving with biological analogies: A verbal protocol study. Artificial Intelligence for Engineering Design, Analysis and Manufacturing: AIEDAM, 28(1), 27–47. DOI: 10.1017/S0890060413000486
- Chimka, J. R., & Atman, C. J. (1998). Graphical representations of engineering design behavior. Proceedings – Frontiers in Education Conference, FIE, 1, 160–165. DOI: 10.1109/FIE.1998.736826
- Christiaans, H. H. C. M., & Dorst, K. H. (1992). Cognitive models in industrial design engineering: A protocol study. Design Theory and Methodology, 42(1), 131–140.
- Coso, A. E., Bailey, R. R., & Minzenmayer, E. (2010). How to approach an interdisciplinary engineering problem: Characterizing undergraduate engineering students’ perceptions. Proceedings – Frontiers in Education Conference, FIE. DOI: 10.1109/FIE.2010.5673313
- Daly, S. R., McKilligan, S., Studer, J. A., Murray, C. K., & Seifert, L. M. (2018). Innovative solutions through innovated problems. International Journal of Engineering Education, 34(2), 695–707.
- Dixon, R. A., & Bucknor, J. (2019). A comparison of the types of heuristics used by experts and novices in engineering design ideation. Journal of Technology Education, 30(2), 39–59. DOI: 10.21061/jte.v30i2.a.3
- Ericsson, K. A. & Simon, H. A. (1984). Protocol analysis: Verbal reports as data, revised edition. The MIT Press.
- Gero, J. S. (2010), Generalizing design cognition research. Dorst, K., et al. (Eds), DTRS8: Interpreting Design Thinking, DAB documents, 187–198. DOI: 10.13140/2.1.3756.5128
- Gero, J. S., Jiang, H., & Williams, C. B. (2013). Design cognition differences when using unstructured, partially structured, and structured concept generation creativity techniques. International Journal of Design Creativity and Innovation, 1(4), 196–214. DOI: 10.1080/21650349.2013.801760
- Gero, J. S., & Kannengiesser, U. (2004). The situated function–behaviour–structure framework. Design Studies, 25(4), 373–391. DOI: 10.1016/j.destud.2003.10.010
- Goldschmidt, G. (1990). Linkography: Assessing design productivity. Proceedings – Cybernetics and Systems ’90, 291–298
- Hurst, A., Rennick, C., & Bedi, S. (2019). A “lattice” approach to design education: Bringing real and integrated design experience to the classroom through engineering design days. ICED19, Delft, The Netherlands. DOI: 10.1017/dsi.2019.46
- Kan, J. W., & Gero, J. S. (2017). Quantitative methods for studying design protocols. Dordrecht, Springer. DOI: 10.1007/978-94-024-0984-0
- Kannengiesser, U., & Gero, J. S. (2017). Can Pahl and Beitz’ systematic approach be a predictive model of designing? Design Science, 3. DOI: 10.1017/dsj.2017.24
- Kavakli, M., & Gero, J. S. (2002). The structure of concurrent cognitive actions: A case study on novice and expert designers. Design Studies, 23(1), 25–40. DOI: 10.1016/S0142-694X(01)00021-7
- Lame, G. (2019). Systematic literature reviews: An introduction. Proceedings of the Design Society: International Conference on Engineering Design, 1(1), 1633–1642. DOI: 10.1017/dsi.2019.169
- Lammi, M. D. (2011). Thinking in terms of systems through engineering design. In American Society of Engineering Education Annual Conference Proceedings, Vancouver, BC. DOI: 10.18260/1-2--18784
- Lammi, M. D., & Gero, J. S. (2011). Comparing design cognition of undergraduate engineering students and high school pre-engineering students. Proceedings – Frontiers in Education Conference, FIE. DOI: 10.1109/FIE.2011.6142816
- Lane, D., & Seery, N. (2011). Examining the development of sketch thinking and behaviour. In American Society of Engineering Education Annual Conference Proceedings. DOI: 10.18260/1-2--17944
- Lemons, G., Carberry, A., Rogers, C., Swan, C., & Jarvin, L. (2009). Using a model-building task to compare the design process of service learning and non-service learning engineering students. Research in Engineering Education Symposium.
- Lemons, G., Carberry, A., Swan, C., Jarvin, L., & Rogers, C. (2010a). The benefits of model building in teaching engineering design. Design Studies, 31(3), 288–309. DOI: 10.1016/j.destud.2010.02.001
- Lemons, G., Carberry, A., Swan, C., Rogers, C., & Jarvin, L. (2010b). The importance of problem interpretation for engineering students. In American Society of Engineering Education Annual Conference Proceedings, Louisville, KY. DOI: 10.18260/1-2--16000
- Liikkanen, L. A., & Perttula, M. (2009). Exploring problem decomposition in conceptual design among novice designers. Design Studies, 30(1), 38–59. DOI: 10.1016/j.destud.2008.07.003
- Mentzer, N., Becker, K., & Sutton, M. (2015). Engineering design thinking: High school students’ performance and knowledge. Journal of Engineering Education, 104(4), 417–432. DOI: 10.1002/jee.20105
- Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLOS Medicine, 6(7),
e1000097 . DOI: 10.1371/journal.pmed.1000097 - Moore, P. L., Atman, C. J., Bursic, K. M., Shuman, L. J., & Gottfried, B. S. (1995). Do freshmen design texts adequately define the engineering design process? In American Society of Engineering Education Annual Conference Proceedings, Anaheim, CA. 164–171
- Moore, D., Sauder, J., & Jin, Y. (2014). A dual-process analysis of design idea generation. Proceedings – International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. DOI: 10.1115/DETC2014-34657
- Morozov, A., Kilgore, D., & Atman, C. (2007). Breadth in design problem scoping: Using insights from experts to investigate student processes. In American Society of Engineering Education Annual Conference Proceedings, Honolulu, HI. DOI: 10.18260/1-2--2318
- Nespoli, O. G., Hurst, A., & Gero, J. S., (n.a.) Exploring a virtual tutor studio pedagogy: A case study. Unpublished manuscript.
- Patel, A., Kramer, W. S., Flynn, M., Summers, J. D., & Shuffler, M. L. (2018). Function modeling: An analysis of pause patterns in modeling activities. Proceedings –International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. DOI: 10.1115/DETC2018-86103
- Perttula, M. K., & Liikkanen, L. A. (2006). Exposure effects in design idea generation: Unconscious conformity or a product of sampling probability?. Development Process: From Idea to the World’s First Bionic Prosthetic Foot. DOI: 10.1115/DETC2006-99123
- Rennick, C., Hulls, C., Wright, D., Milne, A. J. B., Li, E., & Bedi, S. (2018). Engineering design days: Engaging students with authentic problem-solving in an academic hackathon. In American Society of Engineering Education Annual Conference Proceedings, Salt Lake City, UT. 24–27. DOI: 10.18260/1-2--30407
- Roberts, C., Yasar, S., Morrell, D., Henderson, M., Danielson, S., & Cooke, N. (2007). A pilot study of engineering design teams using protocol analysis, In American Society of Engineering Education Annual Conference Proceedings, Austin, TX. DOI: 10.18260/1-2--2811
- Sanderson, P. M. (1995). Macshapa: Software Support for Exploratory Sequential Data Analysis. Proceedings – Human Factors and Ergonomics Society Annual Meeting, 39(9), 511–515. DOI: 10.1177/154193129503900915
- Sauder, J., & Jin, Y. (2013, August). Training the participatory renaissance man: Past creative experiences and collaborative design. Proceedings – International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. DOI: 10.1115/DETC2013-12288
- Sauder, J., Lian, E., & Jin, Y. (2013). The effect of collaborative stimulation on design novelty. Proceedings – International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. 5. DOI: 10.1115/DETC2013-12289
- Song, T., & Becker, K. (2014). Expert vs. novice: Problem decomposition/recomposition in engineering design. International Conference on Interactive Collaborative Learning, ICL, 181–190. DOI: 10.1109/ICL.2014.7017768
- Sriram, A. R., Tolbert, D., Cardella, M., & Ramani, K. (2015, August). Bridging the gaps: Augmenting design learning through computer-aided exploration. Proceedings –International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. DOI: 10.1115/DETC2015-46978
- Sutcliffe, A. G., & Maiden, N. A. M. (1992). Analysing the novice analyst: Cognitive models in software engineering. International Journal of Man-Machine Studies, 36(5), 719–740. DOI: 10.1016/0020-7373(92)90038-M
- Wang, W., Duffy, A., Boyle, I., & Whitfield, R. (2013). Creation dependencies of evolutionary artefact and design process knowledge. Journal of Engineering Design, 24(9), 681–710. DOI: 10.1080/09544828.2013.825103
- Wells, J., Lammi, M., Gero, J., Grubbs, M. E., Paretti, M., & Williams, C. (2016). Characterizing design cognition of high school students: Initial analyses comparing those with and without pre-engineering experiences. Journal of Technology Education, 27(2), 78–91. DOI: 10.21061/jte.v27i2.a.5
- Williams, C. B., Lee, Y., Gero, J., & Paretti, M. C. (2013). Exploring the effect of the design prompt on students’ design cognition. Proceedings – International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. DOI: 10.1115/DETC2013-13557
- Williams, C. B., Paretti, M. C., Lee, Y. S., & Gero, J. (2012). Exploring the effect of design education on the design cognition of sophomore engineering students. In American Society of Engineering Education Annual Conference Proceedings, San Antonio, TX. DOI: 10.1115/DETC2011-48357
DOI: https://doi.org/10.21061/see.27 | Journal eISSN: 2690-5450
Language: English
Page range: 14 - 30
Submitted on: Jan 15, 2020
Accepted on: Nov 17, 2020
Published on: Jan 20, 2021
Published by: Virginia Tech Publishing
In partnership with: Paradigm Publishing Services
Keywords:
© 2021 Gregory Litster, Ada Hurst, published by Virginia Tech Publishing
This work is licensed under the Creative Commons Attribution 4.0 License.