Figure 1
PRISMA systematic review phases for PA publications (Moher et al., 2009).
| IC1: | In order to get the most accurate and useful information, only peer-reviewed journal articles, conference papers, book sections, and books were sought out. In instances when a study was published as a conference paper first and then later as journal article, the former record was excluded. |
| IC2: | PA was the main research method used in the study. |
| IC3: | The study used a design or complex problem solving task. |
| IC4: | The study was conducted in an engineering education setting, where participants were design novices, belonging to one or more of these categories: high school students (in the context of pre-engineering courses), undergraduate students, graduate students. |
Table 1
A summary of the reviewed articles.
| Author(s), Year | Purpose | Design task | Study participants | Coding scheme | Sample size | # of coders |
|---|---|---|---|---|---|---|
| Dixon & Bucknor, 2019 | Explore the heuristics used by novice and expert designers as they go through the ideation phase of a design problem. | Improve existing device (motorcycle) | Engineering students; Professional Engineers | Specific to research question | 8 (4 for each participant type) | 2 |
| Akinci-Ceylan et al., 2018 | Characterize the design approaches of students, academic faculty and practicing engineers in Civil engineering | Design of a process (gum and trash removal) | Engineering students & faculty; Practicing engineers | Specific to research question | 50 (25 for each participant type) | Not reported |
| Patel et al., 2018 | Understand how pauses effect a function modelling activity | Design of a device (ironing machine) | Engineering students, graduate and undergraduate | Specific to research question | 30 | Not reported |
| Daly et al., 2018 | Introduce a new approach to identify the cognitive strategies used by designers is the problem exploration phase. | Design of a device (solar oven or disaster relief) | Engineering students | Unknown | 5 | Not reported |
| Bailey & McFarland, 2018 | Describe how prototyping was integrated into and influenced the design process. | Design of a device (newspaper dispenser) | Engineering students | Atman’s | 36 (9 groups) | 2 |
| Kannengiesser & Gero, 2017 | Provide evidence that there is a causal relationship between unexpected discoveries and novel ideas. | Window problem; door opener; wheelchair kerb assist; bath helper | Engineering students | FBS | 30 (15 groups) | 2 |
| Wells et al., 2016 | Characterize design cognition of highschool students with and without formal pre-engineering experience | Window problem | High school students | FBS | 40 (20 groups) | 6 |
| Cash & Maier, 2016 | Understand the role of gestures used during a design task | Design of a device (remote control camera mount) | Engineering students (graduate) | FBS; Specific to research question | 12 (4 groups) | 2 |
| Sriram et al., 2015 | Understand if/how a framework effects student learning of mechanics and material concepts | Design optimization | Engineering students (undergraduate and graduate) | Unknown | 8 | Not reported |
| Mentzer et al., 2015 | Compare the design process of high school students to experts. Explore the effct of a sequence of engineering courses on design process. | Playground design problem | High school students | Atman’s | 30 | 4 |
| Cardella & Tolbert, 2014 | Explore problem solving approaches for students with formal problem solving experience and those that do not. | Playground design problem | Engineering students | Specific to research question | 25 (at time of publication) | Not reported |
| Moore et al., 2014 | Test hypothesis that Type 1 processes will be greater than Type 2 processes with respect to novel idea generation | Design of a device (secure skateboard rack) | Engineering students (graduate) | FBS; Specific to research question | 10 (5 groups) | Not reported |
| Cheong et al., 2014 | Develop an understanding of conceptual design processes that involve biological analogies | Improve existing systems | Engineering students | Specific to research question | 34 | 2 |
| Song & Becker, 2014 | Explore expert and novice use of problem decomposition and recomposition. | Window problem | Engineering students & faculty; Practicing engineers | FBS; Specific to research question | 50 | 2 |
| Wang et al., 2013 | Investigate the creation dependencies that can exist between engineering artefacts and design process knowledge. | Design of a device (pedestrian barrier) | Engineering students (graduate) | Specific to research question | 2 | Not reported |
| Gero et al., 2013 | To examine and compare design cognition of designers using three different concept generation techniques | Window problem; door opener; wheelchair kerb assist | Engineering students | FBS | 22 (11 groups) | 2 |
| Williams et al., 2013 | Determine if small changes in a design task requirements effect how students approach designing a solution | Door opener problem | Engineering students | FBS | 24 | 2 |
| Sauder & Jin, 2013 | Observe and measure the relationship between collaborative stimulation and cognitive processes. | Design of a device (secure skateboard rack) | Engineering students | Specific to research question | 10 | 2 |
| Williams et al., 2012 | To understand the differences that might exist between the same set of students after they complete an introductory design course. | Window problem; door opener; | Engineering students | FBS | 28 (14 groups) | 2 |
| Lane & Seery, 2011 | Examine students sketching ability and behaviour before undertaking specially designed activities. | Design of a device (boil water) | Undergraduate students | Specific to research question | 41 | 1 |
| Lammi, 2011 | Understand high school students system thinking when engaged in an engineering design challenge. | Window problem | High school students | Specific to research question | 12 (6 groups) | Not reported |
| Lammi & Gero, 2011 | Understand the differences in cognition between undergraduates and senior high school students. | Window problem | Engineering students; High school students | FBS | Unknown | at least 2 |
| Coso et al., 2010 | Examine undergraduate engineering students perceptions of interdisciplinarity and determine if these perceptions vary due to gender or disciplinary affiliation. | Midwest flood problem | Engineering students | Atman’s; Specific to research question | 18 | 2 |
| Lemons et al., 2010b | Understand if engineering students can effectively interpret a design task. | Jar opening problem | Engineering students | Specific to research question | 4 | 2 |
| Lemons et al, 2010a | Investigate the cognitive processes of engineering students during an open-ended, model-building task. | Jar opening problem | Engineering students | Specific to research question | 8 | 2 |
| Liikkanen & Perttula, 2009 | Examines problem decomposition in the solution search phase of conceptual product design | Design of a device (tree trunk demolition; automatic plant watering) | Engineering students | Specific to research question | 16 | Not reported |
| Carberry et al., 2009 | To understand if the inclusion of a model building component enhance the assessment of engineering design. | Jar opening problem | Engineering students | Specific to research question | 4 | 2 |
| Lemons et al., 2009 | Explore if service learning students differ from non-service learning students in their approach to engineering design process. | Jar opening problem | Engineering students | Specific to research question | 4 | 2 |
| Cardella et al., 2008 | Explore how student design processes change over time and how of those changes relate to student preparedness. | Ping pong problem; street crossing problem; Midwest flood problem | Engineering students | Atman’s | 75 (18 within- subject) | 2 |
| Roberts et al., 2007 | Investigate the cognitive processes of engineering students who work in groups rather than individually. | Street crossing problem | Engineering students | Atman’s; Specific to research question | 15 (4 teams) | Not reported |
| Morozov et al., 2007 | Explore expert ways of thinking about problem scoping a design task and determine how this might help inform understanding of novice thinking. | Midwest flood problem | Engineering students; Professional Engineers | Specific to research question | 128 (124 students; 4 profesionals) | 3 |
| Perttula & Liikkanen, 2006 | Explore the effects of examples shown to students in a design task. | Design of a device (house-plant watering system) | Engineering students | Specific to research question | 16 | Not reported |
| Cardella et al., 2006 | Explore engineering students’ use of external representations according to a type of design activity or a stage of design process | Playground design problem | Engineering students | Atman’s | 4 | 2 |
| Atman et al., 2005 | Compare the design process of freshman and senior engineering students, when they engage a different types of design problems. | Ping pong problem; street crossing problem | Engineering students | Atman’s | 75 (18 within- subject) | 2 |
| Ball et al., 2004 | Understand the use of analogies used by expert and novice designers. | Design of a device (automated rental facility kiosk) | Engineering students (graduate); Practicting engineers | Specific to research question | 16 (8 for each participant type) | 1 |
| Adams et al., 2003 | Understand the extent of reflective practice in both freshman and senior engineering students. | Ping pong problem; street crossing problem; Midwest flood problem | Engineering students | Atman’s | >100 | 2 |
| Adams et al., 2002 | Identify the cross-validity between observable design behaviours and written self-reports of team design activity. | Design of testing procedure | Engineering students | Atman’s | 18 teams (2–6 per team) | 2 |
| Cardella et al., 2002 | Exploring if a more complex design problem will require more evaluation from the solver than a less complex problem. | Ping pong problem; street crossing problem | Engineering students | Atman’s | 75 (18 within- subject) | 2 |
| Kavakli & Gero, 2002 | Explore the structure of concurrent cognitive actions in expert and novice design protocools and discuss differences in performance. | Unknown | Engineering students; Practicing engineers | Specific to research question | 2 | Not reported |
| Chimka & Atman, 1998 | Provide examples of how design process data can be shown and what conclusions can be drawn from that. | Playground design problem | Engineering students | Atman’s | 60 | 2 |
| Atman & Bursic, 1998 | Provide an example of a verbal protocol study | Playground design problem | Engineering students | Atman’s; Specific to research question | 60 | 2 |
| Atman et al., 1996 | Provide an example of protocol analysis as a method to study engineering design. | Playground design problem | Engineering students | Atman’s; Specific to research question | 60 | 2 |
| Atman & Bursic, 1996 | The purpose of the work is to investigate the effects of reading a design text on student approaches to three design problems. | Ping pong problem; street crossing problem; Midwest flood problem | Engineering students | Atman’s | 10 | 2 |
| Sutcliffe & Maiden, 1992 | Investigate the cognitive factors underlying performance and to build a model of that process. | Design of a process | Graduate students | Specific to research question | 17 | 2 |
| Christiaans & Dorst, 1992 | Explore the role of knowledge in an industrial design setting | Design of a device (litter disposal) | Engineering students | Specific to research question | 20 | Not reported |