Exploring Patterns in Engineering Brainstorming Conversations by Combining Discourse Analysis and Statistical Methods

GROUP	YEAR IN SCHOOL	MEN (HE/HIM)	WOMEN (SHE/HER)	WHITE	NON-WHITE	TOTAL PARTICIPANTS
1	2–4	4	2	3	3	6
2	3–4	3	2	0	5	5
3	2–3	2	2	2	2	4
4	2–3	3	3	4	2	6
5	2–3	4	2	4	2	6
6	2	1	3	2	2	4
7	4–5	2	2	1	3	4
8	1–2	2	0	1	1	2
9	1	2	1	1	2	3
10	1–2	2	3	2	3	5
11	1–4	2	4	3	3	6
12	4	1	2	2	1	3
13	2–4	3	2	4	1	5
*Total*		31	28	29	30	59

[i] *All student respondents used either he/him or she/her pronouns.

Table 2

Discourse analysis codebook identifying the different discursive moves made by participants during brainstorming.

CODE	DEFINITION
Ideation	Proposal of an idea/potential solution, including additions to and variations of a root idea
Facilitation	An effort to guide the focus of the discussion and the group (typically to move forward or invoke more ideas)
Acknowledgment	An indication of acknowledging someone’s comment or responding to another member’s query
Support	An indication of supporting and approving someone’s idea
Clarification	A reiteration, elaboration, or prompting of elaboration on a previously stated idea, question, or prompt
Criticism	An indication of not supporting or rejecting someone’s (or one’s own) comment
User Consideration	Direct discussion of user’s needs, desires, or limitations, whether it be during idea proposal or otherwise
Building Credibility	Efforts to boost an individual’s reputation in the group through mention of previous experiences
Technical Talk	Invocation of engineering-specific terms to describe or elaborate on, usually used to “perform” engineering knowledge
Small Talk	Conversation about topics not directly related to the prompt, usually helps to build rapport within the team

Example of coding process using an excerpt from a transcript.

Table 3

Descriptive statistics showing code counts for each transcript and the mean, standard deviation, maximum, and minimum. The proportion of statements withing a given transcript is noted in parentheses. Note: The data for Group 6 is derived from incomplete data collection due to technical errors.

GROUP #	IDEATION	FACILITATION	ACKNOWLEDGMENT	SUPPORT	CLARIFICATION	CRITICISM	USER CONSIDERATION	BUILDING CREDIBILITY	TECHNICAL TALK	SMALL TALK	TOTAL CODED LINES
1	67 (20%)	14 (4%)	120 (36%)	47 (14%)	38 (11%)	8 (2%)	4 (1%)	1 (0.3%)	7 (2%)	29 (9%)	335
2	48 (20%)	16 (7%)	65 (27%)	39 (16%)	46 (19%)	2 (1%)	12 (5%)	0 (0%)	2 (0.8%)	13 (5%)	243
3	47 (20%)	15 (6%)	66 (28%)	20 (8%)	55 (23%)	5 (2%)	7 (3%)	3 (1%)	5 (2%)	14 (6%)	237
4	57 (15%)	12 (3%)	52 (14%)	19 (5%)	140 (38%)	24 (7%)	35 (9%)	2 (0.5%)	9 (2%)	19 (5%)	369
5	57 (15%)	19 (5%)	102 (28%)	53 (14%)	80 (22%)	10 (3%)	28 (8%)	5 (1%)	6 (2%)	8 (2%)	368
6	24 (26%)	9 (10%)	17 (18%)	7 (8%)	22 (24%)	1 (1%)	8 (9%)	0 (0%)	0 (0%)	5 (5%)	93
7	56 (31%)	8 (4%)	29 (16%)	16 (9%)	49 (28%)	1 (1%)	7 (4%)	0 (0%)	1 (0.6%)	11 (6%)	178
8	25 (17%)	8 (5%)	21 (14%)	4 (3%)	80 (55%)	3 (2%)	4 (3%)	0 (0%)	0 (0%)	1 (0.7%)	146
9	27 (15%)	8 (4%)	20 (11%)	8 (4%)	87 (49%)	3 (2%)	11 (6%)	0 (0%)	8 (4%)	7 (4%)	179
10	43 (23%)	10 (5%)	26 (14%)	18 (10%)	57 (30%)	10 (5%)	18 (10%)	0 (0%)	3 (2%)	3 (2%)	188
11	31 (23%)	4 (3%)	17 (13%)	16 (12%)	46 (35%)	2 (2%)	2 (2%)	1 (0.8%)	1 (0.8%)	12 (9%)	132
12	52 (19%)	19 (7%)	65 (24%)	35 (13%)	59 (22%)	7 (3%)	19 (7%)	1 (0%)	10 (4%)	7 (3%)	274
13	40 (14%)	18 (6%)	63 (22%)	26 (9%)	87 (31%)	9 (3%)	11 (4%)	0 (0%)	4 (1%)	23 (8%)	281
*Mean*	44 (19%)	12 (5%)	51(22%)	24 (10%)	65 (28%)	7 (3%)	13 (5%)	1 (0.4%)	4 (2%)	12 (5%)	233
*Standard Deviation*	14.0	4.9	33.4	15.5	30.0	6.2	9.8	1.5	3.5	8.1	89.7
*Maximum*	67	19	120	53	140	24	35	5	10	29	369
*Minimum*	24	4	17	4	22	1	2	0	0	1	93

Table 4

Correlation matrix showing correlation coefficients and p-values for all codes.

CORRELATIONS
		IDEATION	FACILITATION	ACKNOWLEDGMENT	SUPPORT	CLARIFICATION	CRITICISM	USER CONSIDERATION	BUILDING CREDIBILITY	TECHNICAL TALK	SMALL TALK
Ideation	Pearson Correlation	—
Facilitation	Pearson Correlation	.557*	—
Facilitation	Sig. (2-tailed)	0.048
Acknowledgment	Pearson Correlation	.759**	.766**	—
Acknowledgment	Sig. (2-tailed)	0.003	0.002
Support	Pearson Correlation	.683*	.762**	.913**	—
Support	Sig. (2-tailed)	0.01	0.002	0
Clarification	Pearson Correlation	0.206	0.135	0.017	–0.066	—
Clarification	Sig. (2-tailed)	0.499	0.661	0.955	0.832
Criticism	Pearson Correlation	.586*	0.369	0.357	0.273	.760**	—
Criticism	Sig. (2-tailed)	0.035	0.215	0.231	0.367	0.003
User Consideration	Pearson Correlation	0.477	0.434	0.235	0.313	.687**	.832**	—
User Consideration	Sig. (2-tailed)	0.099	0.138	0.439	0.298	0.009	0
Building Credibility	Pearson Correlation	0.479	0.449	.573*	.559*	0.239	0.424	0.496	—
Building Credibility	Sig. (2-tailed)	0.098	0.124	0.04	0.047	0.431	0.149	0.085
Technical Talk	Pearson Correlation	.567*	0.516	0.506	0.432	0.493	.629*	.588*	0.393	—
Technical Talk	Sig. (2-tailed)	0.043	0.071	0.078	0.14	0.087	0.021	0.035	0.184
Small Talk	Pearson Correlation	.580*	0.314	.643*	0.473	0.07	0.299	–0.069	0.083	0.313	—
Small Talk	Sig. (2-tailed)	0.038	0.296	0.018	0.103	0.821	0.321	0.824	0.788	0.298

[i] *. Correlation is significant at the 0.05 level (2-tailed).

**. Correlation is significant at the 0.01 level (2-tailed).

Visual model showing statistically significant correlations among discursive moves.

Diagram of correlational mapping surrounding ideation. Red refers to *Active Engagement*. Green refers to *Group Rapport*. Blue refers to *Exploring the Problem*.

Table 5

Emergent themes from each code cluster and their operational definitions.

CODE CLUSTER	THEME	OPERATIONAL DEFINITION
Ideation – Acknowledgment – Support – Facilitation	Active Engagement	Conversation that surrounds heavy participation, through offering, guiding, or responding to ideas
Acknowledgment – Small Talk – Ideation	Group Rapport	Conversation that develops a level of comfort with groupmates
Ideation – Criticism – User Consideration – Clarification	Exploring the Problem	Conversation around the boundaries of the scenario to better address users’ needs and define the problem

LINE	SPEAKER	TRANSCRIPT	CODE(S)
331	S1:	10-foot pole. Really cool. Also really unsanitary.	Support, Criticism
332	S2:	I don’t think play structures, as a rule, are, though.	Clarification
333	S1:	Yeah.	Acknowledgment
334	S1:	What else?	Facilitation
335	S3:	Those mazes … You know those wheel mazes? Where there’s a ball in a wheel and you’re spinning it, and you have to get it to go the way that-	Ideation
336	S1:	Come out. Yeah. Those are cool.	Support
337	S3:	We’ve got to exercise their minds and stuff. Right?	User Consideration
338	S1:	Yeah. And those will be good because they use all their hands and stuff.	Support
339	S4:	The balls that you can climb into and run around with. If we could have this standard wheelchair that was discussed earlier that has more degrees of freedom in the rotation on the bottom wheels, so then you could go in all directions like inside a ball.	Ideation
340	S3:	Yeah.	Acknowledgment
341	S2:	Wow.	Support

LINE	SPEAKER	TRANSCRIPT	CODE(S)
137	S1:	Well do you know the guy at [Redacted University] who’s like outfitted his skateboard and --	Small talk
138	S2:	Yeah.	Acknowledgment
139	S1:	I’ve seen him around town, he has a wheelchair, and he still skateboards.	Small talk
140	S2:	Yeah, he--	Acknowledgment
141	S3:	Pretty insane.	Acknowledgment
142	S2:	He’s gnarly. He’s, he’s shown up on like, Sports Center and all that stuff. Yeah. That’s pretty cool.	Small talk
143	S4:	I’ve never seen that. What do you mean like he outfitted it, is it electric?	Small talk
144	S2:	[inaudible]--	NA
145	S1:	Yeah.	Acknowledgment
146	S2:	Go ahead.	Facilitation
147	S1:	It’s electric, and then he has some sort of mounts where like, he can put his wheel sideways on it, and so he’s like looking this way and like, going like, this way as he’s controlling it electrically but he’s on his wheelchair on the skateboard.	Small talk
148	S2:	Like one of those boosted board things.	Small talk
149	S4:	Wow.	Acknowledgment
150	S2:	Yeah, it’s super neat. Like super mesmerizing to watch how he’s able to just balance himself, and you know, just lean back and forth, and he, he looks like he’s gonna fall the whole time but he’s actually like, totally fine.	Small talk

LINE	SPEAKER	TRANSCRIPT	CODE(S)
151	S4:	You can also have like a, a ramp that looks like it’s a jump over something, but it’s actually like the, the part where there would be the jump is like, clear so like, it’s the feeling of getting air over something in a wheelchair.	Ideation
152	S2:	Oh yeah.	Acknowledgment
153	S3:	I like that.	Support
154	S2:	I don’t know how I’d make this work with someone in a wheelchair but a foam pit is super cool.	Ideation
155	S3:	That is cool. Might get a little dirty but uh, yeah, very cool.	Support, Criticism
156	S2:	Yeah.	Acknowledgment
157	S5:	You already have a trampoline on here right? Oh yeah.	Clarification (requesting)
158	S2:	Yep, yeah I have no idea how that will work either but--	Clarification (offering)
159	S5:	Haha.	Acknowledgment
160	S2:	But I’m putting it down.	Clarification (offering)
161	S4:	I think some kinda, wheelchair bumper car attachment would also be fun.	Ideation
162	S2:	Oh yeah, oh my god.	Support

LINE	SPEAKER	TEXT	CODE(S)
24	S1:	What I’ve seen before is a swing that’s kind of more similar to a car seat, so where you wheel up backwards to it. And then you push yourself into the seat, and it clips you in. Because someone else will have to be there to push them on the swing. So that way, they’re secure. And then – I don’t know – it’s less difficult to secure the entire wheelchair in place if that makes sense.	Ideation, User consideration
25	S2:	That kind of makes sense. Yeah.	Acknowledgment
26	S3:	Yeah, I feel like the only worry is that then that means that you have to have someone else. But I guess if you’re on a playground, you’re mostly with a little kid. You probably [crosstalk] the whole field.	Criticism
27	S1:	I mean, I think for a swing, you have to have someone to push you.	Clarification (requesting)
28	S2:	Yeah, that’s a two-person activity right there.	Clarification (offering)
29	S4:	Yeah. So I guess to stop, yeah, you will have that one person to stop you.	Clarification (requesting)
30	S2:	Yeah, so. Okay. I think I came up with another idea on the spot, which was, what if a slide, we make one, obviously, with less twists and turns and stuff. But then you create a track system so that the person with their wheelchair can just kind of slot right into the track, and then slide down the slide.	Ideation
31	S4:	Would they be sliding with the wheelchair on them?	Clarification (requesting)
32	S2:	Yeah, yeah. Kind of a roller coaster kind of model where they would be in the chair and then rolling-on-ish rail system [sic].	Clarification (offering)