The Added Value of Black Engineering Students’ Participation in Identity-based Organizations: A Systematic Literature Review

Step 1. Formulate Guiding Research Questions and Corresponding Inclusion Criteria

The research question proposed in this study is: According to existing scholarship, what outcomes do Black engineering students experience because of participation in identity-based professional organizations? For the purpose of this study, we defined outcomes as the direct results that occur because of a professional organization or programs’ resources and interventions. These organizations and programs are not limited to the National Society of Black Engineers, Black Greek Letter Organizations, and Minority Engineering Programs (MEPs). The keywords used for identifying primary sources were based on the proposed research question. Primary sources were identified based on the following criteria: 1) a scholarly publication or national report published in the United States from 1994 to 2019; 2) target population of interest and/or participants explicitly reference Black engineering students; and 3) explicit focus on topics related to access or the experiences of Black students in engineering education.

To ensure a comprehensive set of sources, the study includes sources that include African Americans among other underrepresented racial/ethnic minorities. This was done to get a wider breadth of articles since most of the engineering education literature tends to use terms like minority or underrepresented minority (URM) students. Finally, the keywords and inclusion criteria were tested through a brief pilot phase and adjusted if known publications were not captured. This pilot phase also involved the simulation of a search using sentinel articles. The pilot is a method of validating the search by enabling the researcher to modify the search stream using a set of various keywords (i.e., “African American,” “Black,” “engineering,” etc.). Two researchers were involved in this phase of the study. Sentinel articles are a subset of articles conceptually like those being targeted through the search. The pilot search was tested and refined until all but one of the sentinel articles were located through the search.

Step 2. Finding and Cataloging Sources

The search for the sources included in this review came from the following three electronic databases: Education Source (EBSCO host interface), ProQuest Dissertations & Theses Global, and Compendex (EI Village interface). This search was conducted from March 2019 through May 2019. To identify as many potential sources as possible, we also employed additional search approaches: hand searching, citation searching, and relying on input from experts on our advisory board (Papaioannou et al., 2010). While conducting the search, the research team met with a member of our project advisory board who is an engineer and librarian at a research-intensive PWI. They generated the search strings and conducted the actual search for articles in the library databases as shown in Table 1. By the end of the searches, 325 articles were produced from Compendex, 28 articles were produced from Education Source, and 49 were produced from ProQuest. Collectively, a total of 402 articles resulted from using these search strings.

Members of the research team hand-searched (Armstrong et al., 2005) the tables of contents of the Journal of Women and Minorities in Engineering and Science, ASEE Annual Conference & Exposition Proceedings from the Minorities in Engineering Division, and IEEE Frontiers in Education Conference Proceedings. The citation lists of identified sources for additional articles and reports were screened, and the results were discussed regularly. A member of the research team conducted an initial evaluation of each primary source based on abstracts, with each potential source discussed and differences resolved by reaching a consensus. A member of the team logged the results of the searches in a RefWorks database and a flowchart that clearly illustrates the number of sources included and excluded at each phase (Liberati et al., 2009). This procedure is consistent with what was practiced by other STEM education scholars conducting SLRs (e.g., Benitti, 2012; Borrego et al., 2014). An additional 40 articles resulted from this part of the search—resulting in a total of 442 initial articles.

Once the corpus of articles was complete, the research team moved on to screening the articles. The goal of this screening process was to find relevant articles. Three people were involved in the screening process. Communicative validation in the form of member checking was done to ensure that all articles meeting the eligibility criteria were included after screening.

There were multiple phases of screening. An article was selected for inclusion based on the “yes” or “no” criteria questions. First, 25 duplicate articles were removed. Then, the remaining 417 articles were evaluated to see if they met the first round of eligibility criteria. The three hierarchical eligibility criteria were: 1) “Was the article written in English?”; 2) “Was the research setting in a US context?”; and 3) “Was the article published 1994 to 2019?”. For an article to be included in the study, the researcher performing the screening must have answered “yes” to all three eligibility criteria. 22 articles were excluded from the study based on this round, resulting in 395 articles.

These 395 articles then underwent a second round of screening. Figure 1 depicts the PRISMA flowchart associated with this phase of the systematic review. The five eligibility criteria applied during this screening were as follows: 1) “Is the article about a co-curricular program?”; 2) “Is the article about student involvement with a Greek organization?”; 3) “Is the article about student involvement in an identity-based organization?”; 4) “Is the article about Black or underrepresented minority (URM) students?”; and 5) “Is the article about engineering college students?”. For the second round, the articles were required to meet at least one of the first three questions about program involvement. If the article did not meet at least one of these criteria, it was excluded from the systematic literature review. For the last part of the screening process, articles needed to meet the Black or URM criteria and the engineering college student criteria. After two rounds of screening, 21 articles were deemed relevant to the research question posed in this systematic review.

Figure 1

Step 3. Critique and Quality Appraisal of the Literature

After compiling the primary sources and removing duplicates, we assessed the quality of each source by first abstracting details from each. A template was created for abstracting important details from each source:

1. Is the article about identity-based organizations, Greek-letter organizations, or both?

2. Is the study an evaluative study or a research study?

3. What are the research questions?

4. What do we already know based on the themes of the questions?

5. What research methods, evaluation methods, and theoretical frameworks were used?

6. What are the limitations of this study?

7. What implications, recommendations, and future work have been proposed?

8. Are there instances where suggestions have been addressed?

9. What populations are being used to generate information about people at each juncture?

10. When the context is not HBCU, how often is it PWI?

11. What factors positively/negatively influence the participation of African Americans in engineering?

12. Critique: What are some problematic things we observed in the article?

Three members of the research team were involved in this process, with one of the members creating the final critique and appraisal based on the entire team’s work.

Step 4. Address Bias, Validity, and Reliability Concerns

As it is with any research project, the limitations of a systematic literature review stem from both the quality and quantity of the primary studies and the quality of the systematic review procedures (Cook et al., 1997). While the limitations of systematic reviews cannot be eliminated, they can be minimized by employing strategies like defining and following clear procedures and reporting decisions and procedures in detail (Mullen & Ramírez, 2006). To avoid selection bias, primary studies were masked for author names, affiliations, and journal names during the quality appraisal phase. Sources were discussed using a reference ID number as opposed to the authors. Furthermore, the quality of the systematic literature review was determined by consistency and transparency in selecting and reporting procedures for every step along the review. The readers of the review should have enough information (e.g., criteria, protocols) to evaluate the validity of the search procedures, the studies identified as part of the search, and whether the conclusions drawn were warranted. The use of a team to conduct the review, where multiple team members apply and discuss criteria for inclusion/exclusion and the quality assessment, helps establish reliability and minimize our own biases. Important details of primary studies will be presented and cited in the discussion and conclusion sections. A review of our procedures and documentation by the library sciences expert on our advisory board also helped reduce researcher biases.

Step 5. Synthesize Insights

The value of a systematic literature review is largely derived from a thorough synthesis of the primary sources. According to Petticrew and Roberts (2006), there are three basic steps in the synthesis: organize the studies into categories (e.g., by study design, outcome); critique the studies within each category; and critique all studies across all categories.

Thematic analysis was used to complete the first step of organizing the articles into categories. Thematic analysis is a descriptive method that is used to identify, analyze, and report patterns within data (Castleberry & Nolen, 2018). It can be used for multiple types of data, making it ideal for analyzing documents. When conducting a thematic analysis, it is important that the patterns, or themes, identified within go beyond the research topic. In an inductive thematic analysis specifically, the themes are data-driven, whereas in deductive thematic analysis the themes are theory-driven (Braun & Clarke, 2006; Castleberry & Nolen, 2018).

Two members of the research team were involved during this phase. When conducting the thematic analysis (Braun and Clarke, 2006), we familiarized ourselves with the data by creating an article profile for each article, generating initial codes that addressed the guiding research question, then looked for themes (abstract ways to describe the similarities across articles).

During the first phase of thematic analysis, the researchers created an article profile to assess the quality of the literature. During the second phase, initial codes were generated from the article profiles. This was done through an iterative process of descriptive coding and attribute coding to ensure the validity of the findings (Miles et al., 2019). Descriptive codes were assigned to label the types of outcomes that resulted from identity-based professional organizations to accomplish the research question. According to Petticrew and Roberts (2006), by grouping studies by outcomes we can see how the outcomes can be achieved.

Attribute codes were used to distinguish each article that resulted from the systematic literature review (Miles et al., 2019). Articles were distinguished by two attributes: study type and organization type. The study type consisted of research study or evaluative study. If the study was assessing the quality of a program or organization, it was deemed an evaluative study. If the study was based on empirical evidence and aimed to broaden a body of knowledge, it was deemed a research study. This was helpful because the research studies were primarily about identity-based societies such as NSBE, whereas the evaluative studies were primarily about programs hosted by societies and universities. Organization type was used to distinguish whether the article was about a Greek letter organization or any other type of identity-based organization. The third phase began once the coding was complete, which entailed collating codes into themes. After creating themes, they were reviewed to see if each code within each theme properly represented the theme. We relied on terms that are commonly found in engineering education literature to name the themes. A definition for each theme was created and the name for each theme was refined. The themes will be presented in the next two sections. The Results section outlines insights that emerged from critiquing each individual article while the Discussion section presents insights about ideas associated with multiple articles.

Identification and Classification of Studies

Of the 21 articles, 10 articles were classified as evaluative studies and 11 were classified as research studies. Two studies were exclusively focused on BGLOs, 17 focused exclusively on identity-based organizations or programs, and two focused on both BGLOs and identity-based organizations and programs. We found four types of outcomes emerging from students being academically and socially integrated from participation in professional organizations and programs relating to each construct in the model. We relied on terms that are commonly found in engineering education literature to name the themes—namely, self-efficacy, sense of belonging, KSA, and student success. Table 2 contains a list of each theme along with quotes from the articles that exemplify the theme. Table 3, in the Appendix, contains a list of all 21 articles along with their attributes.

Self-Efficacy

Research has shown that self-efficacy is a positive predictor of engineering students’ success as it relates to academic achievement (Hsieh et al., 2012) and persistence (Concannon & Barrow, 2010). Self-efficacy is defined as a person’s belief in their abilities to complete a task to achieve a goal (Bandura, 1997). When students believe in their own efficacy, they will then act in ways that will continue their success. One’s self-efficacy can contribute to how they operate within a domain; for the context of this study, the domain is engineering (Mamaril et al., 2016). Within the context of engineering, self-efficacy can be more pronounced due to students’ learning experiences in engineering (Carberry et al., 2010; Jordan & Gaskins, 2011). While only one study used Bandura’s (1997) operationalization of self-efficacy (Jordan & Gaskins, 2011), the definition was used throughout the systematic literature review to recognize when a students’ belief in their ability to be engineers stemmed from participating in a identity-based organization or program. From attribute coding, it was seen that studies reviewed as part of the systematic literature review primarily discussed self-efficacy as a result of participation in identity-based organizations, such as NSBE.

While self-efficacy can operate in reference to a student’s ability to perform engineering tasks and their academic capabilities, this theme was primarily conveyed through a person’s confidence in their ability to be an engineer or pursue an engineering profession. Self-efficacy was discussed by authors revealing that by participating in NSBE, students felt more confident to successfully navigate engineering education or work as an engineer after graduation (Ross & McGrade, 2016). A student’s confidence in their ability to be an engineer was a result of being highly integrated within an identity-based organization such as NSBE by participating in events hosted by the organization (Cola et al., 2010; Ross & McGrade, 2016), and assuming leadership positions in their local NSBE chapter (Ross & McGrade, 2016). An example of this theme comes from Ross and McGrade (2016), which discussed that by seeing other successful Black engineers in their local chapter and at the national level, Black students gained the confidence to work towards becoming an engineer. Jordan and Gaskins (2011) found that students in NSBE reported high mean values of engineering self-efficacy and engineering career success from taking the Longitudinal Assessment of Engineering Self-Efficacy.

Sense of Belonging and Community

It was found throughout the articles that from participating in identity-based societies and programs students felt a sense of belonging within engineering and their community (Cola et al., 2010; Erickson-Ludwig & Clyne, 2014; Prewitt et al., 2007; Ross & McGrade, 2016; Schulte et al., 1999; St. Omer et al., 1999; L. D. Thomas et al., 2009; Trenor et al., 2010). This was conveyed through three codes: sense of belonging, support, and social capital. When defining a sense of belonging as a code, the researcher referred to literature resulting from the systematic literature review. For this work, Hagerty’s definition of a sense of belonging is used. Hagerty et al (1992) defines sense of belonging as “a person’s experience of being valued or important to an external referent and experiencing a fit between self and that referent” (p. 174). For this study, the external referent was defined as engineering. An example of this code was conveyed when Ross and McGrade (2016) reported that NSBE members perceived integration within the society contributed to their sense of belonging within engineering. By gaining a sense of belonging, students were able to perceive themselves as engineers.

Support was defined as people that provide assistance as a result of participating in an organization, and people they are able to assist as a result of participating in an organization. Through support, students were able to build community. There were two sub-codes within the category of support: gaining support and giving support. When discussing gaining support, this is referring to students receiving assistance in the form of role modeling and mentoring. St. Omer et al (1999) and Schulte et al (1999) discussed the importance of professional identity-based organizations in providing mentors and role models for both undergraduate and K-12 members. Oftentimes students in these organizations were paired with a mentor through a formal program or found an older member of the organization or an alumnus who served as a mentor or role model. Additionally, students found role models by going to conferences or other events. Another way students received assistance was by gaining a “family” or sense of community from these organizations (Ross & McGrade, 2016; St. Omer et al., 1999). Organizations providing a sense of family were mentioned multiple times throughout the studies (Prewitt et al., 2007; Trenor et al., 2010). This sense of family was accomplished by having support structures within the organization itself that were beyond academic interactions and more social interactions through social events and mentoring (L. Thomas & Smith, 2010).

The sub-code giving support refers to students having the opportunity to assist others as a result of participation in an organization. This was usually facilitated by serving as mentors and role models, as well as serving the broader community. Giving support was discussed by authors revealing that students who were academically successful were able to mentor and tutor students through programs created by their identity-based organization (Schulte et al., 1999). Additionally, students expressed the desire to become role models and mentors for underclassmen students due to previous support they gained by participating in an identity-based organization (St. Omer et al., 1999). This code was found in both research and evaluative studies and was seen because of identity-based organizations and programs. Schulte et al (1999) discussed that by Black undergraduate students participating in an identity-based organization, students were able to be academically successful. Thus, these members can mentor students and motivate them to pursue engineering.

Lastly, along with support, organizations promoted social capital to build a community within the society or program. Prewitt et al (2007), whose paper was included in the systematic literature review, used Staton-Salazaar’s (2001) definition of social capital “a set of properties existing within socially patterned association among people that, when activated, enable them to accomplish their goals or to empower themselves in some meaningful way” (p. 256). This can be developed through personal relationships and networks with people within these organizations (Prewitt et al., 2007). Social capital was demonstrated by members of an organization perceiving their organization as a way to establish relationships with their peers (Ross & McGrade, 2016). Another paper discussed how through the formation of a “family-like bond”, BGLOs were found to promote social capital at a PWI (Trenor et al., 2010). Social capital was also seen at conferences hosted by professional identity-related organizations, such as the NSBE national convention, by students meeting people with similar interests like them (St. Omer et al., 1999). Social capital was accomplished by organizations having events where members could interact with other Black students such as conferences, mentoring programs, networking events, and workshops. By students being in spaces where there were people who looked like them, they were able to form personal relationships and networks with people that helped empower them succeed in engineering (Prewitt et al., 2007).

KSAs (Knowledge, Skills, and Attributes)

KSAs were defined as knowledge, skill, or attributes that students gained from participating in an organization (Erickson-Ludwig & Clyne, 2014; Loftus, 2008; Prewitt et al., 2007; Ross & McGrade, 2016; Schulte et al., 1999; Simmons & Martin, 2011; L. Thomas & Smith, 2010; Trenor et al., 2010; Young et al., 2014). KSAs were seen in both research studies and evaluative studies that studied identity-based programs and organizations. Oftentimes, KSAs were seen as an outcome for Black students participating in activities that were housed by a MEP or an identity-based organization. In one study, several KSAs resulted from a NSBE program housed within the organization (Schulte et al., 1999). These KSAs include remote control car building, internet usage and web page design, study skills, scholastic aptitude test preparation, and how to apply for college. In another study, it was found that taking advanced math and science classes within NSBE Jr. enhanced the STEM ability of students within the program, an example of an ability that was produced from an identity-related program (Cola et al., 2010).

However, authors did discuss how identity-based organizations collectively produced KSAs for students. Young, Knight, and Simmons (2014) found evidence that students who were involved in engineering clubs reported higher teamwork skills and reflective behavior outcomes than students who were not involved in engineering clubs. Teamwork skills are an example of a skill that a student may gain, and reflective behaviors are an example of an attribute that a student may gain. From a separate study, BGLOs were found to provide some of the traits that the National Academy of Engineering wishes for engineers to have by the time they graduate (Simmons & Martin, 2011). These traits were: “practical ingenuity, good communication skills, business and management skills, leadership, high ethical standards, and professionalism” (Simmons & Martin, 2011, p. 2). While specific factors contributing to the KSAs were not included for these studies, we do know that students can gain these KSAs from participation in an identity-based organization (Erickson-Ludwig & Clyne, 2014; Loftus, 2008; Prewitt et al., 2007; Ross & McGrade, 2016; Schulte et al., 1999; Simmons & Martin, 2011; L. Thomas & Smith, 2010; Trenor et al., 2010; Young et al., 2014).

Additionally, within this theme was the sub-code persistence & motivation. Persistence and motivation was defined as students having the determination or willingness to either pursue engineering or stay in engineering. This attribute is the result of students’ participation in identity-based societies and programs. Persistence and motivation was not seen as much within identity-related programs in the evaluative studies; however, it was discussed when talking about the success of the programs (Erickson-Ludwig & Clyne, 2014; Roemer et al., 1999; Schulte et al., 1999). Examples of this code were seen through authors discussing that students in the organizations believed their membership influenced their decision to not leave engineering and persist (Erickson-Ludwig & Clyne, 2014; Prewitt et al., 2007; Schulte et al., 1999). There were no specific factors mentioned that played into persistence and motivation besides being integrated in the organization.

Success

The success theme included two sub-codes: academic success and professional success. Success was defined as an accomplishment that occurred from participation in an organization. For academic success, this was anything related to students benefiting academically. This can be in terms of GPA or reaching academic goals. Academic success was seen in both evaluative and research studies that examined identity-based programs and organizations and BGLOs. An example of academic success came from a paper discussing how being in a BGLO motivated members to achieve and maintain high academic performance in engineering (Trenor et al., 2010). This was also seen in NSBE and MEPs as student participation was seen to increase GPAs and retain students in engineering (Erickson-Ludwig & Clyne, 2014; Newell et al., 2004; Yates & Nagle, 2016). Academic success was often contributed by identity-based organizations and programs being academically supportive, particularly through having study groups, having tutors and mentors that can help students accomplish their academic goals, and hosting academic related workshops (Cola et al., 2010; Erickson-Ludwig & Clyne, 2014; Trenor et al., 2010; Yates & Nagle, 2016).

Professional success was primarily seen as an outcome for K–12 programs, such as NSBE Jr. which is housed under NSBE’s Pre-College Initiative programs (PCI). When looking specifically at professional success, this was defined as the advancement of progress in students’ careers or education after graduation of either high school or undergraduate. Professional success was primarily exhibited as an outcome in evaluative studies for both identity-based programs and organizations. At the undergraduate level, professional success was exhibited in Yates and Nagle’s (2016) work which found that 92.8 percent of students who participated in NSBE pursued graduate education or a job in the engineering field (Yates & Nagle, 2016). Factors that contributed to professional success after undergraduate were not mentioned. NSBE Jr. was said to influence students to STEM careers because it exposed the students to advanced math and science courses (Cola et al., 2010). In another K–12 program, a former participant went on to attend veterinary school, and feedback from this participant said that tutoring provided by the program was helpful in their success (Roemer et al., 1999). Best practices that contributed to professional success for K–12 students included offering advanced classes in math and science and tutoring from members of NSBE undergraduate chapters. These programs are to serve primarily K–12 students of color, especially students who identify as Black.

Implications for Researchers

There is a need for more robust studies in this area. Of the final articles, none were published in journals. Publishing in peer-reviewed journals will advance this area of scholarship since they are more widely accessible and searchable than conference proceedings, reports, and other forms of scholarly outputs. Additionally, journal articles that have been peer reviewed also tend to meet higher standards of research quality and have stronger connections to existing literature and/or debates happening among scholars in a field. For these reasons, we want to encourage researchers to publish more peer-reviewed journal articles on this topic.

Lastly, there was a lack of research on BGLOs, as only three of the twenty-one articles studied these organizations. From the articles centered on BGLOs, it was shown that participation in these organizations contributed to traits outlined in the Engineer of 2020 (National Academy of Engineering, 2004). Thus, there is a need to identify how BGLOs are helping engineering students persist and consequently, how other non-engineering identity-based organizations and programs can learn from them to help Black engineering students.

Implications for Practitioners

First, schools should place a heavy emphasis on identity-based programs and organizations for Black students in engineering to participate, especially in schools that are PWIs. A heavier emphasis may include, but is not limited to, investing more resources to promote these organizations and programs, sponsoring student membership in them, and/or providing them a designated space on campus (Blust, 2001; National Academies of Sciences Engineering and Medicine, 2011). This is important because while the results of this study show the importance and benefit of identity-based organizations and programs, the problems that are affecting Black students in engineering still remain since they are still underrepresented in engineering (American Society for Engineering Education, 2021). This should be done at the K–12 and undergraduate levels as our work shows that these outcomes were produced from K–12 programs such as NSBE Jr and tutoring programs for K–12 students. In order to support Black students, all engineering institutions should have a NSBE chapter and/or an MEP. Engineering programs should aim to partner with NSBE chapters and MEPs to bring awareness to their existence and stress their importance. Additionally, partnerships between engineering programs and identity-based engineering programs and societies can foster collaboration that can lead to improved outcomes for all students.

Lastly, all program administrators and organization leaders must assess whether their programs and organizations are producing good outcomes for the students participating in them. Of the 21 articles that were presented, 10 were evaluative studies that were assessing the impact of either local chapters of identity-based organizations or MEPs. We encourage program administrators and organization leaders to continually assess the quality of their programs and use our findings as a guide to what outcomes should be produced in students in their programs. If practitioners choose to share the outcomes of their programs to the research community, we encourage them to situate their studies within larger findings communicated in research studies so that these findings can be used in a broader context, specifically when looking at ways to broaden participation in engineering.