Which Combination of Game Elements Can Lead to a Useful Gamification in Education? Evidence from a Fuzzy-set Qualitative Comparative Analysis

Gamification in Education

Gamification has been defined differently in the literature. Kapp (2012), for instance, defined gamification as “the use of game-based mechanics, aesthetics and game thinking to engage people, motivate action, promote learning, and solve problems.” Zichermann and Cunningham (2011) considered gamification as “the process of game-thinking and game mechanics to engage users and solve problems.” However, one of the most-agreed definition belongs to Deterding et al. (2011) who considered gamification as “the use of game design elements in non-game contexts.” Several studies pointed out that using gamification in educational contexts can make learning activities more attractive and competitive, hence enhancing students’ learning engagements, as well as improving the obtained learning outcomes (Andrade et al., 2016; Denden et al., 2021; Villagrasa et al., 2018).

Various gamification frameworks were reported in the literature to gamify a given course. For example, Werbach and Hunter (2012) proposed a 6D framework that covers 30 game elements (Mora et al., 2015). Klock et al. (2016) proposed another framework which contains 14 game elements. Marczewski (2015) further presented, based on user types, the GAME framework which contains 52 game elements. Despite the variation in game elements across the proposed frameworks, Toda et al. (2019) highlighted that there are still notable similarities in the game elements utilized within these frameworks. However, not all game design elements yield the same benefits when combined (Mazarakis & Brauer, 2023). Sümer and Aydın (2022) identified badges, leaderboards, points, and levels as the most frequently used game elements in online learning environments.

Moreover, a survey conducted by Schöbel et al. (2016) revealed that the combination of levels, point, and goals is from the most preferred combinations in learning environments. An experimental study by Mazarakis and Bräuer (2023) showed that the combination of badges, progress bar, and feedback can significantly motivate students to engage more actively, such as answering a greater number of quiz questions. Additionally, the chat game element has been shown in multiple gamification studies to effectively satisfy the need for relatedness (Wee & Choong, 2019). Therefore, this present study identified seven game elements (see Table 1) which are commonly used in 6D and GAME frameworks and have been proven to form an effective combination for online learning. These elements are later used for the design of our gamified course (see Section 2).

Research Gap and Study Objectives

Despite that gamification is introduced and implemented for more than twenty years since it was introduced in 2002, it is not fully adopted in education due to the scant information on how to make it useful in a given learning environment, or which game elements to use, as well as which combination of game elements is more appropriate (Martí-Parreño et al. 2016; Sánchez-Mena & Martí-Parreño 2016; Toda et al. 2018). Yasemin and Samur (2018), in this context, revealed that gamification might lead to students’ negative behaviors if it is not correctly implemented. For instance, the use of only competitive game elements, such as badges and leaderboards, can impact negatively low performing students (Andrade et al., 2016). Several gamification taxonomies and frameworks were therefore proposed in the literature and research efforts sought to explore game element recommendations based on user preferences, gender, and culture (Schöbel et al., 2017; Toda et al., 2022). However, little information is provided by each taxonomy/framework about the rationale for using a particular set of game elements (Toda et al., 2019). Based on this research gap, further research is needed to investigate which combination of game elements can lead to students perceiving gamification is useful in education. This could be challenging as gamification is an interconnection of several elements, where no single combination might be obtained.

To overcome this challenge, this study builds on previous game element taxonomies in the literature to first develop a gamified course using seven game elements (see Section 2). It then relies on the complexity theory and fuzzy-set Qualitative Comparative Analysis (fsQCA) to investigate which combination of the seven-game element can explain students’ perceived usefulness of gamification in the given course. Complexity theory is adopted in this study because it allows obtaining multiple solutions (configurations) for explaining an outcome equally as effectively, depending on the effects of variables, complex conditions, and context settings (Miao et al., 2022). Additionally, fsQCA is considered an adequate methodology as it addresses the net effect flaw of traditional approaches, such as regression-based methodologies (Campbell et al., 2016), by simultaneously considering multiple interdependent factors (causal configurations) that should be studied together (Campbell et al., 2016; El Sawy et al., 2010). This allows handling significant levels of causal complexity (Fiss, 2011; Ragin, 2008; Woodside, 2013).

Several studies have used fsQCA in the gaming industry to investigate, for instance, users’ experiences toward mobile games (Pappas et al., 2019) or gamified mobile wallet (Yang et al., 2023), users’ intention to download mobile games (Tang & He, 2022), and users’ development of entrepreneurial intentions through gamification (Pérez-Macías et al., 2022). However, no previous study, to the best of our knowledge, have applied fsQCA in educational gamification. Therefore, the use of configurational approaches, i.e., fsQCA, can close the aforementioned research gap and reveals insights to various stakeholders (educators, game designers, etc.) on how to design useful educational gamification.

This study contributes to the educational gamification research by providing evidence on the potential combinations of game elements in the gamified courses for perceived useful gamified applications. It supports educators and instructional designers with sets of promised solutions encompassing the game elements that work best together for effective gameful design.

Research Propositions

This study’s goal is to identify which combination of variables (causal conditions in fsQCA terminology) are a necessary and/or a sufficient condition for students to perceive gamification useful in their gamified course. The analyzed variables are the seven game elements implemented in the gamified course and used by the students, namely: (1) leaderboard; (2) badge; (3) points; (4) progress bar; (5) level; (6) feedback; and (7) chat. Specifically, to address this goal, the following two propositions are analyzed. It should be noted that the term proposition was used instead of hypothesis because fsQCA does not test any statistical hypothesis (Wu et al., 2014). Therefore, the term proposition was used accordingly in this study to highlight the precepts to be tested in this research context.

Proposition 1: No single causal condition (i.e., no single game element among the seven implemented game elements) leads to students perceiving gamification useful in their gamified course. This implies that no causal condition is a necessary condition for perceiving gamification useful.

Proposition 2: No single configuration (i.e., no single combination of the seven implemented game elements) leads to students perceiving gamification useful in their gamified course; However, various configurations might do so, where single causal conditions (leaderboard, badge, points, progress bar, level, feedback, and chat) might be present or absent depending on each configuration.

As no information exist in the literature about the role of each specific causal condition, this study answers the following research question: which of the seven variables (game elements) are necessary or/and sufficient to create causal combinations that explain students’ perceived usefulness of gamification in the gamified course?

Participants and Procedure

The experiment was conducted at a public university. Specifically, ninety undergraduate students took the OODM course, which was taught online, as part of their computer science curriculum at the university for the whole semester (three months). OODM is considered as one of the important courses in computer science field as it is the key for other courses, such as data base design and acts as a fundamental step in project realization. However, some students find difficulties in understanding all the diagrams to successfully design their projects. Therefore, teaching such courses need to much attention as recommended by Moreira (1998). All the students were familiar with Moodle and had used it before for non-gamified courses at the university.

At the beginning of the course, the teacher started by introducing each game element and its purpose to ensure that students will make full use of it while learning. The students then took the gamified course for the whole semester. At the end of the semester, all the students answered two questionnaires to collect their perceptions toward the seven implemented game elements within the gamified course, as well as their perceived usefulness of gamification as a treatment combining the set of game elements within the gamified course. Among the ninety students, only eighty-three of them have completed all the required questionnaires. Table 2 presents the demographic information of students.

Instruments

Game elements perception questionnaire: It is a 5-point Likert-scale questionnaire ranging from 1 (strongly agree) to 5 (strongly disagree), and aims to collect students’ perception of each game element, among the 7 implemented game elements, in terms of enjoyment and usefulness. It was adapted based on several studies (Bajko et al. 2016; Halko & Kientz, 2010; Koivisto & Hamari, 2014). Enjoyment is defined as students’ degree of enjoyment and contentment when using each game element. This construct includes statements, such as “I enjoy using the leaderboard while learning” and “I find it enjoyable to receive badges while learning.” Usefulness is defined as students’ degree of belief that each game element will enhance learning within the gamified course. This construct includes statements, such as “Using the leaderboard motivates me to learn more” and “I find the progress bar useful.” The questionnaire was validated in a previous work (Denden et al., 2017; 2018). Furthermore, to ensure the reliability of the questionnaire, its internal consistency was calculated using Cronbach’s alpha. The obtained alpha value was above 0.7, affirming that this questionnaire is reliable (Yu, 2001).

Gamification usefulness: It is a 5-point Likert-scale questionnaire ranging from 1 (strongly agree) to 5 (strongly disagree), and aims to collect students’ perceived usefulness of gamification in education. It includes statements such as “I believe gamification is important to successfully finish the course” or “gamification increased my motivation to do my course assignments.” The questionnaire is adapted from a previous study by Koivisto and Hamari (2014), which is widely used in the literature. To ensure the reliability of the questionnaire, its internal consistency was calculated using Cronbach’s alpha and the obtained value was above 0.7, affirming that it is reliable (Yu, 2001).

Data Analysis: An fsQCA Approach

To conduct an fsQCA analysis, calibration, which is an important step that aims to convert research variables into fuzzy sets, was first conducted. Particularly, during the calibration process, three categories are obtained, namely, full non-membership (equals 0), full membership (equals 1), and the crossover point (equals 0.5) indicating the maximum ambiguity (Ragin, 2008; Kraus et al., 2018). Since the researchers of this study had partial substantial knowledge for the scale anchors, it recommended that the anchors should be decided based on the sample distribution (Chen & Tian, 2022). Therefore, the minimum, mean and maximum as the three calibration values for each category (full non-membership, crossover point and full membership), as suggested by (Fiss, 2011; Ordanini & Maglio, 2009; Misangyi et al., 2017), were used. Table 3 presents the threshold used for each construct and category. Finally, the calibrate function within the fsQCA software was used to transform the research variables into calibrated sets based on each category (Ragin 2008).

Necessity Analysis (Research Proposition 1)

The test for necessity was conducted to investigate the first proposition. Necessity means that a condition is a superset of the outcome. Table 4 presents the necessity test for all the causal conditions (7 game elements) and their negated values (~). It is seen that all the causal conditions did not reach the minimum values suggested by Ragin (2006), namely 0.90 for consistency and 0.75 for coverage, for a condition to be necessary. This implies that no single causal condition could fully explain the outcome variables (Elçi & Abubakar, 2021). Thus, proposition 1 is supported, affirming that there is no single variable (game element) that can lead to students perceiving gamification useful in the course; rather, there exist multiple, equally effective configurations (combination) of causal conditions (game elements).

Sufficient Condition Analysis (Research Proposition 2)

fsQCA computes three solution, namely complex solution, parsimonious solution, and intermediate solution. Fiss (2011), in this context, pointed out that combining the parsimonious and intermediate solutions can offer a more detailed and aggregated view of the findings. Therefore, as suggested by Pappas and Woodside (2021), this present study highlighted the conditions of the parsimonious solution that also appear in the intermediate solution. This resulted in an intermediate solution with core conditions (i.e., appearing in both parsimonious and intermediate solutions) and with peripheral conditions (i.e., appearing only in the intermediate solutions). Accordingly, Table 5 presents the 10 obtained solutions (i.e., combination of configurations) leading to students’ perceived usefulness of gamification in the course. Particularly, the presence of a configuration is represented with a black circle (•). The absence or negation of a configuration, on the other hand, is represented with a crossed-out circle (⊗), as the two terms have been used interchangeably (Pappas, 2018). To differentiate between core and peripheral conditions, large and small circles were used, respectively.

As shown in Table 5, ten solutions are obtained that can lead to students’ perceived usefulness of gamification in the course. Particularly, Table 5 shows that the obtained model (all the ten solutions together) scored above the minimum values for consistency and coverage which are .75 and .60, respectively (Ragin, 2000; Schneider & Wagemann, 2007). Therefore, the obtained model (including the 10 solutions) is considered acceptable and informative. Thus, proposition 2 is supported, showing that more than one combination of game elements (solution) with different game elements (configurations) might help in explaining students’ perceived usefulness of gamification in the course.

Altogether, these ten solutions allowed the identification of 64% of students’ perceived usefulness of gamification in the course. Particularly, it is seen that, based on Table 5, solutions 1–4 did not involve any core game element, and it combines only peripheral elements. Additionally, compared to the other solutions, solution 8 is considered the easiest to implement since it involves only two elements, namely badge and feedback. Finally, it is seen that badge was the most present game element, where it was present in 8 among 10 solutions. Particularly, it was present as a core element in 5 among these 8 solutions.

When looking at the unique coverage, which refers to the proportion of an outcome that can be attributed solely to that solution, Table 5 reveals that solution 5 has the highest unique coverage (.04), followed by solution 7 (.03), solutions 8 (.02) and solution 10 (.01). Each of the solutions is described below.

Solution 5: This solution considers leaderboard, points and level as core elements so that students perceive gamification useful in the course.

Solution 7: This solution considers badge, progress bar and chat as core elements, and this better to be with the use of level (i.e., not core element) so that students perceive gamification useful in the course.

Solution 8: This solution considers badge and feedback as core elements so that students perceive gamification useful in the course.

Solution 10: This solution considers leaderboard, badge and level as core elements, and this better to be with the use of progress bar (i.e., not core element) so that students perceive gamification useful in the course.

Finally, based on the obtained findings (from proposition 1 and 2), this present study answers the aforementioned research question, and reveals that the implementation of a single game element does not lead to perceiving gamification useful in education. It also reveals that there is no single combination of game elements to perceive gamification useful in education. On the contrary, various combinations of game elements, 10 solutions specifically, could lead to perceiving gamification useful in education, where each solution has core and peripheral game elements.

Research Proposition 1

No single causal condition (game element) leads to students perceiving gamification useful in their gamified course. This denotes that offering meaningful learning experiences in gamified courses requires incorporating a set of game elements. This can be explained in terms of the gamification types, that is, structural and content gamification (Garone & Nesteriuk, 2019) in which several game Mechanics, Dynamics, and Aesthetics (MDA) (Kusuma et al., 2018) should be combined to keep students motivated and engaged. Toda et al. (2019) also reported five dimensions (Performance/measurement, Ecological, Social, Personal, and Fictional) that should be considered when designing gamification. Therefore, a single game element cannot cover all gamification dimension to be considered useful in education, hence requiring a combination of game elements. Specifically, using a group of game elements instead of relying on a single element is a common presumption in vast majority of educational gamification research (Dichev, & Dicheva, 2017). Furthermore, this is in accordance with what Koivisto and Hamari (2019) recommended stating that “Future gamification research should seek to diversify the use of gameful affordances, and concurrently develop an understanding of what constitutes and creates gameful experiences.” While the finding of the present study is in favor of using a set of game elements for a useful gamification, a recent study found out that individual game elements are adequate for motivating students in specific learning activities, namely gamified quizzes (Mazarakis & Brauer, 2023). This further opens-up discussion about in which educational gamification context a single game element or a set of game elements might work better.

Research Proposition 2

No single configuration (set of game elements) leads to students perceiving gamification useful in their gamified course. This refers to the potential solutions with different configurations to explain students’ perceived usefulness of gamification in the course. According to the theory of gamified learning, and the complexity of designing effective gamified solutions, game elements must be connected to particular behavioral or motivational outcomes, which require links to the learning outcomes (Landers et al., 2017). Therefore, it is not possible to assure such successful implementation with only one combination (set) of game elements in a gamified course. Particularly, badges were the most prominent game element which appeared in 8 (among 10) solutions, and as a core element in 5 among these 8 solutions. Digital badges have the potentials with distinguished characteristics to motivate learners, present achievement level, provide indicators of achievement and accomplishment, and credentials and assessment tools (Gibson et al., 2015). This finding encourages using badge as a promising element in the gameful design as it was often overlooked generally in Computer Science Education (Narasareddygari et al., 2018).

While the current study revealed 10 potential, gamified solutions, incorporating two or more game elements in such solution, there are promising solutions with highest unique coverage, namely: solutions 5, 7, 8, and 10. Therefore, this research suggests using both [leaderboards, points and levels] as core elements and/or [badges, progress bars, and chats] as core elements with optional and preferable use of [levels], in computer science courses, especially in the UML course. This leverages the importance of using achievement/progression affordances besides social affordances (e.g., chats and teams) when designing useful educational gamification (Majuri et al., 2018). Another solution revealed the need for using [leaderboards, badges and levels] as core elements with possibilities of using [progress bar], and this confirms the consistency of the progression and achievement game mechanics with a convinced relationship to points and/or badges that can be combined with both leaderboards and levels together as reward-based gamification.

Finally, this study revealed an easy yet useful educational gamification by combining only two core game elements, namely badge and feedback (solution 8). This asserts that utilizing feedback with digital badges when implementing mastery learning approaches (Besser & Newby, 2019) can lead to a positive impact on students’ motivation (Mazarakis & Brauer, 2023). This solution (solution 8), on the other hand, raises concerns in terms of the number of game elements used and whether incorporating more game elements can lead to a better learning experience or increase students’ cognitive load. In this context, Dichev and Dicheva (2017) also raised the question “do more game elements produce better results than less?”. Therefore, more research is needed to investigate how the number of used game elements in an educational gamification might impact learning outcomes.

Interestingly, although the common use of points, badges, and leaderboards (PBL) is addressed in several studies (Dichev & Dicheva, 2017), the findings of this current study revealed that it is not necessarily to use PBL in combination to impact the students’ perceived usefulness of gamification. However, PBL can be used with other elements as presented in solutions 2 and 3. This is in accordance with the debate of using PBL in education in which there are no consensus to decide whether it can promote positive or mixed results (Metwally et al., 2021).

Implications

Limitations and Future Research Directions

This study has some limitations that should be acknowledged and further researched. For instance, this study covered only seven game elements, Therefore, future research could involve more game elements, including structural and content gamification, and investigate the potential combination of game elements that can lead to students’ perceived gamification usefulness in education. Additionally, despite that a sample size of 83 is considered sufficient for an fsQCA analysis (Greckhamer et al., 2008; Marx, 2010), this, however, might limit the generalizability of findings. Finally, this study relied on perception, which is a subjective process influenced by individual differences, meaning how one person perceives something will be totally different from another. Therefore, future research should rely on more metrics and consider students’ individual differences when investigating their perceived usefulness of gamification.

Future research could, therefore, reconduct the experiment with more students and taking into consideration their individual differences (e.g., personality, player style) to see how these differences might impact the obtained combination of game elements. Besides, future research could also focus on implementing these 10 obtained gamification solutions in a given course or field other than computer science (e.g., social sciences, health sciences, etc.) and investigating their impact on students’ learning outcomes. This could help to compare all these solutions in different educational subjects, draw empirical findings about how effective they are, and further generalize the obtained results. In line with this, to further validate our results, our future directions aim to further interview the students and seek their feedback about the obtained combination of game elements, thereby understanding whether these obtained combinations would work or not from their perspective.