Table 1
Eligibility procedures for the published works included in this study.
| INCLUSION CRITERIA |
|---|
|
|
|
|
|
Figure 1
Flowchart of review and selection procedures (PRISMA).
Figure 2
Distribution of the examined studies by year of publication, type, and method.
Table 2
Methodological trends in gamification studies.
| DATA COLLECTION TOOLS | F | TYPES OF ANALYSIS | F | LENGTH OF APPLIED STUDY (DAYS) | F |
|---|---|---|---|---|---|
| Questionnaire | 33 | Comparative analysis | 58 | 4–9 | 2 |
| Interview | 31 | Relational analysis | 27 | 10–49 | 24 |
| Scale | 25 | Descriptive analysis | 21 | 50–100 | 17 |
| Test (performance, achievement) | 18 | Content analysis | 20 | 101–300 | 9 |
| System records (logs) | 12 | ||||
| Student diaries | 8 | ||||
| Self & peer assessment tools | 3 | ||||
| Monitoring | 2 | ||||
| Group discussions | 2 |
Table 3
Learning outcomes discussed in academic studies.
| LEARNING OUTCOMES | STATISTICALLY SIGNIFICANT POSITIVE CHANGE | NO STATISTICALLY SIGNIFICANT POSITIVE CHANGE | TOTAL |
|---|---|---|---|
| Cognitive | 25 | 12 | 37 |
| Academic achievement | 21 | 9 | 30 |
| Learning (qualitatively observed learning outcomes) | 4 | 1 | 5 |
| Task completion | 0 | 1 | 1 |
| Writing performance | 0 | 1 | 1 |
| Affective | 78 | 13 | 91 |
| Participation | 29 | 3 | 32 |
| Motivation | 22 | 6 | 28 |
| Attitude | 10 | 1 | 11 |
| Interaction | 6 | 3 | 9 |
| Satisfaction | 5 | 0 | 5 |
| Interest | 3 | 0 | 3 |
| Perception | 2 | 0 | 2 |
| Self-efficacy | 1 | 0 | 1 |
| Total | 103 | 25 | 128 |
Table 4
Effects of gamification categories and elements in studies.
| CATEGORY | ELEMENTS OF GAMIFICATION | N | REMARKABLE RESULTS WITH CATEGORY | PUBLICATION CODE |
|---|---|---|---|---|
| Reward | Badges/Trophy | 49 | * The most commonly used gamification item in each subject area * Reward elements mostly associated with participation and motivation variables. * Students appear to find the leader boards motivating, which triggers senses of competition and fun. * Badges are the most frequently mentioned motivation element by the users. * The negative effects of gamification can be seen when gamification scores are reflected on the real scores (Y16) and competition has become dominant (Y58). | P1, P2, P3, P4, P5, P6, P7, P8, P9, P10, P11, P12, P13, P14, P16, P17, P18, P19, P20, P21, P22, P23, P24, P25, P26, P27, P28, P29, P30, P31, P32, P33, P34, P35, P36, P37, P38, P39, P40, P41, P43, P44, P45, P46, P47, P48, P49, P50, P51, P52, P53, P54, P55, P56, P57, P58, P59, P60, P61, P62, P63 |
| Collection | 2 | |||
| Virtual items | 1 | |||
| Leader boards | 40 | |||
| Scores | 36 | |||
| Surprises | 1 | |||
| Mastery | Levels | 28 | * Mastery elements are mostly associated with academic achievement and participation variables, creating a statistically significant difference. * Gamification elements in this category are influential in terms of performing more activities and visiting learning environments more often. * Students have been reported to attribute more meaning to the elements in this category. * Giving tasks of interrelated difficulty that would increase the sense of achievement have turned out to be effective in the results. * Elements, such as success and certificates, and social status have been highlighted and new relationships created. | P2, P4, P7, P8, P9, P11, P13, P16, P18, P19, P20, P22, P24, P26, P27, P28, P30, P31, P32, P34, P39, P40, P41, P42, P43, P44, P46, P47, P48, P49, P51, P52, P53, P54, P55, P56, P57, P59, P60, P61, P62 |
| Progress and progress bar | 20 | |||
| Experience score | 5 | |||
| Achievement/Acquisition | 3 | |||
| Certificate | 3 | |||
| Time limit (challenge) | 3 | |||
| Task/Target | 2 | |||
| Autonomy | Avatar | 3 | * The ability of the students to customize their avatars by unlocking them has made the applications more adaptive. * The customization of the content for different learner groups has been shown to increase the effectiveness of the studies. | P11, P13, P16, P25, P28, P40, P42 |
| Customization | 1 | |||
| Fun activities | 3 | |||
| Relatedness | Teams | 3 | * These rarely included elements can create new relationships in the online environment. * The cooperation of small groups in teams and the opportunity to socialize in the studies have positive effects on social presence. * Elements of this category are often regarded by users as unnecessary or tools of which they were not even aware. For these reasons, they have caused challenges in determining the impact of the learning outcomes. | |
| Social graphics | 2 | |||
| Gifts | 1 | |||
| Statuses | 1 | |||
| Likes | 2 | |||
| Role-playing | 1 |
Figure 3
Distribution of learning outcomes by gamification categories.
Table 5
The relationship between gamification and learning outcomes according to environmental variables.
| THEMES | FEATURES | C | A | TOTAL NUMBER OF STUDIES | REMARKABLE RESULTS OF THE STUDIES | PUBLICATION CODE |
|---|---|---|---|---|---|---|
| LEARNING OUTCOMES*NLO/ SLO | ||||||
| Type of subject area | Personal and professional development | 12/7 | 33/26 | 24 | * Although the gamification applications in online education by and large seems to belong to personal and professional development, most of the studies have been carried out in the field of informatics since there are trainings on information and communication technologies in 10 studies in this category. * In the field of personal and professional development, the participants are generally adults and volunteers, and it is seen that the badges used in these environments mostly reduce intrinsic motivation (P45, P23). In such environments, expectations exist of gamification applications that will strengthen autonomy rather than providing a reward (P46). * Although their number is very limited, gamification components suitable for autonomy and relatedness have been used in the fields of personal development and economics (P34, P42, P54, P9, P59). | P7, P10, P11, P12, P20, P21, P22, P23, P25, P31, P33, P34, P40, P41, P42, P46, P47, P49, P50, P52, P54, P55, P57, P60 |
| Information and communication technologies | 11/6 | 32/26 | 21 | P2, P4, P5, P8, P14, P16, P17, P18, P19, P26, P28, P30, P35, P38, P39, P51, P56, P61, P63, P64, P65 | ||
| science | 2/2 | 2/2 | 2 | P13, P24 | ||
| Aviation and space | 2/2 | 4/3 | 3 | P15, P43, P58 | ||
| economics | 1/1 | 4/4 | 3 | P9, P32, P59 | ||
| Natural sciences | 3/3 | 3/3 | 4 | P3, P27, P29, P37 | ||
| Transportation | 1/1 | – | 1 | P62 | ||
| Application method | Online courses given as part of formal education | 17/12 | 33/28 | 29 | * Most of the studies have been conducted in formal online learning environments. * MOOCs (mass open online courses) have been covered in very few publications and no theses have been found on this subject. * Although it seems that full success has been achieved in MOOCs, it is difficult to generalize the results for all participants as it is methodologically limited to only those individuals who respond to the measurement tools. * Quasi-experimental studies often seem to be carried out in online courses in formal education programs with the indications of statistically significant differences among groups. In studies with a control group, no significant difference has been found between the groups in terms of many of the variables (P16, P21, P28). * More autonomy and relatedness have been used in online courses given within the scope of formal education compared to other environments. | P3, P5, P11, P14, P20, P24, P25, P30, P31, P33, P35, P39, P41, P45, P47, P51, P52, P53, P54, P55, P56, P57, P59, P60, P61, P62, P63, P64, P65 |
| Distance education program | 11/6 | 35/29 | 24 | P2, P4, P7, P9, P10, P12, P13, P15, P18, P22, P23, P26, P32, P38, P40, P42, P43, P44, P46, P48, P49, P50, P58 | ||
| MOOCs | 2/2 | 11/11 | 7 | P1, P6, P8, P19, P27, P29, P36, P37 | ||
| Blended learning | 2/1 | 9/6 | 4 | P16, P17, P21, P28 | ||
| Technology used | Web applications | 15/12 | 45/37 | 32 | * Web applications are generally sharing tools, such as discussion forums and social networks, and a reward system is used according to the content contribution of students. * MOODLE and its plugins appear to be mostly used in open-source software. * Paid LMSs can offer automatic tracking and feedback which are part of mastery. However, those environments use reward systems more than mastery-based apps. | P2, P3, P7, P9, P11, P13, P14, P15, P17, P18, P19, P21, P25, P29, P30, P32, P33, P34, P35, P43, P45, P48, P49, P51, P52, P54, P58, P59, P60, P61, P63, P65 |
| Open-source LMS | 10/5 | 23/19 | 18 | P1, P4, P16, P20, P22, P23, P26, P28, P31, P38, P39, P40, P44, P47, P50, P57, P62, P64 | ||
| Commercial LMS | 7/3 | 13/10 | 10 | P2, P10, P12, P13, P24, P41, P42, P53, P55, P56 | ||
| Sample size | 9–54 | 11/7 | 33/30 | 26 | * Although sample sizes do not make a noticeable difference in terms of outputs, they bring about some differences in terms of implementation processes. * While the applications made in distance education courses in formal learning are mostly in the range of nine to 200 people, the number of samples in MOOCs is generally over 500. *In samples below 60, a feedback-based and participatory gamification process is dominant, requiring students to complete various tasks. *In samples with the number of participants over 100, the gamification process is more integrated with learning management systems, providing more automated feedback and motivation. | P6, P9, P10, P11, P12, P14, P19, P22, P25, P30, P31, P38, P39, P40, P42, P44, P45, P46, P47, P51, P52, P53, P56, P58, P59, P62 |
| 55–99 | 8/6 | 22/16 | 13 | P13, P16, P17, P18, P21, P24, P28, P32, P55, P57, P60, P63, P65 | ||
| 100–294 | 10/7 | 18/18 | 14 | P3, P4, P5, P7, P20, P23, P26, P34, P36, P41, P43, P5, P61, P64 | ||
| 295 & 295+ | 4/2 | 13/12 | 12 | P1, P2, P8, P15, P27, P29, P33, P35, P37, P48, P49, P50 | ||
| Educational background of the sample group | Primary school | 2/2 | 2/2 | 2 | * It appears that most of the studies have been conducted with undergraduate students (mostly first-year students). * Factors, such as creating a competition and giving badges at the undergraduate level, cause more anxiety than at the graduate level (P58). * Some studies conducted on distance education at the primary and secondary school levels have actually focused on gamification within a certain multimedia application, which are used face-to-face outside the classroom environment. * Gamification in MOOCs is mostly carried out with the aim of keeping users in the system and, in this sense, the results are generally positive. | P43, P62 |
| Secondary school | 4/4 | 8/6 | 5 | P7, P21, P24, P25, P43 | ||
| High school | 2/2 | 5/4 | 4 | P25, P35, P59, P64 | ||
| Associate degree | 1/– | 2/1 | 2 | P56, P65 | ||
| Bachelor’s degree | 20/13 | 51/42 | 33 | P2, P3, P4, P5, P9, P11, P13, P14, P16, P17, P18, P20, P22, P23, P26, P28, P30, P32, P33, P38, P39, P40, P41, P46, P47, P48, P51, P54, P55, P57, P58, P60, P61, P63 | ||
| Master’s degree | 1/– | 8/7 | 7 | P10, P31, P33, P42, P45, P53, P54 | ||
| teacher | 1/1 | 4/3 | 3 | P12, P25, P52 | ||
| MOOC user | 2/2 | 12/11 | 9 | P1, P6, P8, P15, P19, P27, P29, P36, P37 | ||
| Other | 3/1 | 2/2 | 4 | P34, P44, P49, P50 | ||
| * NLO (Number of Learning Outcomes), SLO (Significant Learning Outcomes), C (Cognitive), A (Affective) | ||||||