Psychometric Validation of the Gaming Disorder Scale for Adolescents (GADIS-A) in Dutch Among Flemish Adolescents

Procedure

To validate the GADIS-A scale in Dutch, a survey was distributed in 13 Flemish high schools between November 2021 and February 2022. The survey included questions about video gaming, simulated gambling, monetary gambling, mental health outcomes and personality traits. Adolescents took part online or on paper, either during class time or in their free time. Parents were invited to take part in the study online through the schools’ mailing system.

Three different versions of the adolescent survey were used to reduce completion time. Respondents were divided across the three versions equally, at random. All versions included questions on video gaming, mental health outcomes, and personality traits, as well as socio-demographic questions (such as gender and age). Next to that, the GADIS-A was included in the core set of each version of the survey. The remaining part of the survey contained additional measures, focusing on either (1) video gaming (version A), (2) personality traits (version B), or (3) persuasive media content (version C; not used in this particular results presentation). Next to that, parents filled out a similar survey.

Participants

In total, 2443 adolescents and 823 parents took part in the study. Only adolescents who filled out all GADIS-A items were analyzed, resulting in 1773 respondents (M age = 13.98, SD = 1.48). 52.5% of the final sample identified themselves as male, 45.9% as female, and 1.5% as “other” or left the question blank. Only parents who could be matched to their child and who completed the GADIS-P (and of whom their child had completed the GADIS-A) were considered, resulting in 147 parent-child dyads. Parents’ mean age was 44.71 years (SD = 5.82). 78.8% of parents identified themselves as female, 21.2% as male (one parent left this gender question blank). Children in these parent-child dyads mostly identified themselves as male (66.0%, 32.7% female, 1.4% other).

Measures

A variety of measures were employed to validate the usability and validity of the Dutch version of the GADIS-A. Table 1 describes the used measures, their n, the corresponding version of the survey in which the variables were used, and Cronbach’s α and McDonald’s ω for each scale.

Statistical analysis

Skewness (absolute value of > 2.0 for substantial non-normality) and kurtosis (> 7.0) were calculated to test for the normality distribution (Kim, 2013). Exploratory Factor Analysis (EFA) was executed on the Pearson correlation matrix in R in order to determine the usability of the proposed two-factor structure (Paschke, Austermann & Thomasius, 2020). The Kaiser-Meyer-Olkin (KMO) criterion and Bartlett’s Test of Sphericity were used to assess suitability for factor analysis. Next, confirmatory factor analysis (CFA) was performed using lavaan in R (Rosseel, 2012), based on the covariance matrix derived from Pearson correlations among the standardized observed variables. Goodness of fit was evaluated using the chi-square value, the standardized root mean squared residual (SRMR), the root mean square error of approximation (RMSEA), the Tucker-Lewis Index (TLI), and the root mean square residual (RMSR). Lastly, the comparative fit (CFI) was calculated. We employed the following criteria to determine model goodness of fit: RMSR ≤ 0.05, and following the original GADIS-A study (Paschke, Austermann & Thomasius, 2020): RMSEA < 0.1, SRMR < 0.08, TLI ≥ 0.95, CFI ≥ 0.95. Sum scores were calculated for each factor scale.

Internal consistency was assessed using Cronbach’s α and McDonald’s ω, with scores ≥ 0.70 deemed acceptable, ≥ 0.80 deemed good, and ≥ 0.90 deemed excellent (Nunnally & Bernstein, 1994). To assure the construct validity of the GADIS-A, Pearson and Spearman rank correlations with video gaming, personality traits, social life and school life, and gambling were assessed. To test the criterion validity of the GADIS-A, partial correlations with mental health outcomes were checked, whilst controlling for gender and age. For these correlations, the following criteria were used: r ≤ 0.29 for weak correlations, 0.30 ≥ r ≤ 0.49 for moderate correlations, and r ≥ 0.50 for large correlations (Cohen, 1988).

Descriptive numbers regarding GADIS-A and GADIS-P scores are provided, including gender differences for the total GADIS-A and both subscales. Because of the low number of respondents in the “other” or missing category, only differences between boys and girls are computed. The cut-off scores, proposed by Paschke, Austermann, and Thomasius, (2020), will be employed to determine the prevalence of gaming disorder in our sample. These cut-off scores include a score of ≥ 12 on the total GADIS-A scale, a score of > 5 on the first subscale (negative consequences), a score of > 9 on the second subscale (cognitive behavioral symptoms), and experiencing problems at least for longer periods or daily.

Ethics

Parents and legal caretakers of the students in participating schools were informed about the study via mail, prior to the start of the survey, and gave passive consent for their child’s participation. Only students who were not deregistered were presented with the survey. Respondents were asked to agree to an informed consent before continuing with the survey. This informed consent stated that participation was completely voluntary, and that all responses remained confidential. Respondents were able to stop their participation, without consequences. The procedure received ethical approval via the Social and Societal Ethics Committee at KU Leuven (G-2021-3439-R2(AMD)), and has been performed in accordance with the Declaration of Helsinki.

Factor structure

KMO was 0.88, with Bartlett’s Test of Sphericity being significant (p < 0.001), meaning the data was suitable for factorial analyses.

Based on the scree plot, the Kaiser criterion (eigenvalues > 1), and Parallel Analysis (Patil et al., 2017), a two-factor solution was proposed. Because we expected the two factors to be correlated, we opted for a promax rotation, using the Minimum Residual (MINRES) method. The eigenvalue was 2.33 for the first factor and 1.97 for the second factor, with communalities ranging from 0.43 to 0.57. One item (item 3) showed a rather low communality of 0.20. However, since the item-total correlation of this item was 0.38, it was retained in the scale. The proposed two factors explained a total variance of 0.48 (variance of factor one = 0.26, factor two = 0.22). Factor loadings ranged between 0.32 and 0.84 for factor one and 0.36 and 0.85 for factor two. Item 4 had similar loadings on both factors (0.39 on factor one and 0.36 on factor two). Here, we followed the reasoning of the original scale, including it in the second factor (Paschke et al., 2020). The model demonstrated good fit (empirical χ²(19) = 150.86, p < 0.001, RMSEA = 0.089, RMSR = 0.030, TLI = 0.910).

CFA models were estimated using maximum likelihood (ML) estimation. CFA applying the two-factor solution revealed a good fit to the data (standard χ²(22) = 271.29, p < 0.001, RMSEA = 0.078, SRMR = 0.040, CFI = 0.957, TLI = 0.929). Item 1 & 2, 4 & 6, 4 & 9 and 6 & 9 were given error correlations in the model, because they used similar expressions. A confirmatory two-factor model with the higher order factor was not identified because the two factors were too highly correlated. Figure 1 describes the CFA-factor loadings for both factors.

Figure 1

In conclusion, the same two-factor model from the original article (Paschke, Austermann & Thomasius, 2020) emerged from our sample, with five items mirroring the negative consequences dimension (item 3, 6, 7, 8, and 9), and four items reflecting the cognitive-behavioral symptoms (item 1, 2, 4, and 5). EFA- and CFA-factor loadings are presented in Table 3.

To test for scalar invariance, the model with equal loadings and intercepts was compared to the unconstrained model. The CFI dropped from 0.95 to 0.93 (ΔCFI = 0.02), exceeding the recommended 0.01 threshold (Cheung & Rensvold, 2002). This suggests that scalar invariance does not hold, and comparisons of observed means between boys and girls may be biased. However, further inspection revealed that two items in particular (item 1 and item 2) did not display scalar invariance. When unconstraining the intercepts and factor loadings of these two items (thereby keeping the intercepts and loadings of all other items constrained), the fit of the model did not differ significantly from the fully unconstrained model (ΔCFI < 0.01). So, with the exception of only two items, there was evidence for scalar invariance across gender.

Internal consistency

For the total GADIS-A scale (nine items without the temporal issue), a Cronbach’s α of 0.85 and a McDonald’s ω of 0.85 were measured, showing good internal consistency. The first subscale showed an acceptable Cronbach’s α of 0.78 and a McDonald’s ω of 0.78. The second subscale also showed an acceptable Cronbach’s α of 0.78 and a McDonald’s ω of 0.78 (see also Table 1).

Construct validity

All items and scales were distributed normally, except for gaming time (school days and weekend days) and online monetary gambling frequency. Table 4 shows the bivariate correlations between the different items and scales and total GADIS-A score. Correlations for the two subscales (negative consequences and cognitive-behavioral symptoms) were similar to correlations with the total GADIS-A score, which is why only the latter are reported here. This also coincides with the strategy used in the original GADIS-A scale development (Paschke, Austermann & Thomasius, 2020).

Criterion validity

Regarding mental health outcomes, we checked for partial correlations, whilst controlling for gender and age. Life satisfaction correlated negatively with total GADIS-A score (r = –0.25, p < 0.001), while depression (r = 0.24, p < 0.001) and anxiety (r = 0.18, p < 0.001) correlated positively. These correlations were small in size.

Data

On average, respondents scored 7.88 on the GADIS-A (SD = 6.33). Boys (M = 9.40, SD = 6.27) scored significantly higher than girls (M = 6.02, SD = 5.72) (t(1732) = 11.68, p < 0.001; Cohen’s d = 0.562). Age was negatively correlated with total GADIS-A score, albeit very weakly (r = –0.097, p < 0.001). The average score on the GADIS-P was 10.25 (SD = 7.03).

Mean score on the first subscale, negative consequences, was 3.12 (SD = 3.43). Boys scored significantly higher than girls (t(1727.31) = 10.29, p < 0.001; Cohen’s d = 0.489). Mean score on the second subscale, cognitive-behavioral symptoms, was 4.76 (SD = 3.55). Again, boys scored significantly higher than girls (t(1732) = 10.85, p < 0.001; Cohen’s d = 0.522). Lastly, mean score on the temporal item was 0.49 (SD = 0.62), with boys scoring significantly higher than girls (t(1727.05) = 10.40, p < 0.001; Cohen’s d = 0.494). All scores are shown in Table 5.

Limitations and future research

Only a small number of respondents in our sample reached the cut-off score for gaming disorder, making it hard to make statements about this group or compare it to non-disordered video game players. Therefore, future research could validate the GADIS-A in a sample of heavy or disordered video game players. While we only made a distinction between disordered and non-disordered video game players, future studies could focus on highly frequent gamers who may be at risk as well. Moreover, since the translation of the GADIS-A is the first scale in Dutch language to use the ICD-11 criteria for diagnosing Gaming Disorder, the only way to compare it with a validated scale was to use a formerly used scale based on DSM 5 criteria, which differs from the ICD-11 criteria. Future studies with gaming disordered adolescents could include a comparison of the GADIS-A scores against semi-structured clinical interviews. Next, similar to the original paper by Paschke, Austermann, and Thomasius (2020), some items of the GADIS-A were intercorrelated. This was mainly due to the similar wording of some of the items (such as the use of “neglect” in both item 4 and 6). These correlations should not be taken lightly, and future research could benefit from slightly rewriting the items, in order to avoid such correlations. While the GADIS-A proposes a two-factor structure for disordered video game play, this structure does not directly map onto the three core diagnostic criteria of the ICD-11 conceptual framework. Moreover, some of the labels are not entirely consistent with the content of the items (e.g., item 3 (loss of interest in other activities), classified under “negative consequences”, more directly reflects Criterion B (“increasing priority”)). Future research could therefore reevaluate both the factor structure and the labels of the proposed two-factor model. On top of that, since the same sample was used for both EFA and CFA, additional studies are needed to accumulate further evidence of validity, as a single study cannot serve as sufficient justification for the validity of the questionnaire. Next, our results pointed towards a positive correlation between parental and adolescent ratings of disordered video game play. While this correlation implies some level of agreement or shared perception between parents and children regarding these traits, it could potentially be influenced by common environmental, behavioral, or genetic factors. Therefore, further analysis and contextualization are needed to understand the meaning of this correlation for the parent-child relationship. Lastly, based on the only moderate correlation between GADIS-A and GADIS-P scores, the results of future studies in the Flemish population may be dependent on the informant; the usage of multiple informants could thus strengthen the reliability of these studies.