Causality Orientations at Work Scale in the Finnish Working Context

Participants and procedures

Adults (at least 18 years old) fluent in Finnish and with any job were invited to participate in a survey via a market research company. Participants completed the study questionnaire online after providing their informed consent. The study protocol was approved by the ethical review board of the first author’s university. Participants received a small fee from the market research company. See the description in the section on statistical analysis for further information on the assessment of careless responding.

Of the 742 respondents reporting having a supervisor, 641 were deemed careful respondents and were included in the study sample. Only the respondents who had a supervisor were included because several of the vignettes in COWS contain some action of one’s supervisor. Mean age of the sample was 40.4 years (standard deviation = 12.2 years). Most identified as women (54%) or men (45%), with about 1 percent being non-binary. Regarding occupational status, over half of the participants were manual workers (52%), with most of the others being non-manual workers (25%) or non-manual workers with managerial duties (22%). The rest of the participants (<1%) were entrepreneurs, in a family business, or could not assess their occupational status.

Measures

Work-related causality orientations were measured with the Finnish translation of the COWS (Halvari & Olafsen 2020). It consists of six vignettes set in the working context (e.g., ‘Your manager has asked you to do something about your interest in work, which has been a bit low lately’). The respondent rates with three items how likely different responses would be for them in each scenario (1 = very unlikely, 7 = very likely). One item describes autonomy orientation (e.g., ‘I want to talk to my manager to see if we can find a good solution’), one describes controlled orientation (‘Feel pressured to do something so my manager is satisfied’), and one describes impersonal orientation (‘Think that, to be honest, I cannot really change my basic interest in work’). For all the vignettes and items, see Halvari & Olafsen (2020). Throughout this paper, we use the vignette numbering of their Table 2 for comparability.

The COWS was translated from the original Norwegian into Finnish by a professional translator. The Finnish translation was then translated back into Norwegian by another professional translator. The final Finnish version to use in the present study was decided in discussions of the first author and the original developers of the scale. These discussions proceeded in English because the participants did not share a common native language.

Satisfaction of the basic psychological needs of autonomy, competence, and relatedness was measured with the Basic Psychological Need Satisfaction and Frustration Scales adapted for the working context (B. Chen et al. 2015; Van der Kaap-Deeder et al. 2020). Each of the three scales consists of four items (e.g., ‘At work, I feel a sense of choice and freedom in the things I undertake’ for autonomy; ‘I feel confident that I can do things well on my job’ for competence; ‘I feel that the people I care at work about also care about me’ for relatedness). Participants respond on a scale from 1 = strongly disagree to 7 = strongly agree.

Intrinsic motivation and amotivation were measured with their respective three-item scales of the Multidimensional Working Motivation Scale (Gagné et al. 2015; Trépanier et al. 2023), which includes a stem about reasons to put effort into work and items such as ‘Because the work I do is interesting’ for intrinsic motivation and ‘I don’t know why I’m doing this job, it’s pointless work’ for amotivation. The items are responded to on a scale from 1 = not at all to 7 = completely.

Vigour was measured with the three-item scale of the Utrecht Work Engagement Scale (Schaufeli et al. 2002) e.g., ‘At my work, I feel bursting with energy’), responded on a scale from 1 = never to 7 = always. Exhaustion was measured with the eight-item exhaustion scale of the Burnout Assessment Tool (e.g., ‘At work, I feel mentally exhausted’), responded to on a scale from 1 = never to 5 = always (Schaufeli et al. 2020).

Statistical analyses

Demographic information, descriptive statistics, and careful responding were examined using IBM SPSS Statistics (Version 28). Before any other analyses, careless responding was examined, such as inconsistent responses, small intraindividual response variability (Dunn et al. 2018), responding regardless of the item content (statistical synonyms) (Meade & Craig 2012), and repeated single responses (with LongString macro for Microsoft Excel) (Landers 2020).

All structural equation modelling was conducted with Mplus (Version 8.9). Both CFA and ESEM were used to test the three-factor structure of the COWS. In CFA, three factors (autonomy orientation, controlled orientation, and impersonal orientation) were defined by their respective indicators. In addition to fit indices, residual covariances suggested by modification indices were inspected.

In ESEM, three oblique factors (one for each causality orientation) were defined by their theoretically assigned indicators by freely estimating their loading on their factors, and the loading of other indicators was estimated but targeted near zero with target rotation (see Marsh et al. 2014; Morin et al. 2013). In addition to the fit indices, we observed both the main loadings and estimated cross-loadings. Cross-loadings were interpreted to suggest the possibility of construct-irrelevant variance if the indicator’s loading was statistically significant (p < .01) and its direction was contrary to the expectations (e.g., positive for the autonomy orientation indicators on the controlled orientation factor). Furthermore, cross-loadings in the expected direction (e.g., statistically significant loading of controlled orientation indicator on the impersonal orientation factor) were considered problematic if they were strong compared to main loadings or other cross-loadings.

Robust maximum likelihood estimation was utilized in all structural equation models. When assessing model fit, both comparative fit index (CFI) and Tucker–Lewis index (TLI) were considered to indicate mediocre (≥.85), acceptable (≥.90) or good fit (≥.95), and both root mean square error of approximation (RMSEA) and standardized root mean square residual (SRMR) were used to indicate mediocre (≤.10), acceptable (≤.08) or good fit (≤.05). For the criteria, see Little (2013).

Measurement invariance was tested by comparing successively more restrictive models. First, a configural invariance model was specified. A satisfactory configural invariance model suggests that in both groups (manual and non-manual workers), COWS can be depicted with the same number of latent factors with the same indicators for each factor. Should the configural invariance be rejected, COWS would measure markedly different phenomena according to occupational status. Next, metric invariance model was specified by restricting the loading of each item on its respective factor to be identical in both groups. If the metric invariance was rejected, it would suggest that respondents in the groups would interpret the items differently regarding their ability to indicate the intended factors (i.e., some items could indicate their intended causality orientations significantly better in one of the groups) (Putnick & Bornstein 2016). Next, scalar invariance model was specified by restricting the intercept of each item identical in both groups. Failure to demonstrate scalar invariance would indicate some systematic group-related bias in responses or some genuine differences in the latent means of the items. Finally, a residual invariance model was specified by restricting the residual variance of each item identical in both groups. Consequences of lacking residual invariance may not be as severe as with other forms of invariance (Vandenberg & Lance 2000), but it suggests that the groups differ in some ways regarding the residual terms of the items. When testing the models, if the statistically significant chance in chi-squared (p < .05) was accompanied by a substantial change in CFI (ΔCFI ≥.010 for all invariance levels) and RMSEA (ΔRMSEA ≥.015 for all invariance levels) or SRMR (ΔSRMR ≥.030 for metric invariance and ΔSRMR ≥.010 for scalar and residual invariance) between successive models, it was deemed to indicate non-invariance (for the criteria, see F. F. Chen 2007).

Internal consistency of the COWS was studied with IBM SPPS Statistic (Version 28). Internal consistency was measured with McDonald’s omega coefficient, based on the scores of the observed variables. Criterion validity was tested by Mplus with latent factor correlations between the causality orientations and satisfaction of basic psychological needs (autonomy, competence, relatedness), intrinsic work motivation and amotivation, vigour, and exhaustion.

Factor structure

According to the CFA, the fit of the three-factor model was mediocre or acceptable depending on the fit index (χ² = 565.20, df = 132, CFI = .879, TLI = .860, SRMR = .073, RMSEA = .072). The factor loadings were relatively strong, except for the autonomy orientation indicator in Vignette 11 (λ = .39, p < .001; for full information on the factor loadings, see Table 1).

In Table 2 are presented five residual covariances with the largest modification index values and corresponding correlations should the covariance be estimated. Most of the possible re-specifications were between indicators of controlled orientation and impersonal orientation, leading to positive correlations. Estimating residual covariance between indicators of autonomy orientation indicators of Vignettes 7 and 10 would result in a negative correlation.

According to the ESEM, the fit of the three-factor model was from acceptable to good according to all the indices except TLI, which indicated mediocre fit (χ² = 351.74, df = 102, CFI = .930, TLI = .895, SRMR = .034, RMSEA = .062). The factor loadings are presented in Table 2. There were some cross-loadings suggesting construct-irrelevant variance. On Vignette 3, the autonomy orientation indicator also loaded on the impersonal orientation factor (λ = .14, p < .001). On Vignette 7 (λ = –.11, p < .001) and 8 (λ = –.11, p = .001), the controlled orientation indicator loaded negatively on the impersonal orientation factor. On Vignettes 10 (λ = .17, p < .001) and 11 (λ = .16, p < .001), the autonomy orientation indicator loaded also on the controlled orientation factor. On Vignette 9 (λ = .27, p < .001), 10 (λ = .12, p < .001) and 11 (λ = .19, p < .001) the controlled orientation indicators loaded also on the impersonal orientation factor. Of these loadings, especially the loading of the controlled orientation indicator on the Vignette 9 (λ = .27) on the impersonal orientation factor is relatively strong compared to the theoretically assigned main loading of the indicator on the controlled orientation factor (λ = .52). The model estimated with ESEM was chosen for further examination. Despite its weaknesses, we considered it theoretically and empirically more appropriate compared to the CFA model with need for extensive content modification (e.g., omitting vignettes or items) and respecifications (e.g., estimating several residual covariances). We interpreted the fit as satisfactory, and thus Hypothesis 1 was supported.

Measurement invariance between groups

Measurement invariance was examined between manual workers and non-manual workers. Configural invariance was from acceptable to good notwithstanding TLI, which indicated mediocre fit (χ² = 464.76, df = 204, CFI = .928, TLI = .892, SRMR = .037, RMSEA = .064). Because the fit was not appreciably worse than the fit of the combined sample, invariance testing was continued.

Metric invariance was supported (Δχ² = 56.71, p = .113). Scalar invariance was supported because, despite a statistical change in chi-squared, the change in CFI did not exceed the set criteria (Δχ² = 49.22, p < .001, ΔCFI = –.009). Residual invariance was likewise supported (Δχ² = 33.78, p = .013, ΔCFI = –.004). For full information on model fits, see Table 3. Hypothesis 2 was supported.

Internal consistency and criterion validity

Internal consistencies, based on the composite scales of the observed variables, are shown in Table 4. Internal consistency of COWS scales was ω ≥.80 for controlled orientation and impersonal orientation, autonomy orientation being slightly lower. Hypothesis 3 was mostly supported.

Correlations among the causality orientation scales, based on the latent factors, were in the expected directions and of expected magnitudes (Hypothesis 4a and 4b). The correlations between causality orientations and criterion variables, based on the latent factors, were in the expected directions. Autonomy orientation was positively associated with satisfaction of all three needs, intrinsic motivation, and vigour, and negatively with amotivation and exhaustion (Hypothesis 4c). Both controlled orientation and impersonal orientation were associated with the criterion variables in reverse direction compared to the autonomy orientation (Hypothesis 4d). For full information on descriptive statistics, internal consistency, and correlative associations, see Table 4. Hypotheses 4a–4d were fully supported.

Exploratory structural equation modelling facilitates the understanding of weaknesses of the factor structure

Regarding the factor structure, the theoretically expected three-factor structure did not fit the data at a satisfactory level with conventional CFA estimation. This was not a surprise as such, because in the original validation study, some re-specifications in the form of residual covariances had to be estimated. The modification indices in the present study also suggested some possible re-specifications. However, it has been argued that such re-specifications should not be applied mechanically, and their implications should be discussed (Cole et al. 2007). Residual covariances imply that some factor that is not measured in the study is behind associations of the indicators. Of the five suggested re-specifications with the largest modification index values in the present study, the positive residual correlation between the controlled orientation indicators of Vignette 3 and 7 could be due to the identical wording of the indicator. The perhaps surprising negative residual correlation between two indicators of autonomy orientation in Vignettes 7 and 10 may be caused by the differences in the scenarios depicted, as Vignette 7 concerns independence and Vignette 10 cooperation. Other suggested re-specifications do not have as clear theoretical or methodological explanations.

We opted to discard the re-specifications and the CFA estimation in favour of the ESEM. It explicitly models the cross-loadings that in CFA are probably erroneously assumed to be zero. However, not all cross-loadings imply good conceptual fit despite the improved fit statistics, and the pattern of loadings should be examined in more detail. One of the more substantial cross-loadings is on Vignette 3, in which the autonomy orientation indicator also loaded positively on the impersonal orientation factor. In the scenario, the manager implies that the respondents should improve their performance. The cross-loading implicates that the scenario activates both representations of obstacles to goal attainment and of personal interest. Perhaps in the autonomy orientation item for that vignette (‘This will be interesting and important for me to try’), the importance suggests a less self-determined motivational factor for the respondents. On Vignettes 10 and 11, the autonomy orientation indicator also positively loaded also on the controlled orientation factor. Both vignettes describe a scenario in which the manager suggests working more with other employees. It can be speculated that in these kinds of scenarios, it is difficult to formulate indicators for autonomy orientation that do not contain variance relevant to controlled orientation also. Perhaps the participants inevitably felt controlled when they imagined their supervisor to suggest increasing cooperation with others. Whether this is a sample-specific chance result or a genuine cultural difference between Norwegian and Finnish participants can not be resolved in this study. Although these cross-loadings suggest that the indicators capture variance not relevant to their main constructs, they are not especially strong.

In ESEM, on Vignettes 9, 10, and 11, the controlled orientation indicators loaded positively also on the impersonal orientation factor. These loadings are in the theoretically sensible direction, but their consistency and magnitude suggest that these indicators include a substantial amount of variance shared with impersonal orientation in addition to construct-relevant variance. Whether this is a serious problem is a matter of interpretation. In a recent examination of the Multidimensional Work Motivation Scale (Trépanier et al. 2023), the models deemed acceptable included cross-loadings of similar magnitude to the strongest cross-loading in the present study.

However, even if the single cross-loadings did not indicate serious problems, the fit statistics and the combination of weak main loadings and several problematic cross-loadings can suggest that a measure is insufficiently valid. Admittedly, the fit statistics are relatively modest for ESEM analyses. Particularly, TLI remains low (however, see Holmquist et al., 2023 for a somewhat similar observation). TLI is an index that penalizes models that estimate many parameters (West et al. 2023), such as ESEM models. Relatively low TLI may indicate that not all cross-loadings are necessary. However, the satisfactory results in the between-groups measurement invariance tests suggest that the pattern of loadings is not arbitrary but replicates reasonably well in both manual workers and non-manual workers. This implies that workers in different positions interpret the measure in a similar fashion and that occupational status and associated differences in relationship with one’s supervisor (see, for example, Cho et al. 2024; Pajic et al. 2021) do not significantly affect how respondents see the contents of COWS. Furthermore, reliability coefficients comparable to those in the original validation study (Halvari & Olafsen 2020) indicate that the causality orientation factors capture some sufficiently consistent dimension of participants’ experience.

Can criterion validity be excellent despite modest factorial validity?

Regarding criterion validity, the correlational results demonstrate that the causality orientations assessed with the Finnish translation of the COWS are associated among themselves and with other relevant variables fully according to the hypotheses and the results of the original validation study (Halvari & Olafsen 2020). The associations between the causality orientations were virtually the same as in Halvari and Olafsen’s original study. Magnitudes of correlations between the criterion variables and the causality orientations differed somewhat more, but the directions were always as expected. Criterion validity, assessed with conceptually highly relevant variables, is excellent.

However, one should consider that the factorial validity of COWS was somewhat modest, both in the present study and in the original validation study. In the present study, the potential construct-relevant and construct-irrelevant variance of the items was explicitly modelled with ESEM. The results suggest that, despite some weaknesses discussed above, the vignettes and their items mostly manage to capture the key elements of the three causality orientations; thus, the factors are able to covary with theoretically relevant criterion variables measured with valid instruments. Nonetheless, as some vignettes or items may be problematic, results should be interpreted with some caution.

Suggestions for further research

There are both suggestions and caveats for utilising the Finnish COWS in further research. Firstly, due to the modest factorial validity, including the COWS in a high-stakes data collection carries a certain risk. Rather, focus should be on developing the COWS further regarding both vignettes and items assessing the different causality orientations. This could call for qualitative studies to investigate experiences with the orientations and quantitative studies with several samples to test modified scenarios and items.

Secondly, the present study provides some suggestions to consider in the possible revision of the Finnish COWS. Identically worded items, even if they measure the same factor, should be avoided. More interestingly, it can be speculated that many Finnish employees may have difficulties not feeling pressured if they are encouraged to work more with others. Reducing these kinds of scenarios in the measure may result in weaker cross-loadings of autonomy orientation indicators on the controlled orientation factor. However, some substantial cross-loadings signify construct-relevant variance, and a good measure should not be expected to contain only non-significant or otherwise very small cross-loadings.

Strengths and limitations of the present study

Among the strengths of the present study are the large and heterogenous sample, attempts to control careless responding, careful translation procedure, carefully chosen criterion variables, and conducting the analyses with latent variables. Naturally, the study also has some weaknesses to acknowledge.

The large and heterogenous sample and control for careless responding increase the probability that the results are not due to some sample-specific factors. It would have been preferable to use another sample for cross-validation, but the measurement invariance tests between the participants of the different occupational statuses in the sample somewhat support the generalizability of the results.

The back-translation procedure and discussions with the measure’s original developers facilitated capturing the intended meanings of vignettes and the items. However, the first author and the original developers did not share a native tongue, and although the first author has basic knowledge of Swedish, a language related to Norwegian, some nuance may have been lost. Furthermore, translating all the eleven vignettes in the development version of the measure instead of only the six in the final version would have provided more vignettes to examine and choose from for the final Finnish measure. That could have led to a different vignette composition that would have had better fit with the data, but it is also possible that vignettes deemed unsatisfactory with the Norwegian sample would have been inferior in the Finnish sample as well. Vignette-style measures may demand a lot of effort from the respondents (Koestner & Levine 2023), and the choice to include only the six final vignettes probably facilitated careful responding.

Fewer criterion variables were utilized in the present study compared to the original validation (Halvari & Olafsen 2020). However, all the criterion variables were highly relevant for the validity of the causality orientations in the SDT framework. Satisfaction of the basic psychological needs is considered a direct consequence of the causality orientations (Deci et al. 2017; Ryan & Deci 2017); intrinsic motivation and amotivation are the motivational qualities in the different poles of the self-determination continuum expected to be associated especially with autonomous orientation and impersonal orientation (Ryan & Deci 2017); and energy-related phenomena, operationalized as vigour and exhaustion in the present study, are considered essential outcomes in SDT (Frederick & Ryan 2023). Furthermore, using only latent variables, essentially without measurement error, produced maximally accurate estimates of the associations. As a conclusion, we suggest that the Finnish Causality Orientations Scale at Work Scale is a good translation of the original measure that can be used in research with caution and that the present study provides a solid foundation for its further development.