Testing Longitudinal Measurement Invariance of the Dutch PSS-10, MBI-ES, and PANAS in a Bimonthly 6-Month Panel

Justine Van de Velde; Bert Weijters; Katia Levecque

doi:10.5334/pb.1528

Testing Longitudinal Measurement Invariance of the Dutch PSS-10, MBI-ES, and PANAS in a Bimonthly 6-Month Panel

Psychologica Belgica

Volume 66 (2026): Issue 1

By: Justine Van de Velde , Bert Weijters and Katia Levecque

Open Access

|May 2026

Abstract

Monitoring teacher well-being over time is essential in both research and practice to understand changes, evaluate interventions, and support occupational health. This study examines the longitudinal measurement invariance of the Dutch versions of three commonly used scales related to well-being: the Perceived Stress Scale (PSS-10), the Emotional Exhaustion subscale of the Maslach Burnout Inventory (MBI), and the Positive and Negative Affect Schedule (PANAS). Utilizing three bimonthly waves of data collected from a cohort of elementary school teachers in Flanders (n = 237), we evaluated the scales’ configural, metric, scalar, and residual invariance over time. Confirmatory factor analyses indicated that all three scales demonstrate adequate longitudinal measurement invariance, supporting their stability and reliability for repeated measurement. Notably, this is among the first studies to test longitudinal measurement invariance of these widely used scales over a 6-month period in an applied occupational setting. These findings strengthen confidence in their use for monitoring well-being over time.

References

Aboagye, M. O., Qin, J., Qayyum, A., Antwi, C. O., Jababu, Y., & Affum-Osei, E. (2018). Teacher burnout in pre-schools: A cross-cultural factorial validity, measurement invariance and latent mean comparison of the Maslach Burnout Inventory, Educators Survey (MBI-ES). Children and Youth Services Review, 94, 186–197. 10.1016/j.childyouth.2018.09.041
Open DOI Search in Google Scholar Back to article
Aelterman, N., Vansteenkiste, M., Haerens, L., Soenens, B., Fontaine, J. R. J., & Reeve, J. (2019). Toward an integrative and fine-grained insight in motivating and demotivating teaching styles: The merits of a circumplex approach. Journal of Educational Psychology, 111(3), 497–521. 10.1037/edu0000293
Open DOI Search in Google Scholar Back to article
Alatlı, B. (2020). A Systematic Review of Research Articles on Measurement Invariance in Education and Psychology. International Journal of Assessment Tools in Education, 7(4), 607–630. 10.21449/ijate.738560
Open DOI Search in Google Scholar Back to article
Barbosa-Leiker, C., Kostick, M., Lei, M., McPherson, S., Roper, V., Hoekstra, T., & Wright, B. (2013). Measurement Invariance of the Perceived Stress Scale and Latent Mean Differences across Gender and Time. Stress and Health, 29(3), 253–260. 10.1002/smi.2463
Open DOI Search in Google Scholar Back to article
Baumgartner, H., & Steenkamp, J. B. E. (2006). An extended paradigm for measurement analysis of marketing constructs applicable to panel data. Journal of Marketing Research, 43(3), 431–442. 10.1509/jmkr.43.3.431
Open DOI Search in Google Scholar Back to article
Belkin, L. Y., & Kong, D. T. (2022). Beliefs in government benevolence can promote individuals’ compliance with government-issued guidelines: The role of positive affect and general construal level. Journal of Applied Social Psychology, 52(2), 85–94. 10.1111/jasp.12836
Open DOI Search in Google Scholar Back to article
Brenning, K. M., Soenens, B., Van Petegem, S., & Kins, E. (2017). Searching for the Roots of Overprotective Parenting in Emerging Adulthood: Investigating the Link with Parental Attachment Representations Using An Actor Partner Interdependence Model (APIM). Journal of Child and Family Studies, 26(8), 2299–2310. 10.1007/s10826-017-0744-2
Open DOI Search in Google Scholar Back to article
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. 10.1037/0033-2909.105.3.456
Open DOI Search in Google Scholar Back to article
Ceulemans, M., Foulon, V., Ngo, E., Panchaud, A., Winterfeld, U., Pomar, L., Lambelet, V., Cleary, B., O’Shaughnessy, F., Passier, A., Richardson, J. L., Hompes, T., & Nordeng, H. (2021). Mental health status of pregnant and breastfeeding women during the COVID-19 pandemic-A multinational cross-sectional study. Acta Obstetricia et Gynecologica Scandinavica, 100(7), 1219–1229. 10.1111/aogs.14092
Open DOI Search in Google Scholar Back to article
Chen, F. F. (2007). Sensitivity of Goodness of Fit Indexes to Lack of Measurement Invariance. Structural Equation Modeling: A Multidisciplinary Journal, 14(3), 464–504. 10.1080/10705510701301834
Open DOI Search in Google Scholar Back to article
Chen, W., Liang, Y., Cheng, J., Xie, E., Zhu, Q., & Liu, J. (2024). Psychometric properties of the perceived stress scale short form (PSS-10) in Chinese adolescents: Validity, measurement invariance across gender and left-behind status. Children and Youth Services Review, 156, 107252. 10.1016/j.childyouth.2023.107252
Open DOI Search in Google Scholar Back to article
Cohen, S., Kamarck, T., & Mermelstein, R. (1983). A Global Measure of Perceived Stress. Journal of Health and Social Behavior, 24(4), 385–396. 10.2307/2136404
Open DOI Search in Google Scholar Back to article
Cohen, S., Tyrrell, D. A., & Smith, A. P. (1993). Life stress, emotional support, and susceptibility to the common cold. Journal of the American Medical Association, 271(20), 1684–1689. 10.1001/jama.1993.03510200064032
Open DOI Search in Google Scholar Back to article
Cohen, S., & Williamson, G. M. (1988). Perceived Stress in a Probability Sample of the United States. In S. Spacapan & S. Oskamp (Eds.), The Social Psychology of Health (pp. 31–67). Sage.
Search in Google Scholar Back to article
Cole, D. A., Ciesla, J. A., & Steiger, J. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12(4), 381–398. 10.1037/1082-989X.12.4.381
Open DOI Search in Google Scholar Back to article
Crawford, J. R., & Henry, J. D. (2004). The Positive and Negative Affect Schedule (PANAS): Construct validity, measurement properties and normative data in a large non-clinical sample. The British Journal of Clinical Psychology, 43, 245–265. 10.1348/0144665031752934
Open DOI Search in Google Scholar Back to article
Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual Review of Sociology, 40, 55–75. Annual Reviews Inc. 10.1146/annurev-soc-071913-043137
Open DOI Search in Google Scholar Back to article
Demerouti, E., Bakker, A. B., Geurts, S. A., & Taris, T. W. (2014). Daily recovery from work-related effort during non-work time. In S. Sonnentag, P. L. Perrewé, & D. C. Ganster (Eds.), Current perspectives on job-stress recovery: Research in occupational stress and well-being (Vol. 11, pp. 85–123). Emerald Group Publishing. 10.1108/S1479-355520140000012003
Open DOI Search in Google Scholar Back to article
Dijkstra, P., Barelds, D. P. H., & Hoeneveld, A. (2017). Mindfulness, werkstress en irrationele gedachten. Gedrag & Organisatie, 30(2), 123–141. www.boomuitgeversamsterdam.nl. 10.5117/2017.030.002.003
Open DOI Search in Google Scholar Back to article
Engelen, U., De Peuter, S., Victoir, A., Van Diest, I., & Van den Bergh, O. (2006). Verdere validering van de “Positive and Negative Affect Schedule” (PANAS) en vergelijking van twee Nederlandstalige versies. Gedrag En Gezondheid, 34, 89–102. 10.1007/bf03087979
Open DOI Search in Google Scholar Back to article
Giusti, E. M., Veronesi, G., Callegari, C., Castelnuovo, G., Iacoviello, L., & Ferrario, M. M. (2022). The North Italian Longitudinal Study Assessing the Mental Health Effects of SARS-CoV-2 Pandemic Health Care Workers-Part II: Structural Validity of Scales Assessing Mental Health. International Journal of Environmental Research and Public Health, 19(15). 10.3390/ijerph19159541
Open DOI Search in Google Scholar Back to article
Hellebaut, S., Haerens, L., Vanderlinde, R., & De Cocker, K.. (2023). Burnout, motivation, and (de-)motivating teaching style in different phases of a teaching career. Teaching and Teacher Education, 129, 104168. 10.1016/j.tate.2023.104168
Open DOI Search in Google Scholar Back to article
Isik Akin, R., Breeman, L. D., & Branje, S. (2024). Parent-Child Relationship, Well-Being and Home-Leaving during the Transition from High School to University. Youth, 4(1), 80–96. 10.3390/youth4010006
Open DOI Search in Google Scholar Back to article
Jiang, C., Ma, H., Luo, Y., Fong, D. Y. T., Umucu, E., Zheng, H., Zhang, Q., Liu, X., Liu, X., Spruyt, K., & Meng, R. (2023). Validation of the Chinese version of the Perceived Stress Scale-10 integrating exploratory graph analysis and confirmatory factor analysis. General Hospital Psychiatry, 84, 194–202. 10.1016/j.genhosppsych.2023.07.008
Open DOI Search in Google Scholar Back to article
Jønsson, T. F., & Kähler, H. G. (2022). The savvy and cheerful employee innovation champions: The roles of political skill and trait-positive affect in employees’ championing and salary levels. Creativity and Innovation Management, 31(2), 236–247. 10.1111/caim.12496
Open DOI Search in Google Scholar Back to article
Kim, H., & Juye, Ji. (2009). Factor Structure and Longitudinal Invariance of the Maslach Burnout Inventory. Research on Social Work Practice, 19(3), 325–339. 10.1177/1049731508318550
Open DOI Search in Google Scholar Back to article
Korten, N. C. M., Comijs, H. C., Penninx, B. W. J. H., & Deeg, D. J. H. (2017). Perceived stress and cognitive function in older adults: which aspect of perceived stress is important? International Journal of Geriatric Psychiatry, 32(4), 439–445. 10.1002/gps.4486
Open DOI Search in Google Scholar Back to article
Laporte, N., Soenens, B., Brenning, K., & Vansteenkiste, M. (2021). Adolescents as active managers of their own psychological needs: The role of psychological need crafting in adolescents’ mental health. Journal of Adolescence, 88(1), 67–83. 10.1016/j.adolescence.2021.02.004
Open DOI Search in Google Scholar Back to article
Laporte, N., Soenens, B., Flamant, N., Vansteenkiste, M., Mabbe, E., & Brenning, K. (2022). The role of daily need crafting in daily fluctuations in adolescents’ need-based and affective experiences. Motivation and Emotion, 46(2), 137–149. 10.1007/s11031-021-09921-2
Open DOI Search in Google Scholar Back to article
Lee, B., & Jeong, H. I. (2019). Construct validity of the perceived stress scale (PSS-10) in a sample of early childhood teacher candidates. Psychiatry and Clinical Psychopharmacology, 29(1), 76–82. 10.1080/24750573.2019.1565693
Open DOI Search in Google Scholar Back to article
Lee, E.-H. (2012). Review of the Psychometric Evidence of the Perceived Stress Scale. Asian Nursing Research, 6(4), 121–127. 10.1016/j.anr.2012.08.004
Open DOI Search in Google Scholar Back to article
Mäkikangas, A., Hätinen, M., Kinnunen, U., & Pekkonen, M. (2011). Longitudinal factorial invariance of the Maslach Burnout Inventory-General Survey among employees with job-related psychological health problems. Stress and Health, 27(4), 347–352. 10.1002/smi.1381
Open DOI Search in Google Scholar Back to article
Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471–491. 10.1037/a0019227
Open DOI Search in Google Scholar Back to article
Maslach, C., & Jackson, S. E. (1986). Maslach Burnout Inventory manual (2nd ed.). Consulting Psychologists Press.
Search in Google Scholar Back to article
Maslach, C., Jackson, S. E., & Leiter, M. P. (2016). Maslach Burnout Inventory manual (4th ed.). Mind Garden. 10.1016/B978-0-12-397045-9.00149-X
Open DOI Search in Google Scholar Back to article
Maslach, C., Schaufeli, W. B., & Leiter, M. P. (2001). Job burnout. Annual Review of Psychology, 52(1), 397–422. 10.1146/annurev.psych.52.1.397
Open DOI Search in Google Scholar Back to article
Merz, E. L., Malcarne, V. L., Roesch, S. C., Ko, C. M., Emerson, M., Roma, V. G., & Sadler, G. R. (2013). Psychometric properties of Positive and Negative Affect Schedule (PANAS) original and short forms in cancer patients. Psychological Assessment, 25(4), 1069–1073. 10.1037/a0033647
Open DOI Search in Google Scholar Back to article
Ortuño-Sierra, J., Santarén-Rosell, M., de Albéniz, A. P., & Fonseca-Pedrero, E. (2015). Dimensional structure of the Spanish version of the Positive and Negative Affect Schedule (PANAS) in adolescents and young adults. Psychological Assessment, 27(3), e1–e9. 10.1037/pas0000107
Open DOI Search in Google Scholar Back to article
Pressman, S. D., & Cohen, S. (2005). Does positive affect influence health? Psychological Bulletin, 131(6), 925–971. 10.1037/0033-2909.131.6.925
Open DOI Search in Google Scholar Back to article
Reis, D., Lehr, D., Heber, E., & Ebert, D. D. (2019). The German Version of the Perceived Stress Scale (PSS-10): Evaluation of Dimensionality, Validity, and Measurement Invariance With Exploratory and Confirmatory Bifactor Modeling. Assessment, 26(7), 1246–1259. 10.1177/1073191117715731
Open DOI Search in Google Scholar Back to article
Schaufeli, W. B., Leiter, M. P., & Maslach, C. (2009). Burnout: 35 years of research and practice. Career Development International, 14(3), 204–220. 10.1108/13620430910966406
Open DOI Search in Google Scholar Back to article
Schaufeli, W. B., Martínez, I. M., Pinto, A. M., Salanova, M., & Bakker, A. B. (2002). Burnout and Engagement in University Students: A Cross-National Study. Journal of Cross-Cultural Psychology, 33(5), 464–481. 10.1177/0022022102033005003
Open DOI Search in Google Scholar Back to article
Schnitker, S., Shubert, J., Houltberg, B., & Fernandez, N. (2020). Bidirectional associations across time between entitativity, positive affect, generosity, and religiousness in adolescents training with a religiously affiliated charity marathon team. International Journal of Environmental Research and Public Health, 17(3). 10.3390/ijerph17030686
Open DOI Search in Google Scholar Back to article
Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. The Journal of Educational Research, 99(6), 323–338. 10.3200/JOER.99.6.323-338
Open DOI Search in Google Scholar Back to article
Shields, G. S., Toussaint, L. L., & Slavich, G. M. (2016). Stress-related changes in personality: A longitudinal study of perceived stress and trait pessimism. Journal of Research in Personality, 64, 61–68. 10.1016/j.jrp.2016.07.008
Open DOI Search in Google Scholar Back to article
Soenens, B., Sierens, E., Vansteenkiste, M., Dochy, F., & Goossens, L. (2012). Psychologically controlling teaching: Examining outcomes, antecedents, and mediators. Journal of Educational Psychology, 104(1), 108–120. 10.1037/a0025742
Open DOI Search in Google Scholar Back to article
Sonnentag, S., & Frese, M. (2012). Dynamic performance and well-being: The changing world of work as a challenge for occupational health psychology. Work & Stress, 26(3), 195–196. 10.1080/02678373.2012.710697
Open DOI Search in Google Scholar Back to article
Steenkamp, J. B. E., & Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. Journal of consumer research, 25(1), 78–90. 10.1086/209528
Open DOI Search in Google Scholar Back to article
Steinmetz, H. (2013). Analyzing Observed Composite Differences Across Groups: Is Partial Measurement Invariance Enough? Methodology, 9(1), 1–12. 10.1027/1614-2241/a000049
Open DOI Search in Google Scholar Back to article
Tikka, S. K., Malathesh, B. C., Bhatia, V., Sahoo, D. P., Sreepada, N., & Meena, S. K. (2022). Factor Structure of the Telugu Version of the COVID-19 Pandemic-Related Perceived Stress Scale (PSS-10-C) Administered on Grassroots Frontline Health Care Workers of Rural Telangana. Indian Journal of Psychological Medicine, 44(3), 272–278. 10.1177/02537176221086848
Open DOI Search in Google Scholar Back to article
Turhan, D., Scheunemann, A., Schnettler, T., Bäulke, L., Thies, D. O., Dresel, M., Fries, S., Leutner, D., Wirth, J., & Grunschel, C. (2021). Psychometric Properties of the German Short Version of the Maslach Burnout Inventory - Student Survey. European Journal of Health Psychology, 28(2), 45–58. Hogrefe Verlag GmbH & Co. KG. 10.1027/2512-8442/a000067
Open DOI Search in Google Scholar Back to article
Ullman, J. B., & Bentler, P. M. (2012). Structural Equation Modeling. In Handbook of Psychology, Second Edition. Wiley. 10.1002/9781118133880.hop202023
Open DOI Search in Google Scholar Back to article
Van De Schoot, R., Schmidt, P., De Beuckelaer, A., Lek, K., & Zondervan-Zwijnenburg, M. (2015). Measurement invariance. Frontiers in psychology, 6, 1064. 10.3389/fpsyg.2015.01064
Open DOI Search in Google Scholar Back to article
Van den Berghe, L., Cardon, G., Aelterman, N., Tallir, I. B., Vansteenkiste, M., & Haerens, L. (2013). Emotional Exhaustion and Motivation in Physical Education Teachers: A Variable-Centered and Person-Centered Approach. Journal of Teaching in Physical Education, 32(3), 305–320. 10.1123/jtpe.32.3.305
Open DOI Search in Google Scholar Back to article
Van den Berghe, L., Soenens, B., Aelterman, N., Cardon, G., Tallir, I. B., & Haerens, L. (2014). Within-person profiles of teachers’ motivation to teach: Associations with need satisfaction at work, need-supportive teaching, and burnout. Psychology of Sport and Exercise, 15(4), 407–417. 10.1016/j.psychsport.2014.04.001
Open DOI Search in Google Scholar Back to article
van Eck, M., Berkhof, H., Nicolson, N., & Sulon, J. (1996). The Effects of Perceived Stress, Traits, Mood States, and Stressful Daily Events on Salivary Cortisol. Psychosomatic Medicine, 58(5), 447–458. 10.1097/00006842-199609000-00007
Open DOI Search in Google Scholar Back to article
Van Kerkhoven, J., Derwael, D., Hannosset, D., Wauters, L., & Dewolf, P. (2022). Stress levels of Flemish emergency medicine residents and the implications for clinical practice and education. Acta Clinica Belgica, 77(3), 663–670. 10.1080/17843286.2021.1946936
Open DOI Search in Google Scholar Back to article
Vermote, B., Vansteenkiste, M., Aelterman, N., van der Kaap-Deeder, J., & Beyers, W. (2023). Teachers’ Psychological Needs Link Social Pressure with Personal Adjustment and Motivating Teaching Style. The Journal of Experimental Education, 91(4), 696–717. 10.1080/00220973.2022.2039584
Open DOI Search in Google Scholar Back to article
Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063–1070. 10.1037/0022-3514.54.6.1063
Open DOI Search in Google Scholar Back to article
Wedderhoff, N., Gnambs, T., Wedderhoff, O., Burgard, T., & Bosnjak, M. (2021). On the Structure of Affect. Zeitschrift für Psychologie, 229(1), 24–37. 10.1027/2151-2604/a000434
Open DOI Search in Google Scholar Back to article
Widaman, K. F., Ferrer, E., & Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child development perspectives, 4(1), 10–18. 10.1111/j.1750-8606.2009.00110.x
Open DOI Search in Google Scholar Back to article
Xiang, G., Li, Q., Li, X., & Chen, H. (2023). Development of self-concept clarity from ages 11 to 24: Latent growth models of Chinese adolescents. Self and Identity, 22(1), 42–57. 10.1080/15298868.2022.2041478
Open DOI Search in Google Scholar Back to article
Yılmaz Koğar, E., & Koğar, H. (2024). A systematic review and meta-analytic confirmatory factor analysis of the perceived stress scale (PSS-10 and PSS-14). Stress and Health, 40(1), e3285. 10.1002/smi.3285
Open DOI Search in Google Scholar Back to article
Zevon, M. A., & Tellegen, A. (1982). The structure of mood change: An idiographic/nomothetic analysis. Journal of Personality and Social Psychology, 43(1), 111–122. 10.1037/0022-3514.43.1.111
Open DOI Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.5334/pb.1528 | Journal eISSN: 0033-2879

Journal RSS Feed

Language: English

Page range: 65 - 80

Submitted on: Mar 27, 2026

Accepted on: May 4, 2026

Published on: May 25, 2026

Published by: Ubiquity Press

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

longitudinal measurement invariance,

stress,

emotional exhaustion,

burnout,

affect,

teachers

© 2026 Justine Van de Velde, Bert Weijters, Katia Levecque, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 66 (2026): Issue 1