Development and Validation of an Instrument to Measure Work-Related Learning

Ilke Grosemans; Kelly Smet; Ellen Houben; Nele De Cuyper; Eva Kyndt

doi:10.16993/sjwop.99

Development and Validation of an Instrument to Measure Work-Related Learning

Scandinavian Journal of Work and Organizational Psychology

Volume 5 (2020): Issue 1

By: Ilke Grosemans, Kelly Smet, Ellen Houben, Nele De Cuyper and Eva Kyndt

Open Access

|Apr 2020

Abstract

This paper describes the development and validation of an instrument for measuring work-related learning, which can be applied in different occupational contexts. Based on a comprehensive literature review and group discussions among the authors, the instrument was carefully constructed and examined among a heterogeneous sample of Flemish employees (N = 3232). The dataset was randomly divided into two subsets. An exploratory factor analysis was conducted on the first dataset (n = 1616) to provide insight into the underlying structure of the instrument. The second subset of the data (n = 1616) was used to validate the retrieved structure by means of a confirmatory factor analysis and to investigate the internal consistencies, convergent and discriminant validity, and the measurement invariance across different groups. After six months, the instrument was retested among the same respondents to examine longitudinal measurement invariance and predictive validity. The results showed that three factors could be distinguished and confirmed, namely informal learning activities using personal sources, informal learning activities using environmental sources, and formal learning activities. The results regarding the reliability and validity of the instrument were satisfactory.

References

Baert, H. (2018). Informal learning at work: What do we know more and understand better? In G. Messmann, M. Segers & F. Dochy (Eds.), Informal learning at work: Triggers, antecedents, and consequences (pp. 153–187). London: Routledge. DOI: 10.4324/9781315441962-8
Open DOI Search in Google Scholar Back to article
Bandalos, D. L. (2018). Measurement theory and applications for the social sciences. New York, NY: Guilford Press.
Search in Google Scholar Back to article
Berg, S. A., & Chyung, S. Y. (2008). Factors that influence informal learning in the workplace. Journal of Workplace Learning, 20, 229–244. DOI: 10.1108/13665620810871097
Open DOI Search in Google Scholar Back to article
Berings, M. G., Poell, R. F., & Gelissen, J. (2008). On-the-job learning in the nursing profession: Developing and validating a classification of learning activities and learning themes. Personnel Review, 37, 442–459. DOI: 10.1108/00483480810877606
Open DOI Search in Google Scholar Back to article
Berings, M. G., Poell, R. F., Simons, P. R. J., & Van Veldhoven, M. J. (2007). The development and validation of the on-the-job learning styles questionnaire for the nursing profession. Journal of Advanced Nursing, 58, 480–492. DOI: 10.1111/j.1365-2648.2007.04252.x
Open DOI Search in Google Scholar Back to article
Berntson, E., Sverke, M., & Marklund, S. (2006). Predicting perceived employability: Human capital or labour market opportunities? Economic and Industrial Democracy, 27, 223–244. DOI: 10.1177/0143831X06063098
Open DOI Search in Google Scholar Back to article
Birdi, K. S., Patterson, M. G., & Wood, S. J. (2007). Learning to perform? A comparison of learning practices and organizational performance in profit- and non-profit-making sectors in the UK. International Journal of Training and Development, 11, 265–281. DOI: 10.1111/j.1468-2419.2007.00285.x
Open DOI Search in Google Scholar Back to article
Blau, G., Andersson, L., Davis, K., Daymont, T., Hochner, A., Koziara, K., … Holladay, B. (2008). The relation between employee organizational and professional development activities. Journal of Vocational Behavior, 72, 123–142. DOI: 10.1016/j.jvb.2007.10.004
Open DOI Search in Google Scholar Back to article
Brown, C. (1989). Empirical evidence on private training. In Investing in People (pp. 301–329). US Department of Labor, Commission on Workforce Quality and Labor Market Efficiency.
Search in Google Scholar Back to article
Chan, D. C., & Auster, E. (2003). Factors contributing to the professional development of reference librarians. Library and Information Science Research, 25, 265–286. DOI: 10.1016/S0740-8188(03)00030-6
Open DOI Search in Google Scholar Back to article
Choi, W., & Jacobs, R. L. (2011). Influences of formal learning, personal learning orientation, and supportive learning environment on informal learning. Human Resource Development Quarterly, 22, 239–257. DOI: 10.1002/hrdq.20078
Open DOI Search in Google Scholar Back to article
Coertjens, L., Donche, V., De Maeyer, S., Vanthournout, G., & Van Petegem, P. (2012). Longitudinal measurement invariance of Likert-type learning strategy scales: Are we using the same ruler at each wave? Journal of Psychoeducational Assessment, 30, 577–587. DOI: 10.1177/0734282912438844
Open DOI Search in Google Scholar Back to article
Colley, H., Hodkinson, P., & Malcolm, J. (2002). Non-formal learning: mapping the conceptual terrain, a consultation report. Retrieved from http://eprints.hud.ac.uk/id/eprint/13176/
Search in Google Scholar Back to article
Coltman, T., Devinney, T. M., Midgley, D. F., & Venaik, S. (2008). Formative versus reflective measurement models: Two applications of formative measurement. Journal of Business Research, 61, 1250–1262. DOI: 10.1016/j.jbusres.2008.01.013
Open DOI Search in Google Scholar Back to article
Crouse, P., Doyle, W., & Young, J. D. (2011). Workplace learning strategies, barriers, facilitators, and outcomes: A qualitative study among human resource management practitioners. Human Resource Development International, 14, 39–55. DOI: 10.1080/13678868.2011.542897
Open DOI Search in Google Scholar Back to article
De Cuyper, N., & De Witte, H. (2011). The management paradox: Self-rated employability and organizational commitment and performance. Personnel Review, 40, 152–172. DOI: 10.1108/00483481111106057
Open DOI Search in Google Scholar Back to article
Doornbos, A. J., Bolhuis, S., & Denessen, E. (2004). Exploring the relation between work domains and work-related learning: The case of the Dutch police force. International Journal of Training and Development, 8, 174–190. DOI: 10.111/j.1360-3736.2004.00207.x
Open DOI Search in Google Scholar Back to article
Doornbos, A. J., Simons, R. J., & Denessen, E. (2008). Relations between characteristics of workplace practices and types of informal work-related learning: A survey study among Dutch police. Human Resource Development Quarterly, 19, 129–151. DOI: 10.1002/hrdq.1231
Open DOI Search in Google Scholar Back to article
Eraut, M. (2000). Non-formal learning and tacit knowledge in professional work. British Journal of Educational Psychology, 70, 113–136. DOI: 10.1348/000709900158001
Open DOI Search in Google Scholar Back to article
Eraut, M. (2004). Informal learning in the workplace. Studies in Continuing Education, 26, 247–273. DOI: 10.1080/158037042000225245
Open DOI Search in Google Scholar Back to article
Fornell, C., & Larcker, D. F. (1981). Evaluating structural equation models with unobservable variables and measurement error. Journal of Marketing Research, 19, 39–50. DOI: 10.2307/3151312
Open DOI Search in Google Scholar Back to article
Froehlich, D., Beausaert, S., Segers, M., & Gerken, M. (2014). Learning to stay employable. Career Development International, 19, 508–525. DOI: 10.1108/CDI-11-2013-0139
Open DOI Search in Google Scholar Back to article
Hair, J., Black, W., Babin, B., Anderson, R., & Tatham, R. (2009). Multivariate data analysis. Upper Saddle River: Pearson Prentice Hall.
Search in Google Scholar Back to article
Harrell, F. (2016). Hmisc: Harrell Miscellaneous. R package version 3.17-2. Retrieved from http://CRAN.R-project.org/package=Hmisc
Search in Google Scholar Back to article
Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1, 104–121. DOI: 10.1177/109442819800100106
Open DOI Search in Google Scholar Back to article
Hurtz, G. M., & Williams, K. J. (2009). Attitudinal and motivational antecedents of participation in voluntary employee development activities. Journal of Applied Psychology, 94, 635–653. DOI: 10.1037/a0014580
Open DOI Search in Google Scholar Back to article
Iacobucci, D. (2009). Everything you always wanted to know about SEM (structural equations modelling) but were afraid to ask. Journal of Consumer Psychology, 19, 673–680. DOI: 10.1016/j.jcps.2009.09.002
Open DOI Search in Google Scholar Back to article
Jacobs, R. L., & Park, Y. (2009). A proposed conceptual framework of workplace learning: Implications for theory development and research in human resource development. Human Resource Development Review, 8, 133–150. DOI: 10.1177/1534484309334269
Open DOI Search in Google Scholar Back to article
Kwakman, K. (2003). Factors affecting teachers’ participation in professional learning activities. Teaching and Teacher Education, 19, 149–170. DOI: 10.1016/S0742-051X(02)00101-4
Open DOI Search in Google Scholar Back to article
Kyndt, E., & Baert, H. (2013). Antecedents of employees’ involvement in work-related learning: A systematic review. Review of Educational Research, 83, 273–313. DOI: 10.3102/0034654313478021
Open DOI Search in Google Scholar Back to article
Kyndt, E., & Beausaert, S. (2017). How do conditions known to foster learning in the workplace differ across occupations? In R. Noe & J. Ellingson (Eds.), Autonomous learning in the workplace (pp. 201–218). Milton Park, Abington, UK: Taylor and Francis. DOI: 10.4324/9781315674131-11
Open DOI Search in Google Scholar Back to article
Kyndt, E., Gijbels, D., Grosemans, I., & Donche, V. (2016). Teachers everyday professional development: Mapping informal activities, antecedents, and learning outcomes. Review of Educational Research, 86, 1111–1150. DOI: 10.3102/0034654315627864
Open DOI Search in Google Scholar Back to article
Kyndt, E., & Onghena, P. (2013). The integration of work and learning: Tackling the complexity with structural equation modelling. In C. Harteis, A. Rausch & J. Seifried (Eds.), Discourses on professional learning. On the boundary between learning and working (pp. 255–291). Dordrecht: Springer. DOI: 10.1007/978-94-007-7012-6_14
Open DOI Search in Google Scholar Back to article
Lauber, T. B., Taylor, E. J., Decker, D. J., & Knuth, B. A. (2010). Challenges of professional development: Balancing the demands of employers and professions in federal natural resource agencies. Organization and Environment, 23, 446–464. DOI: 10.1177/108602661038776
Open DOI Search in Google Scholar Back to article
Livingstone, D. W. (2001). Adults’ informal learning: Definitions, findings, gaps, and future research. Toronto: Centre for the Study of Education and Work. Retrieved from https://eric.ed.gov/?id=ED452390
Search in Google Scholar Back to article
Lohman, M. C. (2006). Factors influencing teachers’ engagement in informal learning activities. Journal of Workplace learning, 18, 141–156. DOI: 10.1108/13665620610654577
Open DOI Search in Google Scholar Back to article
Lohman, M. C., & Woolf, N. H. (2001). Self-initiated learning activities of experienced public school teachers: Methods, sources, and relevant organizational influences. Teachers and Teaching, 7, 59–74. DOI: 10.1080/13540600123835
Open DOI Search in Google Scholar Back to article
Mallon, M., & Walton, S. (2005). Career and learning: The ins and the outs of it. Personnel Review, 34, 468–487. DOI: 10.1108/00483480510599789
Open DOI Search in Google Scholar Back to article
Manuti, A., Pastore, S., Scardigno, A. F., Giancaspro, M. L., & Morciano, D. (2015). Formal and informal learning in the workplace: A research review. International Journal of Training and Development, 19, 1–17. DOI: 10.111/ijtd.12044
Open DOI Search in Google Scholar Back to article
Marsick, V. J., & Watkins, K. E. (2001). Informal and incidental learning. New Directions for Adult and Continuing Education, 2001, 25–34. DOI: 10.1002/ace.5
Open DOI Search in Google Scholar Back to article
Maurer, T. J., & Tarulli, B. A. (1994). Investigation of perceived environment, perceived outcome, and person variables in relationship to voluntary development activity by employees. Journal of Applied Psychology, 79, 3–14. DOI: 10.1037/0021-9010.79.1.3
Open DOI Search in Google Scholar Back to article
Miller, G. A. (1994). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 101, 343–352. DOI: 10.1037/h0043158
Open DOI Search in Google Scholar Back to article
Nelissen, J., Forrier, A., & Verbruggen, M. (2017). Employee development and voluntary turnover: Testing the employability paradox. Human Resource Management Journal, 27, 152–168. DOI: 10.1111/1748-8583.12136
Open DOI Search in Google Scholar Back to article
Nikolova, I., Van Ruysseveldt, J., De Witte, H., & Syroit, J. (2014). Work-based learning: Development and validation of a scale measuring the learning potential of the workplace (LPW). Journal of Vocational Behavior, 84, 1–10. DOI: 10.1016/j.jvb.2013.09.004
Open DOI Search in Google Scholar Back to article
Noe, R. A., Tews, M. J., & Marand, A. D. (2013). Individual differences and informal learning in the workplace. Journal of Vocational Behavior, 83, 327–335. DOI: 10.1016/j.jvb.2013.06.009
Open DOI Search in Google Scholar Back to article
Park, Y., & Choi, W. (2016). The effects of formal learning and informal learning on job performance: The mediating role of the value of learning at work. Asia Pacific Education Review, 17, 279–287. DOI: 10.1007/s12564-016-9429-6
Open DOI Search in Google Scholar Back to article
Pierce, H. R., & Maurer, T. J. (2009). Linking employee development activity, social exchange and organizational citizenship behavior. International Journal of Training and Development, 13, 139–147. DOI: 10.1111/j.1468-2419.2009.00323.x
Open DOI Search in Google Scholar Back to article
Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., & Podsakoff, N. P. (2003). Common method bias in behavioral research: A critical review of the literature and recommended remedies. Journal of Applied Psychology, 88, 879–903. DOI: 10.1037/002-1-9010.88.5.879
Open DOI Search in Google Scholar Back to article
Price, J. (1997). Handbook of organizational measurement. International Journal of Manpower, 18, 301–558. DOI: 10.1108/01437729710182260
Open DOI Search in Google Scholar Back to article
Raemdonck, I., Gijbels, D., & van Groen, W. (2014). The influence of job characteristics and self-directed learning orientation on workplace learning. International Journal of Training and Development, 18, 188–203. DOI: 10.1111/ijtd.12028
Open DOI Search in Google Scholar Back to article
Reio, T., & Shuck, B. (2015). Exploratory factor analysis: Implications for theory, research, and practice. Advances in Developing Human Resources, 17, 12–25. DOI: 10.1177/1523422314559804
Open DOI Search in Google Scholar Back to article
Revelle, W. (2015) psych: Procedures for personality and psychological research. Evanston: Northwestern University. Retrieved from http://CRAN.R-project.org/package=psych
Search in Google Scholar Back to article
Rosseel, Y. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. Retrieved from http://www.jstatsoft.org/v48/i0. DOI: 10.18637/jss.v048.i02
Open DOI Search in Google Scholar Back to article
Rowden, R. W., & Conine, C. T. (2005). The impact of workplace learning on job satisfaction in small US commercial banks. Journal of Workplace Learning, 17, 215–230. DOI: 10.1108/13665620510597176
Open DOI Search in Google Scholar Back to article
Sahinidis, A. G., & Bouris, J. (2008). Employee perceived training effectiveness relationship to employee attitudes. Journal of European Industrial Training, 32, 63–76. DOI: 10.1108/03090590810846575
Open DOI Search in Google Scholar Back to article
Schulz, M., & Ronagel, C. S. (2010). Informal workplace learning: An exploration of age differences in learning competence. Learning and Instruction, 20, 383–399. DOI: 10.1016/learninstruc.2009.03.003
Open DOI Search in Google Scholar Back to article
Simmering, M. J., Colquitt, J. A., Noe, R. A., & Porter, C. O. L. H. (2003). Conscientiousness, autonomy fit, and development: A longitudinal study. Journal of Applied Psychology, 88, 954–963. DOI: 10.1037/0021-9010.88.5.954
Open DOI Search in Google Scholar Back to article
Slotte, V., Tynjälä, P., & Hytönen, T. (2004).How do HRD practitioners describe learning at work? Human Resource Development International, 7, 481–499. DOI: 10.1080/1367886042000245978
Open DOI Search in Google Scholar Back to article
Spector, P. E. (1992). Summated Rating Scale Construction. London: SAGE Publications. DOI: 10.4135/9781412986038
Open DOI Search in Google Scholar Back to article
Statbel. (2019). Actieve (werkende en werkloze) en inactieve bevolking sinds 2017 op basis van de enquête naar de arbeidskrachten, per jaar, gewest, leeftijdsklasse en onderwijsniveau. Brussel: Statbel.
Search in Google Scholar Back to article
Streumer, J. N. (2006). Work-related Learning. Dordrecht: Springer. DOI: 10.1007/1-4020-3939-5
Open DOI Search in Google Scholar Back to article
Streumer, J. N., & Kho, M. (2006). The world of work-related learning. In J. Streumer (Ed.), Work-related Learning (pp. 3–50). Dordrecht: Springer. DOI: 10.1007/1-4020-3939-5
Open DOI Search in Google Scholar Back to article
Taris, T., & Kompier, M. (2005). Job demands, job control, strain and learning behavior: Review and research agenda. In A. Antonio & C. Cooper (Eds.), Research Companion to Organisational Health Psychology (pp. 132–150). Cheltenham, England: Elgar.
Search in Google Scholar Back to article
Tharenou, P., & Conroy, D. (1994). Men and women managers’ advancement: Personal or situational determinants? Applied Psychology, 43, 5–31. DOI: 10.1111/j.1464-0597.1994.tb00807.x
Open DOI Search in Google Scholar Back to article
Tynjälä, P. (2008). Perspectives into learning at the workplace. Educational Research Review, 3, 130–154. DOI: 10.1016/j.edurev.2007.12.001
Open DOI Search in Google Scholar Back to article
van de Vijver, F. J. R., & Leung, K. (1997). Methods and Data Analysis for Cross-cultural Research. Thousand Oaks, CA: Sage.
Search in Google Scholar Back to article
Watkins, K. E., & Marsick, V. J. (1992). Towards a theory of informal and incidental learning in organizations. International Journal of Lifelong Education, 11, 287–300. DOI: 10.1080/0260137920110403
Open DOI Search in Google Scholar Back to article
Wielenga-Meijer, E. G. A., Taris, T. W., Kompier, M. A. J., & Wigboldus, D. H. J. (2010). From task characteristics to learning: A systematic review. Scandinavian Journal of Psychology, 51, 363–375. DOI: 10.1111/j.1467-9450.2009.00768.x
Open DOI Search in Google Scholar Back to article
Wolfson, M. A., Tannenbaum, S. I., Mathieu, J. E., & Maynard, M. T. (2017). A cross-level investigation of informal field-based learning and performance improvements. Journal of Applied Psychology, 103, 14–36. DOI: 10.1037/ap10000267
Open DOI Search in Google Scholar Back to article

Articles in this issue

DOI: https://doi.org/10.16993/sjwop.99 | Journal eISSN: 2002-2867

Journal RSS Feed

Language: English

Page range: 3 - 3

Submitted on: Aug 26, 2019

Accepted on: Feb 25, 2020

Published on: Apr 15, 2020

Published by: Stockholm University Press

In partnership with: Paradigm Publishing Services

Keywords:

work-related learning,

© 2020 Ilke Grosemans, Kelly Smet, Ellen Houben, Nele De Cuyper, Eva Kyndt, published by Stockholm University Press
This work is licensed under the Creative Commons Attribution 4.0 License.

Volume 5 (2020): Issue 1