Executive functioning in children with ADHD Investigating the cross-method correlations between performance tests and rating scales

Olsen, Kristoffer Dalsgaard; Sukhodolsky, Denis; Bikic, Aida

doi:10.2478/sjcapp-2024-0001

Abstract

Objective

Replicated evidence shows a weak or non-significant correlation between different methods of evaluating executive functions (EF). The current study investigates the association between rating scales and cognitive tests of EF in a sample of children with ADHD and executive dysfunction.

Method

The sample included 139 children (aged 6–13) diagnosed with ADHD and executive dysfunctions. The children completed subtests of the Cambridge Neuropsychological Test Automated Battery (CANTAB). Parents completed the Behavior Rating Inventory of Executive Function (BRIEF) and the Children’s Organizational Skills Scale (COSS).

Analysis

Pairwise Spearman correlations were calculated between the composite and separate subscales of cognitive tests and rating scales. In secondary analyses, pairwise Spearman correlations were conducted between all composite scales and subscales, stratified by child sex and child ADHD subtype.

Results

The correlation analyses between composite scores yielded no significant correlations. The results when comparing CANTAB TO and BRIEF GE are r=−.095, p=.289, and r=.042, p=.643 when comparing CANTAB TO and COSS TO. The analyses between all composite scales and subscales found one significant negative correlation (r=−.25, p<.01). There are significant cross-method differences when stratified by the ADHD-Inattentive subtype, showing significant negative correlations (moderate) between CANTAB and BRIEF composite (r=−.355, p=.014) and subscales.

Discussion

It is possible that the different methods measure different underlying constructs of EF. It may be relevant to consider the effects of responder bias and differences in ecological validity in both measurement methods.

Conclusion

The results found no significant correlations. The expectation in research and clinical settings should not be to find the same results when comparing data from cognitive tests and rating scales. Future research might explore novel approaches to EF testing with a higher level of ecological validity, and designing EF rating scales that capture EF behaviors more so than EF cognition.

References

Diamond A. Executive functions. Annu Rev Psychol. 2013;64:135–68.
Search in Google Scholar Back to article
Whittingham K, Bodimeade HL, Lloyd O, Boyd RN. Everyday psychological functioning in children with unilateral cerebral palsy: does executive functioning play a role? Dev Med Child Neurol. 2014;56(6):572–9.
Search in Google Scholar Back to article
Krivitzky L, Bosenbark DD, Ichord R, Jastrzab L, Billinghurst L. Brief report: Relationship between performance testing and parent report of attention and executive functioning profiles in children following perinatal arterial ischemic stroke. Child Neuropsychol. 2019;25(8):1116–24.
Search in Google Scholar Back to article
McLean RL, Johnson Harrison A, Zimak E, Joseph RM, Morrow EM. Executive function in probands with autism with average IQ and their unaffected first-degree relatives. J Am Acad Child Adolesc Psychiatry. 2014;53(9):1001–9.
Search in Google Scholar Back to article
de Vries M, de Ruiter MA, Oostrom KJ, Schouten-Van Meeteren AYN, Maurice-Stam H, Oosterlaan J, et al. The association between the behavior rating inventory of executive functioning and cognitive testing in children diagnosed with a brain tumor. Child Neuropsychol. 2018;24(6):844–58.
Search in Google Scholar Back to article
Krieger V, Amador-Campos JA. Assessment of executive function in ADHD adolescents: contribution of performance tests and rating scales. Child Neuropsychol. 2018;24(8):1063–87.
Search in Google Scholar Back to article
Claesdotter E, Cervin M, Akerlund S, Rastam M, Lindvall M. The effects of ADHD on cognitive performance. Nord J Psychiatry. 2018;72(3):158–63.
Search in Google Scholar Back to article
Nigg JT, Willcutt EG, Doyle AE, Sonuga-Barke EJ. Causal heterogeneity in attention-deficit/hyperactivity disorder: do we need neuropsychologically impaired subtypes? Biol Psychiatry. 2005;57(11):1224–30.
Search in Google Scholar Back to article
Willcutt EG, Doyle AE, Nigg JT, Faraone SV, Pennington BF. Validity of the executive function theory of attention-deficit/hyperactivity disorder: a meta-analytic review. Biol Psychiatry. 2005;57(11):1336–46.
Search in Google Scholar Back to article
Lambek R, Tannock R, Dalsgaard S, Trillingsgaard A, Damm D, Thomsen PH. Executive dysfunction in school-age children with ADHD. J Atten Disord. 2011;15(8):646–55.
Search in Google Scholar Back to article
Kofler MJ, Sarver DE, Harmon SL, Moltisanti A, Aduen PA, Soto EF, et al. Working memory and organizational skills problems in ADHD. J Child Psychol Psychiatry. 2018;59(1):57–67.
Search in Google Scholar Back to article
Wahlstedt C, Thorell LB, Bohlin G. Heterogeneity in ADHD: neuropsychological pathways, comorbidity and symptom domains. J Abnorm Child Psychol. 2009;37(4):551–64.
Search in Google Scholar Back to article
Toplak ME, Bucciarelli SM, Jain U, Tannock R. Executive functions: performance-based measures and the behavior rating inventory of executive function (BRIEF) in adolescents with attention deficit/hyperactivity disorder (ADHD). Child Neuropsychol. 2009;15(1):53–72.
Search in Google Scholar Back to article
Toplak ME, West RF, Stanovich KE. Practitioner review: do performance-based measures and ratings of executive function assess the same construct? J Child Psychol Psychiatry. 2013;54(2):131–43.
Search in Google Scholar Back to article
Soto EF, Kofler MJ, Singh LJ, Wells EL, Irwin LN, Groves NB, et al. Executive functioning rating scales: Ecologically valid or construct invalid? Neuropsychology. 2020;34(6):605–19.
Search in Google Scholar Back to article
Ledochowski J, Andrade BF, Toplak ME. A novel unstructured performance-based task of executive function in children with attention-deficit/hyperactivity disorder. J Clin Exp Neuropsychol. 2019;41(5):445–59.
Search in Google Scholar Back to article
Bikic A, Dalsgaard S, Olsen KD, Sukhodolsky DG. Organizational skills training for children with ADHD: study protocol for a randomized, controlled trial. Trials. 2021;22(1):752.
Search in Google Scholar Back to article
Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al. Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry. 1997;36(7):980–8.
Search in Google Scholar Back to article
Reynolds RK, C. Intellectual Assessment Scales and Reynolds Intellectual Screeining Test Professional Manual. Psychological Assessment Resources. 2003.: Psychological Assessment Resources; 2003.
Search in Google Scholar Back to article
Gioia GA, Isquith PK, Retzlaff PD, Espy KA. Confirmatory factor analysis of the Behavior Rating Inventory of Executive Function (BRIEF) in a clinical sample. Child Neuropsychol. 2002;8(4):249–57.
Search in Google Scholar Back to article
Gioia GA, Isquith PK, Guy SC, Kenworthy L.. Behavior Rating Inventory of Executive Function Child Neuropsychology. 2000;6(3):235–8.
Search in Google Scholar Back to article
Schneider H, Ryan M, Mahone EM. Parent versus teacher ratings on the BRIEF-preschool version in children with and without ADHD. Child Neuropsychol. 2020;26(1):113–28.
Search in Google Scholar Back to article
Abikoff H, Gallagher, R. . Children’s Organizational Skills Scales manual. North Tonawanda, NY Multi-Health Systems; 2009.
Search in Google Scholar Back to article
Pfiffner LJ, Yee Mikami A, Huang-Pollock C, Easterlin B, Zalecki C, McBurnett K. A randomized, controlled trial of integrated home-school behavioral treatment for ADHD, predominantly inattentive type. J Am Acad Child Adolesc Psychiatry. 2007;46(8):1041–50.
Search in Google Scholar Back to article
Tamm L, Peugh J. Concordance of teacher-rated and performance-based measures of executive functioning in preschoolers. Child Neuropsychol. 2019;25(3):410–24.
Search in Google Scholar Back to article
Dekker MC, Ziermans TB, Spruijt AM, Swaab H. Cognitive, Parent and Teacher Rating Measures of Executive Functioning: Shared and Unique Influences on School Achievement. Front Psychol. 2017;8:48.
Search in Google Scholar Back to article