The efficacy of manualized Cognitive Behavior Therapy conducted by student-therapists treating Danish youths with anxiety using a benchmark comparison

Participants

Participants in both the ST and PT group consisted of Danish youths and their parents, seeking help regarding youth anxiety. Assessment and treatment were conducted at Centre for the Psychological Treatment of Children and Adolescents (CEBU), Department of Psychology and Behavioral Science, Aarhus University, Denmark. Procedures for assessment and inclusion were identical for both groups. Inclusion criteria were that participants (7-16 years) met the criteria for an anxiety disorder according to DSM-IV as the primary diagnosis. Exclusion criteria were psychosis, untreated attention deficit hyperactivity disorder (ADHD), intellectual disability, severe behavior disorders, eating disorders, and substance abuse. Participants were encouraged not to engage in other forms of treatment or change psychopharmacological medication during the treatment period. All families signed written consent forms regarding the use of their data in scientific publications, audio-visual recording, and information exchange (e.g. physicians and teachers).

Participants in the PT group were assessed for eligibility from January 2011 to April 2012, and consisted of 56 youths aged 7 to 16 years and their parents, who were enrolled in a randomized waitlist controlled trial (24). The randomization procedure was done in three blocks of six groups per block stratified to age groups (7-9, 10-12, 13-16), resulting in three treatment groups for each age group in both the treatment and waitlist condition (24). All participants were treated by PT. In total, 56 mothers (100%) and 55 fathers (98.2%) participated in the treatment.

Participants in the ST group were assessed for eligibility from August 2014 to September 2015 and consisted of 54 youths aged 7 to 12 years and their parents. All enrolled participants received treatment in nine separate groups, with six participants in each group. Participants were allocated in groups corresponding to their age, four groups with youths aged 7 to 9 years, and five groups with youths aged 10 to 12 years. All participants were treated by ST. In total, 54 mothers (100%) and 49 fathers (90.7%) participated in the treatment.

The attrition rates were low in both groups for the clinical assessment. In the ST group, two (3.7%) and four (7.4%) participants were lost at post and follow-up (FU) clinical assessment, respectively. In the PT group, no participants were lost to post or FU clinical assessment. Missing responses on the questionnaires were generally low for the youths, mothers, and fathers at post (n: 4 youths (7.4%); 2 mothers (3.7%); 1 father (2.0%) and FU (n: 5 youths (9.2%); 3 mothers (5.5%); 8 fathers (16.3%)) in the ST group. Missing response rates were also low in the PT group among youths, mothers, and fathers at post (n: 1 youth (1.7%) and FU (n: 4 youths (7.1%); 1 mother (1.7%); 4 fathers (7.2%). There were no significant differences between the groups relating to attrition or missing responses.

Primary outcome measure

The Anxiety Disorder Interview Schedule for DSM-IV, Parent and Child Versions (ADIS-C/P) (25) was utilized as the primary measure. The ADIS-C/P is a semi-structured diagnostic interview assessing youth anxiety disorders based on the criterions from the DSM-IV. Information is gathered by separate interviews with the youths (ADIS-C) and their parents (ADIS-P). A Clinician Severity Rating (CSR) is given, ranging from 0 (no interference) to 8 (extreme interference). A CSR of 4 or above indicates a disorder, while a CSR of less than 4 is considered subclinical. The most impairing ADIS diagnosis was considered the primary diagnosis. Diagnostic interviews were conducted by psychologists or graduate students trained in the use of ADIS-C/P. The training consisted of a two day workshop, watching two gold standard ADIS interviews of parents and children, observing one live interview conducted by an experienced interviewer, and conducting one ADIS interview observed by an experienced ADIS interviewer.

The ADIS-C/P has previously proven to be a reliable instrument for assessing anxiety disorder symptoms and diagnoses based on the DSM-IV, with good to excellent test-retest reliability (26). In the RCT study used as the PT group benchmark, an interrater reliability check of the Danish version of ADIS-C/P was conducted by two trained assessors watching and rating 22 (20.2%) of the video-recorded baseline interviews (24). The interrater reliability (Cohen’sκ) for the primary anxiety diagnosis was 0.77. The intraclass coefficient for the CSR of the primary anxiety diagnosis was 0.69. The education, procedure, and supervision of the assessors in the ST group were identical to the assessors in the PT study. No interrater reliability check was conducted for the ST group (24).

Secondary outcome measures

The Spence Children’s Anxiety Scale (SCAS) is a self-report rating scale assessing youth anxiety symptoms (27). The scale consists of 44 items (including six positive filler items), rated from 0 (never) to 3 (always). Within the scale are six subscales based on the anxiety disorders from DSM-IV: social phobia (SoP), panic disorder (PD) and agoraphobia (AP), separation anxiety disorder (SAD), generalized anxiety disorder (GAD), obsession compulsive disorder (OCD), and specific phobia (called fear of physical injury [SP]). The Danish version of the SCAS has demonstrated excellent internal consistency for the total scale in a clinical sample of youths with anxiety disorders, good test-retest reliability after two weeks and three months and good convergent and divergent validity (28). For the current study, the internal consistency for the total SCAS was good, and within the range of previous findings (α= 0.88). The SCAS parent version (SCAS-P) includes the same items as the SCAS, without the six filler items, and is administered and scored like the SCAS (29). The Danish version of SCAS-P has shown good internal consistency for both mothers (α= 0.87) and fathers (α= 0.87) in a clinical sample (Arendt et al., 2014). The current study showed good internal consistency for both mothers (α= 0.90) and fathers (α= 0.88).

The Child Anxiety Life interference Scale (CALIS) measures life interference and impairment associated with anxiety (e.g. in areas like school, at home or with friends) from the youth and parents perspective (30). The CALIS consists of 10 items where the youth reports the interference of their own life. The parent version of the CALIS (CALIS-P) consists of two subscales examining the interference of their child’s life (9 items) and of their own life (9 items). The scale is scored from 0 (not at all) to 4 (a great deal). CALIS and CALIS-P has previously demonstrated acceptable internal consistency for youths and parents, moderate-to-high test retest reliability, significant interrater reliability, and good convergent and divergent validity in a Australian population (30). The internal consistency of CALIS and CALIS-P for the current study was good for youths (α=0.85), and excellent for mothers (α= 0.91) and for fathers (α= 0.92).

The Experience of Service Questionnaire (ESQ) is a measure used to assess the youth’s and parents’ satisfaction with the treatment (31). There are separate versions for youths, with seven items, and parents, with 10 items, rated 0 (not true), 1 (partly true), or 2 (true).

Treatment

The Cool Kids program is a manualized generic group CBT program for youth with anxiety disorders, involving both youths and their parents (32). The treatment aims to reduce levels of anxiety by focusing on teaching youths to recognize their emotions, restructure negative automatic thinking, and gradually confront feared situations (i.e. exposure). A systematic review and meta-analysis of 16 studies exploring the effects of the Cool Kids program, showed a moderate pre-post effect size for all the studies combined (g= 0.65). The study also included 11 studies comparing the Cool Kids program with a control group, which yielded a moderate between groups effect size (g= 0.54) (33). The efficacy and effectiveness of the Danish version of the Cool Kids program has also been found effective in treating anxiety disorders among youths (13,24).

The treatment in the current study consisted of 10 2-hr weekly group sessions, over approximately 12 weeks. The treatment sessions were typically divided so that the youths and parents were introduced to new techniques and worked with different exercises separately with assistance from the therapists. In addition to learning and assisting with the strategies taught to their children, the parents also received psycho-education in parent management strategies (34,35).

All sessions took place at CEBU, except for thein vivosession (Session 8) which took place at a local shopping center where the families and therapists practiced exposure relevant to the youths’ anxiety. Three months after the end of treatment, participants were offered a 1-hr booster group session.

The treatment was identical for both the PT and ST group. Treatment in the PT group was conducted by a psychologist, and assisted by three graduate psychology students (helped with practical tasks, and assisted families during in-session assignments). Treatments in the ST group were conducted solely by three graduate students.

Therapists

Student Therapists: 24 graduate psychology students functioned as therapists in the ST group. They were in their first or second year of their master program. All students had received training in the Cool Kids program (10-hr workshop) and had assisted in one or two treatment groups using the Cool Kids program where they had been assigned to one family, and assisted in treatment assignments and exposure exercises. The ST had no other prior experience with the Cool Kids program. They received weekly (2-hr) group supervision throughout the treatment. The group supervision included a maximum of six ST, led by a psychologist with extensive experience with the Cool Kids program. During each supervision session, three ST presented their treatment case, their assigned families, their progress and possible challenges in treatment. The cases were discussed within the group, and the future work with the youth and their families were planned (e.g. planning gradual exposure or homework). Each ST presented their case, a minimum of two times during the treatment course.

Professional Therapists: two psychologists functioned as therapist in the PT group. They were both trained and supervised in the use of the Cool Kids program by an authorized psychotherapy specialist who had either developed or received training in the Cool Kids program at Macquarie University. The psychologists had five, and one year prior clinical experience, respectively. In each of the treatment groups, three graduate psychology students assisted the psychologist in treatment. Differences in therapist effects were explored for the primary treatment outcomes, and no significant differences were found between the two therapists.

Design and procedure

Families had prior to assessment, referred themselves to CEBU, in response to recommendations from other health institutions, friends, advertisements, and the clinic’s webpage. The families where required to send a description of the youth’s anxiety symptoms, and relevant families were called in for assessment using the diagnostic ADIS-C/P interview. Families who were offered and accepted treatment completed electronically administered rating scales prior to treatment start.

Participants were assessed, at pre-treatment (pre), post-treatment (post), and at a three-month follow-up (FU), using the ADIS-C/P interview and electronically administered rating scales. Assessors were masked to youth’s prior diagnosis at post- and three-month follow-up assessments.

Statistical analyses

The analyses included all participants with pre-treatment data. Missing data were controlled for by using Little’s Missing Completely at Random test. Results indicated that the missing data were missing completely at random. Missing data at post or FU were accounted for by using the last observation carried forward approach, bringing forward the participant’s last observation (i.e. pre or post measures) replacing the missing values at post or FU. All analyses and results were conducted based on the intention to treat sample (ITT).

Participant’s baseline demographics, clinical characteristics and scores on outcome measures were compared between the two groups using independent samplest-tests and Fisher’s exact test as appropriate (i.e. gender).

The magnitude of change (effect size [ES]) within groups was estimated using Hedges’g, to control for the small sample size (36). Effect sizes were calculated for changes from pre to post and pre to FU on all measures of continuous anxiety outcomes.

Pre to post and pre to FU comparisons between the two groups were based on degree of change on continuous outcome measures following treatment. Repeated-measure, mixed-model analyses of variance (ANOVA) were conducted with group (ST vs PT) as the between-subjects variable and time (pre- vs post-treatment/and pre- vs three-month follow-up) as the within subjects variable. The magnitude of change between groups was calculated using partialη².

The proportion of youths meeting criteria for reliable change (RC) and clinical significant change (CSC) on the primary measure of youth anxiety (i.e. SCAS and SCAS-P) was calculated, in accordance to Jacobs and Truax criteria, using age and gender-specific Danish norms to calculate individual cut-off scores (28,37). The RC represents the amount of change required to make the change statistical significant. The CSC represents a change that is both statistically significant (RC) and clinically significant. Clinically significant change requires a change in means crossing a cut-off score between a clinical and non-clinical population, according to Jacobson and Truax (1991). Clinical cut-off scores were defined as the mid-point between clinical and non-clinical mean-scores from norms reported in the study of Arendt et al. (2014). Clinical significant change (CSC) was defined as the proportion of those scoring above the clinical cut-off before treatment, achieved RC, and scored below clinical cut-off after treatment (37). Reliable change and CSC calculations were compared between the two groups using Fischer’s exact test.

The proportion of participants that were free of their primary anxiety diagnosis and all anxiety diagnosis at either post-treatment or at the three-month follow-up were compared across the two groups using Fisher’s exact test.

All statistical analyses were carried out using Statistical Product and Service Solutions (SPSS) version 22.0.0.0 (38), and all ES were calculated using Comprehensive Meta-Analysis Version 3 (CMA) (39).

Sample characteristics

Tests of normality were conducted using the Shapiro–Wilk test of normality, as well as a visual inspection of the data, to assess the normal distribution for the primary and secondary measures in both groups. All scores were approximately normally distributed in both the ST and PT group, with the exception of the mothers SCAS-P score in the PT group (W= 0.96,p= 0.04) with a skewness of 0.70 (SE = 0.32) and kurtosis of 0.47 (SE = 0.63). By visual inspection, one score was detected as an outlier. By removing the outlier, the mothers SCAS-P scores in the PT were normally distributed (W= 0.96,p= 0.08) with a skewness of 0.51 (SE = 0.32) and kurtosis of 0.08 (SE = 0.63). The analysis was conducted with and without this outlier, resulting in no significant changes in outcome. The outlier was therefore dropped, and analysis conducted without the outlier.

Baseline comparison

Baseline demographic and diagnostic characteristics for the current study at pre-assessment are presented in Table1. There were no significant differences between groups regarding gender distribution, use of medication, type of primary anxiety disorder, or CSR for primary anxiety disorder. However, there was a significant difference in youth mean age, between the ST (M= 10.18, SD = 1.46) and PT group (M= 11.82, SD = 2.49;t(89) = 4.24,p< 0.005, 95% CI [0.87, 2.41]) where participants in the PT group were significantly older. There was also a significant difference in CSR for all anxiety disorders between the ST (M= 4.60, SD = 1.33) and PT group (M= 5.32, SD = 0.95;t(95) = 3.27,p< 0.005, 95% CI [1.16, 0.28] ) where the CSR was significantly higher for participants in the PT group compared to the ST group.

TABLE 1.

Demographic and diagnostic characteristics for youths in student therapist and professional therapist group

1 Notes:ST, student therapist; PT, professional therapist; SAD, separation anxiety disorder; GAD, generalized anxiety disorder; SoP, social phobia; PD, panic disorder; OCD, obsessive compulsive disorder; SP, specific phobia; AP, agoraphobia; CSR, clinician severity rating (ADIS C/P-IV)

2a 4 enuresis

3b 1 sleep terror disorder, 1 selective mutism, 1 enuresis

No significant differences in means between the groups were found for the SCAS or SCAS-P. Youth reported CALIS was significantly higher in the ST group (M= 15.10, SD = 8.09) compared to the PT group (M= 11.90, SD=7.35;t(108) = 2.17,p= 0.03, 95% CI [0.28, 6.12]). There was also a significant difference for fathers score on CALIS-P (self-reported interference of parents life) between the means of ST (M= 10.16, SD = 5.56) and PT (M= 7.85, SD = 6.09;t(102) = 2.00,p= 0.05, 95% CI [0.14, 9.39]) were the fathers ratings were higher for the ST group. No other significant differences were found for CALIS and CALIS-P (Table2).

TABLE 2.

Continuous measure outcomes at pre- and post-treatment for student therapist and professional therapist groups, the time-by-group effect and ES

	Youths in ST group (n= 54)	Youths in PT group (n= 56)
Mean age in years (SD)	10.18 (1.46)	11.82 (2.49)
Males (%)	32 (59.3)	25 (44.6)
On psychopharmacological medication	3 (6.7)	2 (4.5)
Primary diagnosis (%)
SAD	24 (44.4)	15 (26.8)
GAD	15 (27.8)	14 (25.0)
SoP	7 (13.0)	7 (12.5)
SP	6 (11.1)	12 (21.4)
OCD	1 (1.9)	4 (7.1)
AP without PD	1 (1.9)	4 (7.1)
CSR mean for primary diagnosis	6.33 (0.75)	6.09 (1.07)
CSR mean for all diagnosis	4.60 (1.33)	5.32 (0.95)
Comorbid diagnoses (%)
Anxiety disorders	50 (92.6)	47 (83.9)
Externalizing disorders	6 (11.2)	6 (10.7)
Mood disorder	1 (1.9)	4 (7.1)
Other	4a(7.4)	3a(5.4)
No comorbidity	4 (7.4)	9 (16.1)
Number of anxiety disorders per youth (M)	2.36	2.10

1 Notes.ST, student therapist; PT, professional therapist; ES, effect-size; ADIS, Anxiety Disorder Interview Schedule for DSM-IV; CSR, Clinician Severity Rating; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version; CALIS, Child Anxiety Life Interference Scale; CALIS-P, Child Anxiety Life Interference Scale – Interference on parents’ life

2a Outlier excluded

To avoid the risk of creating biased estimates of the baseline data, no baseline adjustment or analysis controlling for these differences were conducted. The differences were taken into account in the interpretation and discussion of the results (40,41).

Pre- to post-treatment measure outcomes

Changes in CSR for the primary anxiety diagnosis presented significant and large ES for both the ST (g= 1.36,p< 0.01) and PT (g= 1.51,p< 0.01) group. The within group ES were also significant and large for changes in CSR for all anxiety diagnoses for the ST (g= 1.82) and (g= 2.52) PT group. Both groups also showed a significant reduction in all the self- and parent-reported symptoms of anxiety, with most ES ranging from medium to large (ES range:g= 0.61-2.52). One exception was the CALIS-P (fathers self-reported interference of parents life) in the PT group were the ES was small (g= 0.32), but still significant.

The pre- to post-time-by-group comparison of the CSR revealed no interaction effect between the two groups for the primary anxiety diagnosis (CSR primary:F1,108 = 0.18,p= 0.72,η²= 0.01), or for all anxiety diagnosis (CSR all:F1,108 = 0.52,p= 0.47,${\eta }_p^2$= 0.01). Furthermore, the time-by-group comparison of youth and parent self-reported-ratings showed no interaction effect between the two groups, for any of the self-reported outcome measures (Table2).

Treatment maintenance: three-month follow-up

The within group ES for changes in CSR for primary anxiety diagnosis remained significant from pre to FU in both treatment groups (ST:g= 1.88, PT:g= 1.81) group, both showing a higher ES than for pre to post. The CSR for all anxiety diagnosis also remained significant for both groups (ST:g= 2.51, PT:g= 3.27), and showed a higher ES in change at three-month follow-up than at post-treatment. All self-reported ES remained significant at three-month follow-up and there was an increase in the ES for most of the self-reported measures in both groups.

The pre to follow-up time-by-group comparison found no significant interaction effect between the two groups, regarding the CSR for primary anxiety diagnosis (CSR primary:F1,108 = 1.53,p= 0.22,η²= 0.01) or CSR for all anxiety diagnosis (CSR all:F1,108 = 0.13,p= 0.72,${\eta }_p^2$= 0.00) (Table3). No significant interaction effects were found for any of the self-reported anxiety measures (Table3).

TABLE 3.

Continuous measure outcomes at pre- and FU-treatment for the student therapist and professional therapist groups, the time-by-group effect and ES

	Pre –M(SD)	Post –M(SD)	Time-by-group effect	Pre-post ES
ADIS CSR primary diagnosis
ST	6.33 (0.75)	2.85 (2.41)	F1,108= 0.18,p= 0.72,${\eta }_p^2$=0.01	p< 0.001,g= 1.36
PT	6.09 (1.07)	2.16 (2.59)		p< 0.001,g= 1.51
ADIS CSR all diagnoses
ST	4.60 (1.32)	2.13 (1.36)	F1,108= 0.52,p= 0.47,${\eta }_p^2$=0.01	p< 0.001,g= 1.82
PT	5.32 (0.95)	1.69 (1.53)		p< 0.001,g= 2.52
SCAS youth
ST	41.57 (16.93)	24.13 (14.36)	F1,108= 0.80,p= 0.37,${\eta }_p^2$=0.00	p< 0.001,g= 1.08
PT	39.16 (18.06)	21.57 (14.42)		p< 0.001,g= 1.03
SCAS-P mother
ST	43.72 (14.98)	23.76 (11.90)	F1,108= 1.94,p= 0.27${\eta }_p^2$=0.02	p< 0.001,g= 1.41
PTa	38.91 (15.79)	22.25 (12.59)		p< 0.001,g= 1.12
SCAS-P father
ST	38.18 (15.33)	22.55 (12.57)	F1,102= 0.00,p= 0.99,${\eta }_p^2$=0.00	p< 0.001,g= 1.07
PT	37.15 (13.80)	23.56 (13.87)		p< 0.001,g= 0.97
CALIS youth
ST	15.10 (8.09)	7.57 (6.40)	F1,108= 1.87,p= 0.17,${\eta }_p^2$=0.01	p< 0.001,g= 0.99
PT	11.90 (7.35)	7.55 (6.46)		p< 0.001,g= 0.61
CALIS mother
ST	19.65 (6.80)	12.06 (7.28)	F1,108= 1.89,p= 0.17,${\eta }_p^2$=0.01	p< 0.001,g= 1.06
PT	17.78 (7.20)	10.61 (7.28)		p< 0.001,g= 0.98
CALIS father
ST	18.51 (6.93)	11.90 (6.29)	F1,102= 1.96,p= 0.17,${\eta }_p^2$=0.02	p< 0.001,g= 0.98
PT	16.05 (6.84)	10.96 (7.72)		p< 0.001,g= 0.68
CALIS-P mother
ST	13.13 (5.72)	7.26 (5.66)	F1,108= 1.39,p= 0.24,${\eta }_p^2$=0.01	p< 0.001,g= 1.02
PT	11.07 (6.91)	6.82 (6.28)		p< 0.001,g= 0.63
CALIS-P father
ST	10.16 (5.65)	6.86 (4.63)	F1,102= 3.17,p= 0.08,${\eta }_p^2$=0.03	p< 0.001,g= 0.62
PT	7.85 (6.09)	5.71 (5.46)		p< 0.001,g= 0.32

1 Notes.ST, student therapist; PT, professional therapist; ES, effect-size; ADIS, Anxiety Disorder Interview Schedule for DSM-IV; CSR, Clinician Severity Rating; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version; CALIS,Child Anxiety Life Interference Scale; CALIS-P, Child Anxiety Life Interference Scale - interference on parents life

2a Outlier excluded

Only CALIS youth for the ST group showed a significantly lower ES at three-month follow-up (g= 0.81), when compared to the ES from post-treatment (g= 0.99) (Table3).

Diagnostic status at post-treatment and at three-month follow-up

Post-treatment comparison revealed no significant difference regarding number of youths free of primary diagnosis between the ST (27 [50.0%]) and the PT (37 [66.1%]) group (χ²= 2.92,p= 0.12). However, there was a significant difference regarding number of youths free of all anxiety disorders between the ST (14 [25.9%]) and PT (27 [48.2%]) group (χ²= 5.84,p= 0.02) (Figure1).

FIGURE 1

Note.CSR = Clinician Severity Rating, SCAS = Spence Children’s Anxiety Scale

At three-month follow-up, there was an increase in number of participants free of their primary diagnosis for both the ST (40 [74.1%]) and PT (43 [76.8%]) group, and there was no significant difference between groups in number of participants free of primary anxiety diagnosis (χ²= 0.11,p= 0.83). There was also an increase in number of participants free of all anxiety diagnosis at three-month follow-up for both the ST (25 [46.3%]) and PT (33 [58.9%]) group, and there were no difference between the groups in number of participants free off all anxiety diagnosis at three-month follow-up (χ²= 1.76,p= 0.25).

Reliable and clinical significant change

At post-treatment, a significantly larger proportion of youths in the ST group achieved RC but not CSC on the mothers ratings (SCAS-P Motherχ²= 6.79,p= 0.01). There were no other significant differences between the two groups in the proportion of participants achieving RC or CSC.

At three-month follow-up, there was a significant larger proportion of the youths in the PT group who achieved RC but not CSC (SCAS Youthχ²= 6.70,p= 0.02). Consequently there was also a significant larger proportion of the youths in the ST group who achieved CSC compared to the PT group (χ²= 6.11,p= 0.02). These differences between the ST and PT group, on CSC, was also significant for the mothers responses (SCAS-P Motherχ²= 4.66,p= 0.04). Again, there were no other significant differences between the groups achieving RC and CSC on SCAS or SCAS-P (Table4).

TABLE 4.

Proportion of youths no longer meeting criteria for primary anxiety diagnosis and all anxiety diagnosis at post-treatment and follow-up

	Pre –M(SD)	FU -M(SD)	Time-by-group effect	Pre-FU ES
ADIS CSR primary diagnosis
ST	6.33 (0.75)	1.74 (2.31)	F1, 108 = 1.53,p= 0.22,${\eta }_p^2$= 0.01	p< 0.01,g= 1.88
PT	6.09 (1.07)	1.59 (2.49)		p< 0.01,g= 1.81
ADIS CSR all diagnoses
ST	4.60 (1.32)	1.37 (1.22)	F1, 108 = 0.13,p= 0.72,${\eta }_p^2$= 0.00	p< 0.01,g= 2.51
PT	5.32 (0.95)	1.28 (1.32)		p< 0.01,g= 3.27
SCAS youth
ST	41.57 (16.93)	20.91 (12.95)	F1, 108 = 1.47,p= 0.23,${\eta }_p^2$= 0.01	p< 0.01,g= 1.30
PT	39.16 (18.06)	16.70 (13.23)		p< 0.01,g= 1.33
SCAS-P mother
ST	43.72 (14.98)	21.72 (11.53)	F1, 108 = 1.49,p= 0.22,${\eta }_p^2$= 0.01	p< 0.01,g= 1.56
PTa	38.91 (15.79)	19.79 (13.61)		p< 0.01,g= 1.26
SCAS-P father
ST	38.18 (15.33)	18.70 (12.55)	F1, 102 = 0.00,p= 0.99,${\eta }_p^2$= 0.00	p< 0.01,g= 1.34
PT	37.15 (13.80)	18.72 (10.63)		p< 0.01,g= 1.42
CALIS youth
ST	15.10 (8.09)	8.50 (8.21)	F1, 108 = 3.30,p= 0.07,${\eta }_p^2$= 0.03	p< 0.01,g= 0.80
PT	11.90 (7.35)	5.79 (6.07)		p< 0.01,g= 0.88
CALIS mother
ST	19.65 (6.80)	10.13 (7.09)	F1, 108 = 1.70,p= 0.20,${\eta }_p^2$= 0.02	p< 0.01,g= 1.35
PT	17.78 (7.20)	8.80 (7.75)		p< 0.01,g= 1.18
CALIS father
ST	18.51 (6.93)	8.83 (6.67)	F1, 102 = 0.41,p= 0.53,${\eta }_p^2$= 0.00	p< 0.01,g= 1.40
PT	16.05 (6.84)	10.00 (7.85)		p< 0.01,g= 0.80
CALIS-P mother
ST	13.13 (5.72)	6.39 (5.12)	F1, 108 = 1.10,p= 0.30,${\eta }_p^2$= 0.01	p< 0.01,g= 1.22
PT	11.07 (6.91)	5.68 (6.83)		p< 0.01,g= 0.77
CALIS-P father
ST	10.16 (5.65)	5.27 (5.10)	F1,102 = 2.25,p= 0.14,${\eta }_p^2$= 0.02	p< 0.01,g= 0.89
PT	7.85 (6.09)	4.65 (4.77)		p< 0.01,g= 0.56

1 Notes.ST, student therapist; PT, professional therapist

2* Significant atp< 0.05

TABLE 5.

Reliable and clinical significant changes at post-treatment and 3-month follow-up on SCAS, for intention to treat sample

	ST group (n= 54)	PT group (n= 56)	Fischer’s
Post-treatment
Primary diagnosis	50.0% (27)	66.1% (37)	p= 0.122
All diagnosis	25.9% (14)	48.2% (27)	p= 0.019*
Follow up
Primary diagnosis	74.1% (40)	76.8% (43)	p= 0.826
All diagnosis	46.3% (25)	58.9% (33)	p= 0.252

1 Notes: ST, student therapist; PT, professional therapist; RC, reliable change; CSC, clinical significant change; SCAS, Spence Children’s Anxiety Scale; SCAS-P, Spence Children’s Anxiety Scale Parent version.

2a 3 fathers did not participate;

3b 3 fathers did not participate

4* Significant atp< 0.05

Treatment satisfaction

According to answers on the ESQ, families in both the ST and PT group were highly satisfied with the treatment. For the statement ‘The treatment helped me/my child’ 43 (87.8%) youths, 44 (83.0%) mothers, and 40 (85.1%) fathers scored 2 (true) in the ST group, while 44 (83.0%) youths, 42 (77.8%) mothers, 41 (77.4%) in the PT group scored 2 (true). For the statement ‘I had confidence in my/our therapists’ 44 (89.8%) youths, 51 (96.2%) mothers, and 44 (93.6%) fathers scored 2 (true) in the ST group, while 46 (86.8%) youths, 53 (98.1%) mothers, and 48 (90.6%) fathers in the PT scored 2 (true).

Independentt-test revealed that there was no significant difference between the total satisfaction score in the ST and PT group, for youths (t(100) = —-0.92,p=0.362, 95% CI [−0.40, 1.09]) mothers (t(105) = −0.22,p= 0.834, 95% CI [−0.58, 0.73]) or fathers score (t(98) = −1.92,p= 0.058, 95% CI [−0.03, 1.78]).

Discussion

Anxiety disorders among youths are common, impairing, and with considerable personal and societal costs. There is need for effective treatments and enough therapists who can provide treatment for youths suffering from anxiety. The aim of this study was to evaluate the outcomes of a group-based CBT program (Cool Kids) treating youth anxiety disorders, conducted by psychology students in a university clinic, and compare these outcomes with those achieved by PT. By using multiple respondents, clinical assessment, standardized questionnaires, at three different time points, this study allows for a detailed study of the treatment effect of the ST treating youths with anxiety.

Findings from the current study indicate that psychology students with limited therapist training could successfully and effectively, provide group-based CBT to youths with anxiety disorders. The study showed that the treatment effects were statistically significant in clinician-, parent-, and self-reported measures of anxiety, after treatment and with an increased effect at three-month follow-up. Furthermore, the study showed that treatment outcomes in the ST were comparable to those achieved by psychologist in the PT group. Although the study did not find many significant differences between the two groups in regard to treatment outcome, there were some differences and tendencies. These findings could extend our current knowledge of student therapy, and provide possible guidance and recommendations for future treatment conducted by ST.

At baseline, there was a significant difference between the two groups in mean age, and in CSR for all diagnosis on the ADIS. The participants in the ST group sample was included based on availability with no age stratification, and the PT sample was derived from an RCT study with a higher mean age (24). There was also a difference between the two groups in mean CSR for all anxiety disorders, were the PT had a significantly higher mean compared to the ST group. These differences should be taken into consideration when interpreting the results, as there is a heterogeneity in the age of onset for different anxiety disorders, and that symptom severity have been related to more complex problems and predict less favorable treatment outcomes (42,43). The difference in age and CSR could therefore be disadvantageous for the PT group. No other significant pre-treatment differences in terms of gender, CSR for primary diagnosis, comorbidity, or type of primary diagnosis were found, indicating that the samples were relatively comparable. The main findings concurred with the initial hypothesis, showing that ST obtained changes in both clinician- and self-rated anxiety levels in youths, with large effect sizes. When compared to the PT, no significant differences between the groups were found regarding youths free of their primary anxiety diagnosis and reduction in anxiety levels on the clinical and self-reported outcome measures. There was a significant difference between the two groups relating to number of youths free of all anxiety diagnoses at post-treatment, in favor of the PT group. However, this difference was non-significant at the three-month follow-up assessment. These finding are in line with other studies investigating the effect of novice therapists conducting CBT treatment. In an RCT study, Masia Warner et al. showed that counsellors could treat social anxiety disorders among adolescents as effectively as specialized psychologists, using the Skills for Academic and Social Success program (SASS), specifically developed to treat social anxiety disorder in a school setting (44). Like their study, we found no significant differences in the remission of primary anxiety disorder among youth treated by student therapists or psychologists. We also showed that youths in the ST group showed significant improvements on other anxiety disorders as well. We were also pleased to see that the participants in the ST group were satisfied with the treatment they received. Both parents and youths stated that they were satisfied with the effect of treatment, and that they had confidence in the therapist. The satisfaction was also comparable to those achieved in the PT group.

Although few differences between the ST and PT group were found, there were some differences indicating a difference in treatment effect between the two groups. At post-treatment, there was a significant difference in number of youths free of all anxiety disorders, favoring the PT group. There was also a non-significant difference between the groups regarding number of youths free from their primary disorder, 10 more youths were free of their primary anxiety disorder in the PT group. There was thus a clear tendency for the PT to be more effective and achieving better remission rates faster than the ST. These differences were reduced at three-month follow-up. However, there was still a tendency for the PT condition to have higher remission rates. This difference could possibly be explained by a difference in treatment setting, where one professional therapist and three assisting graduate students treated the PT group, compared to the ST group where three student therapists conducted the treatment. Although the assisting graduate students in the PT group only had an assisting role, it is possible that the setting using a PT and three assistants allowed for more youth-therapist collaboration, thus making this setting more effective compared to the setting using three ST. Another explanation could be that the professional therapists were more effective in implementing the Cool Kids program than the ST due to prior clinical experience. This difference in setting and prior experience could give an advantage to the PT providing them with a greater capacity to focus on primary and other anxiety disorders, which in turn could lead to participants achieving greater reduction in all anxiety disorders at post-treatment, in the PT group.

Overall, our findings concur with previous studies finding of investigating the treatment effects of ST and other novice therapists treating anxiety disorders using CBT. The results from the present study are comparable to previous clinical effectiveness and efficacy trials of the Cool Kids program (13,24). The clinical rated pre- to post-treatment ES in the ST group (g= 1.36-1.82) were in range with the ES from a community sample (g= 1.06-1.44) from an effectiveness study of the Cool Kids (13). The ES were also larger than the ES from the waitlist group (g= 0.31-0.40) in a recent efficacy study (24). The remission rates for the primary anxiety disorder in the ST group at post-treatment are also comparable with the recovery rates reported in a recent meta-analysis of CBT treatments treating youths with anxiety (9).

In the current study graduate-level psychology students, received training in an EBT aimed at treating youth anxiety (20-h workshop and 20-h assisting treatment), and weekly group-based supervision (2-h). The study showed that following this education, training and supervision, the ST could effectively implement a manualized 10 session group-based CBT program, with statistically significant positive changes in clinical and self-reported symptoms of anxiety. The small differences in outcomes between the more experienced PT and the inexperienced ST may indicate that comprehensive previous clinical experience is not a necessity when conducting structured and manualized CBT, with sufficient training and supervision. As the PT showed a significantly greater effect compared to the ST in reducing anxiety symptoms across all anxiety diagnosis, novice therapists may benefit in the future from more training and supervision focusing on the treatment of comorbid anxiety disorders.

Limitations

The current study is not a RCT. Although the PT sample was derived from a RCT study (24), the ST sample was not randomly allocated and does not include a control group. Therefore, the possibility that the improvement in the ST group may be due to the effect of time alone cannot be ruled out. However, the pre- and post-treatment clinical rated ESs in the current study are within range of other intervention studies and larger than the ESs reported for other control groups (e.g. waitlist) in other efficacy studies of the Cool Kids program (33). The between groups effects were all very small and statistically insignificant. Apost hocpower analysis found that the study was underpowered to detect significant effects between groups in the range of our findings. As there was a non-significant trend of larger ES and higher remission rates for the PT group for change in clinical rated measures, it is possible that larger samples could uncover significant differences between the two groups. The application of a last observation carried forward procedure could also affect the results of the analyses. However, the number of missing data was relatively small and the imputation method was not expected to have an impact on the results. No analysis on program adherence or therapist competencies were conducted in either of the treatment groups. Another possible limitation is the difference in clinical experience between the therapists in the PT group. However, there was no significant difference in treatment effect on the primary outcome between the two therapists in the PT group. The study lacks control of possible dependency in the data due to clustering of participants within therapy groups.

Clinical significance

As there is a need for more therapists providing effective anxiety treatments worldwide. There is also an implementation problem, getting EBT (e.g. Cool Kids), from the clinical researchers and out into community practices (10,12). There is no single solution to this problem. However, increased training and use of student therapist in clinical research and clinical therapy could contribute to a future solution. If clinical training using EBT was integrated in the education of clinical psychologists, this could possibly help bridge the gap between clinical research and clinical practice in the future.

Notes

[14] Ethics

The current study is an extension of the Arendt et al. (24) study, which was approved by the local county Ethical Committee and by the Danish Data Protection Agency.

[15] Conflicts of interest Conflicts of interest

The authors declare no conflicts of interest.

SCAS Youth				SCAS-P Mother			SCAS-P Father
	ST (n= 54)	PT (n= 56)	Fisher’s	ST (n= 54)	PT (n= 56)	Fisher’s	ST (n= 51a)	PT (n= 53b)	Fisher’s
Pre-treatment
Over clinical cut-off	75.9% (41)	67.9% (38)	p= 0.40	90.7% (49)	85.7% (48)	p= 0.56	78.4% (40)	84.9% (45)	p= 0.45
Post-treatment
RC deterioration	0% (0)	0% (0)	-	0% (0)	0% (0)	-	0% (0)	1.9% (1)	p= 0.99
RC improvement	63.0% (34)	67.9% (38)	p= 0.69	81.5% (44)	69.6% (39)	p= 0.19	74.5% (38)	71.7% (38)	p= 0.83
RC but not CSC	18.5% (10)	19.6% (11)	p= 0.99	33.3% (18)	12.5% (7)	p= 0.01*	17.6% (9)	32.1% (17)	p= 0.11
CSC	44.4% (24)	48.2% (27)	p= 0.71	48.1% (26)	57.1% (32)	p= 0.45	56.9% (29)	41.5% (22)	p= 0.17
Three-month follow-up
RC deterioration	0% (0)	5.4% (3)	p= 0.24	5.6% (3)	10.7% (6)	p= 0.49	7.8% (4)	5.7% (3)	p= 0.71
RC improvement	63.0% (34)	50.0% (28)	p= 0.18	77.8% (42)	66.1% (37)	p= 0.21	66.7% (34)	56.6% (30)	p= 0.32
RC but not CSC	3.7% (2)	19.6% (11)	p= 0.02*	14.8% (8)	28.6% (16)	p= 0.11	21.6% (11)	15.1% (8)	p= 0.45
CSC	59.3% (32)	35.7% (20)	p= 0.02*	68.5% (37)	48.2% (27)	p= 0.04*	52.9% (27)	47.2% (25)	p= 0.70