Judging Students' Texts in a Digital Research Tool: Do Text Quality, Students' Gender, and Migration Background Impact Teachers' Text Assessments? Cover

.blurhash-client-img { display: none !important; }

Judging Students' Texts in a Digital Research Tool: Do Text Quality, Students' Gender, and Migration Background Impact Teachers' Text Assessments?

Research in Subject-matter Teaching and Learning (RISTAL)

Volume 8 (2025): Issue 1 (March 2025)

By: Frederike Strahl, Jörg Kilian and Jens Möller

Open Access

|Dec 2025

Figures & Tables

Text assessment using the holistic scale in the Student Inventory

Post-hoc pairwise comparisons for text quality assessments of Study 2

Quality comparison	M_diff	95 % CI	t(233)	p	d
Holistic Assessment

Low vs. medium	−0.23	[−0.42, 0.04]	−2.34	.060	−0.15
Low vs. high	−0.61	[−0.77, −0.46]	−7.94	< .001	−0.50
Medium vs. high	−0.39	[−0.55, −0.23]	−4.74	< .001	−0.30

Content Assessment

Low vs. medium	−0.28	[−0.41, −0.15]	−4.23	< .001	−0.27
Low vs. high	−0.35	[−0.48, −0.23]	−5.55	< .001	−0.35
Medium vs. high	−0.07	[−0.20, 0.05]	−1.15	.753	−0.07

Style Assessment

Low vs. medium	−0.34	[−0.49, −0.19]	−4.47	< .001	−0.28
Low vs. high	−0.51	[−0.64, −0.37]	−7.46	< .001	−0.47
Medium vs. high	−0.17	[−0.30, −0.03]	−2.48	.041	−0.16

Linguistic accuracy

Low vs. medium	−0.13	[−0.32, 0.06]	−1.37	.513	−0.09
Low vs. high	−0.56	[−0.71, −0.42]	−7.52	< .001	−0.47
Medium vs. high	−0.43	[−0.59, −0.28]	−5.53	< .001	−0.35

Post-hoc pairwise comparisons for text quality assessments of Study 1

Quality comparison	M_diff	95 % CI	t(233)	p	d
Holistic Assessment

Low vs. medium	−0.18	[−0.38, 0.02]	−1.77	.236	−0.12
Low vs. high	−0.57	[−0.76, −0.38]	−6.01	< .001	−0.39
Medium vs. high	−0.39	[−0.56, −0.22]	−4.40	< .001	−0.29

Content Assessment

Low vs. medium	−0.27	[−0.41, −0.13]	−3.87	< .001	−0.25
Low vs. high	−0.27	[−0.40, −3.93]	−3.93	< .001	−0.26
Medium vs. high	0.00	[−0.13, 0.14]	0.06	1.00	0.00

Style Assessment

Low vs. medium	−0.21	[−0.37, −0.06]	−2.74	.018	−0.18
Low vs. high	−0.43	[−0.58, −0.27]	−5.45	< .001	−0.36
Medium vs. high	−0.21	[−0.35, 0.08]	−3.05	.008	−0.20

Linguistic accuracy

Low vs. medium	−0.05	[−0.24, 0.14]	−0.53	1.00	−0.03
Low vs. high	−0.50	[−0.66, −0.35]	−6.31	< .001	−0.41
Medium vs. high	−0.45	[−0.61, −0.30]	−5.88	< .001	−0.38

Results of the exploratory analysis regarding students' gender bias for teachers' assessment of medium-quality texts

Assessment scale	Male name		Female name

	M	SD	M	SD	t(116)	p	Cohen's d
Holistic	2.80	1.06	3.24	1.10	2.80	.01	.26
Content	2.68	0.87	2.96	0.80	2.38	.02	.22
Style	2.50	0.84	2.80	0.82	2.60	.01	.24
Linguistic accuracy	2.44	1.09	2.86	1.04	2.36	.02	.22

Means and standard deviations of the four assessments for all three text quality levels of Study 2

	Text quality

Scale	Low M (SD)	Medium M (SD)	High M (SD)
Holistic	2.73 (1.16)	2.96 (1.13)	3.34 (0.87)
Content	2.44 (0.80)	2.72 (0.88)	2.79 (0.76)
Style	2.33 (0.90)	2.67 (0.80)	2.84 (0.76)
Linguistic Accuracy	2.52 (1.11)	2.65 (1.04)	3.09 (0.70)

Means and standard deviations of the four assessments for all three text quality levels of Study 1

	Text quality

Scale	Low M (SD)	Medium M (SD)	High M (SD)
Holistic	2.84 (1.23)	3.02 (1.10)	3.41 (0.92)
Content	2.55 (0.85)	2.82 (0.85)	2.82 (0.85)
Style	2.44 (0.96)	2.65 (0.84)	2.86 (0.77)
Linguistic Accuracy	2.59 (1.06)	2.65 (1.08)	3.10 (0.66)

Means and standard deviations for all assessment scales by text quality and students' migration back-ground

		Text quality
Scale	Migration Background	Low	Medium	High
Scale	Migration Background	M (SD)	M (SD)	M (SD)

Holistic	Without	2.72 (1.17)	2.96 (1.14)	3.41 (0.88)
Holistic	With	2.74 (1.14)	2.95 (1.12)	3.28 (0.86)

Content	Without	2.40 (0.81)	2.76 (0.90)	2.84 (0.75)
Content	With	2.47 (0.79)	2.68 (0.85)	2.74 (0.77)

Style	Without	2.37 (0.91)	2.70 (0.89)	2.86 (0.77)
Style	With	2.29 (0.89)	2.64 (0.70)	2.82 (0.75)

Linguistic Accuracy	Without	2.61 (1.16)	2.62 (1.08)	3.09 (0.69)
Linguistic Accuracy	With	2.44 (1.07)	2.69 (1.01)	3.09 (0.71)

Means and standard deviations for all assessment scales by text quality and student gender

		Text quality
Scale		Low	Medium	High
Scale	Gender	M (SD)	M (SD)	M (SD)

Holistic	Male	2.88 (1.20)	2.80 (1.06)	3.38 (0.91)
Holistic	Female	2.80 (1.26)	3.24 (1.10)	3.44 (0.93)

Content	Male	2.55 (0.85)	2.68 (0.87)	2.76 (0.89)
Content	Female	2.55 (0.87)	2.96 (0.80)	2.87 (0.80)

Style	Male	2.44 (0.98)	2.50 (0.84)	2.86 (0.80)
Style	Female	2.43 (0.95)	2.80 (0.82)	2.86 (0.75)

Linguistic Accuracy	Male	2.64 (1.05)	2.44 (1.09)	3.09 (0.68)
Linguistic Accuracy	Female	2.55 (1.07)	2.86 (1.04)	3.11 (0.65)

DOI: https://doi.org/10.2478/ristal-2025-0006 | Journal eISSN: 2616-7697

Journal RSS Feed

Language: English

Page range: 47 - 61

Published on: Dec 31, 2025

Published by: Gesellschaft für Fachdidaktik (GfD e.V.)

In partnership with: Paradigm Publishing Services

Publication frequency: 1 issue per year

Keywords:

assessing writing,

teachers' judgment,

diagnostic competence,

migration background,

assessment biases

Related subjects:

Social sciences,

Curriculum and pedagogy,

Education, other

© 2025 Frederike Strahl, Jörg Kilian, Jens Möller, published by Gesellschaft für Fachdidaktik (GfD e.V.)
This work is licensed under the Creative Commons Attribution-NonCommercial 4.0 License.

Previous article Volume 8 (2025): Issue 1 (March 2025)Next article