Have a personal or library account? Click to login
How to Run Linear Mixed Effects Analysis for Pairwise Comparisons? A Tutorial and a Proposal for the Calculation of Standardized Effect Sizes Cover

How to Run Linear Mixed Effects Analysis for Pairwise Comparisons? A Tutorial and a Proposal for the Calculation of Standardized Effect Sizes

By: Marc Brysbaert and  Dries Debeer  
Open Access
|Jan 2025

Figures & Tables

Table 1

Toy dataset for comparison of two groups. Dependent variable is working memory capacity.

65-YEAR OLDSday 1day 275-year oldsday 1day 2
Participant 14240Participant 112727
Participant 22323Participant 123537
Participant 34446Participant 134343
Participant 42020Participant 145149
Participant 54347Participant 151925
Participant 63737Participant 165250
Participant 74846Participant 173434
Participant 85353Participant 182426
Participant 95052Participant 193535
Participant 103333Participant 202321
Table 2

Data of the toy dataset after averaging the scores of day 1 and day 2.

65-YEAR OLDSSCORE75-year oldsSCORE
Participant 141Participant 1127
Participant 223Participant 1236
Participant 345Participant 1343
Participant 420Participant 1450
Participant 545Participant 1522
Participant 637Participant 1651
Participant 747Participant 1734
Participant 853Participant 1825
Participant 951Participant 1935
Participant 1033Participant 2022
M = 39.5M = 34.5
SD = 11.23SD = 10.78
joc-8-1-409-g1.png
Figure 1

Violin plot of the data listed in Table 2.

Table 3

Long format input to run the t-test in R and in jamovi. Make sure you have 20 lines (2 groups * 10 participants in each group).

PARTICIPANTGROUPSCORE
Participant 165yr41
Participant 265yr23
Participant 365yr45
Participant 465yr20
Participant 565yr45
Participant 665yr37
Participant 765yr47
Participant 865yr53
Participant 965yr51
Participant 1065yr33
Participant 1175yr27
Participant 1275yr36
Participant 1375yr43
Participant 1475yr50
Participant 1575yr22
Participant 1675yr51
Participant 1775yr34
Participant 1875yr25
Participant 1975yr35
Participant 2075yr22
joc-8-1-409-g2.png
Figure 2

jamovi output for the t-test of working memory capacity between two age groups.

joc-8-1-409-g3.png
Figure 3

Output jamovi ANOVA between-groups analysis working memory capacity.

joc-8-1-409-g4.png
Figure 4

Output jamovi linear regression analysis of between-groups example working memory.

Table 4

Outline of the input file for an LME analysis in R and jamovi (total number of data lines is 40: 20 participants * 2 days).

PARTICIPANTGROUPTIMESCORE
Participant 165yrday142
Participant 165yrday240
Participant 265yrday123
Participant 265yrday223
joc-8-1-409-g5.png
Figure 5

Output jamovi LME analysis of between-groups example working memory.

Table 5

Longitudinal data of a group of participants tested twice (day 1 & 2) at different ages (65 & 75 years). Dependent variable is working memory capacity.

PARTICIPANT65yr day 165yr day 275yr day 175yr day 2
Participant 132302727
Participant 238423537
Participant 342384343
Participant 457555149
Participant 524301925
Participant 657495250
Participant 737393434
Participant 833332426
Participant 940383535
Participant 1022202321
Table 6

Table for a t-test repeated measure example working memory.

PARTICIPANT65yr75yrdiff
Participant 131274
Participant 240364
Participant 34043–3
Participant 456506
Participant 527225
Participant 653512
Participant 738344
Participant 833258
Participant 939354
Participant 102122–1
M65 = 37.8M75 = 34.5Mdiff = 3.3
SD65 = 10.76SD75 = 10.78SDdiff = 3.23
joc-8-1-409-g6.png
Figure 6

Violin plot of the data shown in Table 6. Lines represent related observations.

Table 7

Input for R and jamovi analysis of t-test related samples.

PARTICIPANTyr65yr75
Participant 13127
Participant 24036
Participant 34043
Participant 45650
Participant 52722
Participant 65351
Participant 73834
Participant 83325
Participant 93935
Participant 102122
joc-8-1-409-g7.png
Figure 7

Output jamovi t-test longitudinal study working memory.

joc-8-1-409-g8.png
Figure 8

Output jamovi ANOVA longitudinal study working memory.

Table 8

Long format version of Table 5, needed as input for LME analysis related samples (must contain a total of 40 data rows: 10 participants * 2 ages * 2 days tested).

PARTICIPANTAGEDAYSCORE
Participant 1yr65day132
Participant 2yr65day138
Participant 3yr65day142
Participant 4yr65day157
Participant 5yr65day124
Participant 6yr65day157
Participant 7yr65day137
Participant 8yr65day133
Participant 9yr65day140
Participant 10yr65day122
Participant 1yr65day230
joc-8-1-409-g9.png
Figure 9

Output jamovi LME analysis longitudinal study working memory.

Table 9

Data of an experiment with two repeated measure variables (Day and Stimulus type) and two measurements per condition. Dependent variable is a hypothetical variable.

PARTICIPANTDAY 1DAY 2
STIMULUS TYPE 1STIMULUS TYPE 2STIMULUS TYPE 1STIMULUS TYPE 2
MEAS1MEAS2MEAS1MEAS2MEAS1MEAS2MEAS1MEAS2
Participant 133555314
Participant 257474543
Participant 343445623
Participant 412456532
Participant 564888979
Participant 654577756
Participant 766777485
Participant 844536454
Participant 953635667
Participant 1078977989
joc-8-1-409-g10.png
Figure 10

Interaction between Day and Stimulus type. The figure includes the standard errors around the means (based on jamovi).

Table 10

Input for jamovi and R to run a 2 × 2 ANOVA of the data of Table 10, together with the means and the standard deviations of the conditions.

PARTICIPANTd1s1d1s2d2s1d2s2
Participant 13542.5
Participant 265.54.53.5
Participant 33.545.52.5
Participant 41.54.55.52.5
Participant 5588.58
Participant 64.5675.5
Participant 7675.56.5
Participant 84454.5
Participant 944.55.56.5
Participant 107.5888.5
M =4.55.655.95.05
SD =1.721.551.472.29
joc-8-1-409-g11.png
Figure 11

Output jamovi ANOVA 2 × 2 repeated-measures design.

Table 11

Table to show how to center the values of Table 10 for each participant.

PARTICIPANTd1s1d1s2d2s1d2s2MEANd1s1Cd1s2Cd2s1Cd2s2C
Participant 13542.53.625–0.6251.3750.375–1.125
Participant 265.54.53.54.8751.1250.625–0.375–1.375
Participant 33.545.52.53.875–0.3750.1251.625–1.375
Participant 41.54.55.52.53.5–212–1
Participant 5588.587.375–2.3750.6251.1250.625
Participant 64.5675.55.75–1.250.251.250.25
Participant 7675.56.56.25–0.250.75–0.750.25
Participant 84454.54.375–0.375–0.3750.6250.125
Participant 944.55.56.55.125–1.125–0.6250.3751.375
Participant 107.5888.58–0.5000.5
joc-8-1-409-g12.png
Figure 12

Jamovi output for an ANOVA on the participant centered values from Table 12.

joc-8-1-409-g13.png
Figure 13

Output jamovi LME analysis 2 × 2 repeated-measures design.

Fixed Effects Parameter Estimates
NAMESEFFECTESTIMATESEdftp
(Intercept)(Intercept)5.2750.4939.0010.707<.001
Day1d2 – d10.4000.3579.181.1210.291
StimulusType1s2 – s10.1500.29114.070.5150.615
Day1 * StimulusType1d2 – d1 * s2 – s1–2.0000.6879.01–2.9090.017
Random Components
GROUPSNAMESDVARIANCEICC
Participant(Intercept)1.5112.2830.665
Day10.8350.697
Stimulus type10.5220.273
Day1 * Stimulus type11.5562.421
Residual1.0731.152
Note. Number of Obs: 80, groups: Participant 10
Table 12

Data from the study of face attractiveness (S1 = stimulus 1, P1 = participant 1, yr18 = 18-year-old, yr75 = 75-year-old). Dependent variable is attractiveness rating on a Likert scale from 1 (unattractive) to 7 (attractive).

PARTICIPANTAGES1S2S3S4S5S6S7S8S9S10
P1yr184324253516
P2yr185455453545
P3yr185325225523
P4yr184333114545
P5yr187246233424
P6yr187516255537
P7yr187324143555
P8yr185443313446
P9yr184454254545
P10yr186335224547
P11yr756253141335
P12yr756125344114
P13yr755212323115
P14yr754345255337
P15yr754143131115
P16yr756346244123
P17yr757356442224
P18yr757552254435
P19yr755354232135
P20yr756115231134
joc-8-1-409-g14.png
Figure 14

Average ratings of the young and old group per stimulus.

Table 13

Summary table for a related-samples t-test across stimuli.

STIMULUSyr18yr75
S15.45.6
S23.42.4
S33.13.6
S44.54.1
S52.12.2
S63.33.7
S73.72.7
S84.81.8
S93.32.2
S105.34.7
joc-8-1-409-g15.png
Figure 15

Output jamovi LME analysis face rating study.

Table 14

Data from a study on text reading (language = language of the text, background = whether or not the reader is expected to be familiar with the topic of the text, p1 = participant 1, t1 = text 1). Dependent variable is seconds needed to read a 125 word text.

LANGUAGEBACKGROUNDTEXTp1p2p3p4p5p6p7p8p9p10p11p12
L1Yest1363125412521373128223025
L1Yest2382932303037383436253032
L1Yest3382532462530423041222836
L1Yest4482428402828402636194030
L1Yest5382228323035373138213324
L1Not6393640423530553434284842
L1Not7341735342519432930193943
L1Not8422636403131443631344330
L1Not9422635323425423139313640
L1Not10453436413528523640283334
L2Yest11342126303329393034302829
L2Yest12392426372729463147273334
L2Yest13412825392835383243283523
L2Yest14322732442025422744183933
L2Yest15462231423331403458333229
L2Not16412744343842494542435137
L2Not17503942353439494237334337
L2Not18573750404649453845384536
L2Not19463238313736563340343826
L2Not20513640354038532742374035
joc-8-1-409-g16.png
Figure 16

Figure of reading times as a function of Language and Background knowledge.

Table 15

Findings in the analyses by participants and by texts, limited to the ANOVAs.

ANOVA BY PARTICIPANTS (F1 ANALYSIS)ANOVA BY TEXTS (F2 ANALYSIS)
2 × 2 analysis with repeated measures2 × 2 analysis with between-text variables
Main effect Language: F(1,11) = 13.85, p = .003, η² = .061, η²p = .557Main effect Language: F(1,16) = 9.45, p = .007, η² = .166, η²p = .371
Main effect Background: F(1,11) = 22.00, p < .001, η² = .181, η²p = .667Main effect Background: F(1,16) = 28.09, p < .001, η² = .493, η²p = .637
Interaction: F(1,11) = 5.66, p = .037, η² = .022, η²p = .340Interaction: F(1,16) = 3.45, p = .082, η² = .061, η²p = .177
DOI: https://doi.org/10.5334/joc.409 | Journal eISSN: 2514-4820
Language: English
Submitted on: Sep 11, 2023
Accepted on: Oct 7, 2024
Published on: Jan 6, 2025
Published by: Ubiquity Press
In partnership with: Paradigm Publishing Services
Publication frequency: 1 issue per year

© 2025 Marc Brysbaert, Dries Debeer, published by Ubiquity Press
This work is licensed under the Creative Commons Attribution 4.0 License.