The following demonstration explains how to compute repeated measures ANOVA that include within-subject factors. Download the R code or the SPSS code. Such models can also be fitted using linear mixed models, see the R code or the SPSS code for a demonstration.
We consider Experiment 3 from Hatano et al. (2022). The data consist in a two by two mixed analysis of variance. The authors studied engagement and enjoyment from waiting tasks, and “potential effects of time interval on the underestimation of task motivation by manipulating the time for the waiting task”. The waiting time was randomly assigned to either short (3 minutes) or long (20 minutes) with equal probability, but participants were either told that there was a 70% chance of being assigned to the short session, or 30% chance.
We first load the data from the package and inspect the content.
data(HOSM22_E3, package = "hecedsm")
str(HOSM22_E3)Classes 'tbl_df', 'tbl' and 'data.frame': 126 obs. of 4 variables:
$ id : Factor w/ 63 levels "1","2","3","4",..: 1 1 2 2 3 3 4 4 5 5 ...
$ waiting : Factor w/ 2 levels "long","short": 2 2 1 1 1 1 2 2 1 1 ...
$ ratingtype: Factor w/ 2 levels "experience","prediction": 2 1 2 1 2 1 2 1 2 1 ...
$ imscore : num 2.33 3.33 1.25 1.92 1 ...with(HOSM22_E3, table(waiting)/2)waiting
long short
32 31 From this, we can see that there each student is assigned to a single waiting time, but that they have both rating types. Since there are 63 students, the study is unbalanced but by a single person; this may be due to exclusions.
We use the afex package (analysis of factorial design) with the aov_ez. We need to specify the identifier of the subjects (id), the response variable (dv) and both between- (between) and within-subject (within) factors. Each of those names must be quoted (strings)
options(contrasts = c("contr.sum", "contr.poly"))
fmod <- afex::aov_ez(id = "id",
dv = "imscore",
between = "waiting",
within = "ratingtype",
data = HOSM22_E3)
anova(fmod)Anova Table (Type 3 tests)
Response: imscore
num Df den Df MSE F ges Pr(>F)
waiting 1 61 2.48926 11.2551 0.126246 0.00137 **
ratingtype 1 61 0.68953 38.4330 0.120236 5.388e-08 ***
waiting:ratingtype 1 61 0.68953 0.0819 0.000291 0.77575
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# MANOVA tests
fmod$Anova
Type III Repeated Measures MANOVA Tests: Pillai test statistic
Df test stat approx F num Df den Df Pr(>F)
(Intercept) 1 0.89286 508.37 1 61 < 2.2e-16 ***
waiting 1 0.15577 11.26 1 61 0.00137 **
ratingtype 1 0.38652 38.43 1 61 5.388e-08 ***
waiting:ratingtype 1 0.00134 0.08 1 61 0.77575
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1The output includes the \(F\)-tests for the two main effects and the interaction and global effect sizes \(\widehat{\eta}^2\) (ges). There is no output for tests of sphericity, since there are only two measurements per person and thus a single mean different within-subject (so the test to check equality doesn’t make sense with a single number). We could however compare variance between groups using Levene’s test
Notice that the degrees of freedom for the denominator of the test are based on the number of participants, here 63.
Study 5 of Halevy & Berson (2022) aimed to demonstrate that events in the distant future rather than the near future influenced the prospect of peace, along with the degree of abstractness. The experimental design is a
2 (current state: war vs. peace) by 2 (predicted outcome: war vs. peace) by 2 (temporal distance: next year vs. twenty years into the future) mixed design
with current state and predicted outcome as between-subject factors and temporal distance as within-subject factor. The response is the estimated likelihood of each outcome on a 7-point Likert scale ranging from extremely unlikely (1) to extremely likely (7). Data are presented in long-format. The question asked was of the form
There is currently [war/peace] between the two tribes in Velvetia. Thinking about [next year/in 20 years], how likely is it that there will be [war/peace] in Velvetia?
We begin by loading the data and looking at the repartition of the measurements over conditions for the between-subject factors:
data(HB22_S5, package = "hecedsm")
xtabs(~ curstate + predout, data = HB22_S5) predout
curstate peace war
peace 148 164
war 118 124The data is unbalanced, but there are sufficient measurements in each condition.
mmod <- afex::aov_ez(
id = "id",
dv = "likelihood",
between = c("curstate","predout"),
within = "tempdist",
data = HB22_S5)
summary(mmod)
Univariate Type III Repeated-Measures ANOVA Assuming Sphericity
Sum Sq num Df Error SS den Df F value Pr(>F)
(Intercept) 10889.5 1 753.70 273 3944.2942 < 2.2e-16
curstate 5.4 1 753.70 273 1.9614 0.16250
predout 0.0 1 753.70 273 0.0145 0.90412
curstate:predout 118.3 1 753.70 273 42.8676 2.893e-10
tempdist 25.8 1 357.03 273 19.7287 1.298e-05
curstate:tempdist 1.8 1 357.03 273 1.3894 0.23953
predout:tempdist 8.6 1 357.03 273 6.5621 0.01095
curstate:predout:tempdist 365.4 1 357.03 273 279.3782 < 2.2e-16
(Intercept) ***
curstate
predout
curstate:predout ***
tempdist ***
curstate:tempdist
predout:tempdist *
curstate:predout:tempdist ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Again, there is no output for the test of sphericity and no difference with the multivariate analysis of variance model because each measurement per person has it’s own variance and correlation parameter, but there is a single comparison for the likelihood scores (for the temporal distance).
Since the three-way interaction is statistically significant, we need to compare each cell within the levels of the other factor: for example, we could fix the temporal horizon to 1 year and compare the predictions based on both current state and predicted outcome.
library(emmeans)
library(ggplot2)
emm <- emmeans(mmod,
specs = c("curstate","predout"),
by = "tempdist")
emmip(emm,
formula = curstate ~ predout | tempdist,
CIs = TRUE) +
theme_classic() +
theme(legend.position = "bottom") +
labs(x = "predicted output",
color = "current state")
The interaction plot suggest quite strongly that the difference in the reaction is much more pronounced score-wise for short-term horizon predictions than for the long-run. The authors commented this finding by stating
when the current state is characterized by war (intergroup conflict), peace is seen as significantly more likely in the distant future than in the near future. In contrast, when the current state is characterized by peace, war is seen as significantly more likely in the distant future than in the near future.