Contrasts and multiple testing

Content

  • Contrasts
  • Multiple testing

Learning objectives

At the end of the session, students should be capable of

  • specifying and calculating custom contrasts in factorial designs
  • determining the number of tests in a family that need to be corrected for
  • understanding how to correct p-values to account for multiple testing
  • listing multiplicity testing methods suitable depending on context

Readings

Complementary readings

  • Gelman & Carlin (2014)
  • Chapter 5 of Maxwell et al. (2017).
  • Chapters 4 and 6 of Keppel & Wickens (2004).

Slides

View all slides in new window Download PDF of all slides

Tip

Fun fact: If you type ? (or shift + /) while going through the slides, you can see a list of slide-specific commands.

References

Gelman, A., & Carlin, J. (2014). Beyond power calculations: Assessing type S (sign) and type M (magnitude) errors. Perspectives on Psychological Science, 9(6), 641–651. https://doi.org/10.1177/1745691614551642
Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher’s handbook. Pearson Prentice Hall.
Maxwell, S. E., Delaney, H. D., & Kelley, K. (2017). Designing experiments and analyzing data: A model comparison perspective (3rd ed.). routledge. https://doi.org/10.4324/9781315642956
Meier, L. (2022). ANOVA and mixed models: A short introduction using R (Chapman & Hall/CRC, Eds.). https://doi.org/10.1201/9781003146216