ANOVA Post-hoc tests

Dr. Peng Zhao (✉ peng.zhao@xjtlu.edu.cn) Department of Health and Environmental Sciences Xi’an Jiaotong-Liverpool University

1 Learning objectives

What is a post-hoc test and why we need it.
Carry out step-by-step post-hoc tests, including the Fisher’s Lease Significant Difference test and the Bonferroni t-test, for ANOVA.

2 Post-hoc tests

Post-hoc

Latin, “after this”
Applied only after the ANOVA that yields a significant difference.

Example: Rats on diets A biologist studies the weight gain of male lab rats on diets over a 4-week period. Three different diets are applied.

dtf <- data.frame(diet1 = c(90, 95, 100),
                  diet2 = c(120, 125, 130),
                  diet3 = c(125, 130, 135))
dtf2 <- stack(dtf)
names(dtf2) <- c("wg", "diet")
wg_aov <- aov(wg ~ diet, data = dtf2)
summary(wg_aov)

Which group(s) is/are different from others? Post-hoc test. Visually:

library(ggplot2)
ggplot(dtf2) + geom_boxplot(aes(wg, diet))

3 Fisher’s Least Significant Difference (LSD) Test

3.1 Principle

Pair-wise comparisons of all the groups based on the t-test.

L S D = t_{α / 2} \sqrt{S_{p}^{2} (\frac{1}{n_{1}} + \frac{1}{n_{2}} + \dots)}

$S_{p}^{2} = \frac{(n_{1} - 1) S_{1}^{2} + (n_{2} - 1) S_{2}^{2} + (n_{3} - 1) S_{3}^{2} + \dots}{(n_{1} - 1) + (n_{2} - 1) + (n_{3} - 1) + \dots}$

$S_{p}^{2} :$ (pooled standard deviation; some use Mean Standard Error)
$t_{α / 2} : t$ critical value at $α = 0.025$
Degree of freedom: $N - k$
- $N$ : total observations
- $k$ : number of factors

Usage

If $| {\bar{x}}_{1} - {\bar{x}}_{2} | > L S D$ , then the difference of $x_{1}$ group and $x_{2}$ group is significant at $α$ .
In multiple comparisons ( $k$ factors), the number of comparison needed is: $\frac{k (k - 1)}{2}$

3.2 Example

library(agricolae)
LSD.test(wg_aov, "diet", p.adj = "bonferroni")

Conclusion: At $α = 0.05$ , Diet 2 and Diet 3 are significantly different from Diet 1 in the mean weight gain, while Diet 2 is not significantly different from Diet 3.

4 Bonferroni t-test

4.1 Principles

A multiple-comparison post-hoc test, which sets the significance cut off at $α / m$ for each comparison, where $m$ represents the number of comparisons we apply.

Overall chance of making a Type I error:

m <- 1:100
siglevel <- 0.05
1 - (1 - (siglevel / m)) ^ m

4.2 Example

Example: Rats on diets

diet_pt <- pairwise.t.test(dtf2$wg, dtf2$diet, pool.sd = FALSE, var.equal = TRUE, p.adj = "none")
diet_pt$p.value < 0.05/m 

pairwise.t.test(dtf2$wg, dtf2$diet, pool.sd = FALSE,var.equal = TRUE, p.adj = "bonferroni")
diet_pt$p.value < 0.05

Conclusion: the same as the LSD test.