WWC clustering correction for mismatched analyses — cluster

Corrects the statistical significance of a finding for clustering, following the What Works Clearinghouse (WWC) procedure based on Hedges (2007). The correction applies when assignment was at the cluster level (classrooms or schools) but the analysis was at the student level, a "mismatch" that leaves the reported standard errors too small. It computes the t statistic implied by the effect size, then corrects both the t statistic and its degrees of freedom for clustering, and returns the clustering-corrected two-tailed p value and significance decision.

Usage

cluster_correction(
  g,
  n_treatment,
  n_comparison,
  m_treatment,
  m_comparison,
  icc = NULL,
  outcome = c("achievement", "other")
)

Arguments

g: Effect size (Hedges' g) ignoring clustering.
n_treatment, n_comparison: Student-level sample sizes.
m_treatment, m_comparison: Number of clusters in each group.
icc: Intraclass correlation. If NULL (default), the WWC default is used: 0.20 for outcome = "achievement", 0.10 otherwise.
outcome: "achievement" or "other", selecting the default ICC when icc is NULL.

Value

A data frame, one row per input, with columns t (ignoring clustering), t_corrected, df (corrected degrees of freedom), p_value (clustering-corrected, two-tailed), and significant (p_value < 0.05).

Details

Because the correction can only reduce significance, the WWC applies it only to findings the study authors reported as statistically significant; a finding that was not significant stays not significant. This function computes the corrected values regardless; apply them where the WWC rules call for it.

References

What Works Clearinghouse (2020). Procedures Handbook, Version 4.1, Appendix F. Hedges, L. V. (2007). Effect sizes in cluster-randomized designs. Journal of Educational and Behavioral Statistics, 32(4), 341-370.

Examples

# A finding with a moderate effect from a clustered design:
cluster_correction(
  g = 0.30, n_treatment = 200, n_comparison = 200,
  m_treatment = 10, m_comparison = 10, outcome = "achievement"
)
#>          t t_corrected       df   p_value significant
#> 1 2.983266    1.348605 231.6576 0.1787808       FALSE