Three-Level Cluster Randomized Trial

2018-02-16

To install PowerUpR:

install.packages("PowerUpR")

Institue of Education Science (IES) reported that evidence on effectiveness of Saxon Math curriclum on secondary school student’s algebra subject is lacking and further research is needed (U.S. Department of Education, 2016). To evaluate the effectiveness Saxon Math curriculum, assume a three-level cluster randomized trial is considered where schools are randomly assigned to treatment (Saxon Math curriclum) and a control condition. In the grant proposal number of schools, teachers, and students, and the cost associated with data collection from this sample needs to be justified. See Vignette B for designing a three-level cluster randomized trial that is sensetive to moderator effect.

Due to nesting of students within classrooms and nesting of classrooms within schools, the treatment effect (\(\delta\)) can be estimated via the following 3-level hierarchical linear model (HLM, Raudenbush & Bryk, 2002)

\[\begin{eqnarray} L1: Y_{ijk} &=& \beta_{0jk} + \beta_{1jk}X + r_{ijk}, \quad r_{ijk} \thicksim N(0,\sigma_{|X}^2) \newline L2: \beta_{0jk} &=& \gamma_{00k} + \gamma_{01k}W + \mu_{0jk}, \quad \mu_{0jk} \thicksim N(0,\tau_{2|W}^2) \newline \beta_{1jk} &=& \gamma_{10k} \newline L3: \gamma_{00k} &=& \xi_{001} + \delta T_k + \xi_{002}V + \varsigma_{00k}, \quad \varsigma_{00k} \thicksim N(0,\tau_{3|V}^2) \newline \gamma_{01k} &=& \xi_{010} \newline \gamma_{10k} &=& \xi_{100} \end{eqnarray}\]

where, \(Y\): Student algebra posttest score. \(X\): Student algebra pretest score. \(W\): Classroom mean algebra pretest score. \(V\): School mean algebra pretest score. \(T\): Treatment status. Consider the following hypothetical values for design parameters

Proportion of variance in algebra posttest

• at the school level (\(\tau_3^2\)): \(.15\)
• at the classroom level (\(\tau_2^2\)): \(.05\)
• at the student level (\(\sigma^2\)): \(.65\)
• explained by school mean algebra pretest: (\(R_3^2 = 1 - \frac{\tau_{3|V}^2}{\tau_{3}^2}\)): \(.45\)
• explained by classroom mean algebra pretest: (\(R_2^2 = 1 - \frac{\tau_{2|W}^2}{\tau_{2}^2}\)): \(.50\)
• explained by student algebra pretest: (\(R_1^2 = 1 - \frac{\sigma_{|X}^2}{\sigma^2}\)): \(.55\)

Then, intraclass correlation coefficients are

• \(\rho_2 = \tau_2^2 / (\sigma^2 + \tau_2^2 + \tau_3^2) = .05 /(.65 + .15 + .05) = 0.059\)
• \(\rho_3 = \tau_3^2 / (\sigma^2 + \tau_2^2 + \tau_3^2) = .15 /(.65 + .15 + .05) = 0.176\)

Further assumptions

• Alpha level (\(\alpha\)): \(.05\)
• Number of schools (\(K\)): \(83\)
• Proportion of schools in treatment condition (\(p\)): \(.40\)
• Average number of classrooms per school (\(J\)): \(2\)
• Average number of students per classroom (\(n\)) : \(10\)
• Statistical power (\(1-\beta\)): \(.80\)

Given design parameters and further assumptions, what is the minimum detectable effect size?

mdes.cra3r3(power=.80, rho2=.06, rho3=.18,
            g3=1, r21=.55, r22=.50, r23=.45,
            p=.40, n=10, J=2, K=83)

## 
##  Minimum detectable effect size: 
##  --------------------------------------- 
##  MDES is 0.23 95% CI [0.069,0.392] with 80% power 
##  ---------------------------------------
##  Degrees of freedom:80
##  Standardized standard error:0.081
##  Type I error rate:0.05
##  Type II error rate:0.2
##  Two-tailed test:TRUE

Given design parameters and further assumptions, what is the statistical power?

power.cra3r3(es=.23, rho2=.06, rho3=.18,
             g3=1, r21=.55, r22=.50, r23=.45,
             p=.40, n=10, J=2, K=83)

## 
##  Statistical power: 
##  --------------------------------------- 
##  80% power to detect an ES of 0.23 95% CI [0.069,0.391] 
##  --------------------------------------- 
##  Degrees of freedom:80
##  Standardized standard error:0.081
##  Type I error rate:0.05
##  Type II error rate:0.2
##  Two-tailed test:TRUE