vlt.stats.power.anovaposthocValidation
ANOVAPOSTHOCVALIDATION - Validate the anovaposthoc simulation against analytical results
vlt.stats.power.anovaposthocValidation
Validates the simulation-based power analysis from vlt.stats.power.anovaposthoc
against the analytical solution provided by vlt.stats.power.calculateTukeyPairwisePower.
It uses vlt.stats.artificialAnovTable to generate the base dataset and
retrieve consistent parameters for both simulation and analytical calculation.
It performs the following steps:
1. Creates a simple, balanced 1-way ANOVA dataset using
vlt.stats.artificialAnovaTable and retrieves the parameters used.
2. Defines a range of effect sizes (differences) to test.
3. For each effect size, it calculates the statistical power using both the
simulation method (on the generated table) and the analytical method
(using the retrieved parameters).
4. Plots the results from both methods on the same axes for visual comparison.
5. Displays a table comparing the simulated and analytical power values.
**Understanding Discrepancies Between Simulation and Theory:**
It is common to observe that the simulated power (blue circles) is slightly
higher than the theoretical power curve (red line), especially in the
middle range (e.g., power between 0.3 and 0.8). This is generally *not*
an error in the simulation but reflects the difference between assuming
perfect knowledge versus estimating parameters from data:
* **Analytical Power:** Assumes the true population variance (Mean Squared
Error, MSE) is known exactly and is fixed.
* **Simulated Power:** In each simulation run, the ANOVA model *estimates*
the MSE from the randomly generated sample data. Due to sampling
variability, this estimated MSE will fluctuate around the true value.
* **Effect:** When, by chance, the estimated MSE in a simulation run is
*lower* than the true MSE, the statistical test (e.g., Tukey's HSD)
has an easier time detecting a significant difference. Over many
simulations, these instances slightly inflate the average observed power
compared to the theoretical calculation.
Therefore, the simulation often provides a slightly more realistic estimate
of the power achievable when analyzing actual experimental data, where the
true variance must also be estimated. The analytical curve serves as a
valuable theoretical benchmark under ideal conditions.
Optional Name-Value Pairs:
'numShuffles' - The number of shuffles for the simulation (default 2000).
Higher numbers give smoother curves but take longer.
'alpha' - The significance level (default 0.05).