Power Rules @ UGA

A talk at the University of Georgia.

Paper and slides

• You can find the paper here.
• The GitHub repo for the paper is here.
• You can find the iCloud version of the slides here and a pdf version of the slides here.

Other things from me

• You can find several relevant blog posts on the topic here.
  
• “Arguing for a Negligible Effect” [PDF] describes how you can use an equivalence test to argue in favor of “no effect.”
• “Substantive Importance and the Veil of Statistical Significance” [PDF] makes a more general argument about how we should test claims.

And the code to compute power is here:

# mean and sd
true_effect <- 1.00
se <- 0.4

# compute power
pnorm(1.64*se,             # want fraction above* 1.64 SE
      mean = true_effect,  # mean of sampling distribution
      sd = se,             # sd of sampling distribution
      lower.tail = FALSE)  # fraction above, not below

Other relevant papers

Here are the articles worth reading if you want to learn more:

• Arel-Bundock et al. (2022) for an argument that political scientists need to think harder about statistical power.

• Bloom (1995) on the simple and effective concept of “minimum detectable effects.”

• Lakens (2022) for a clear and complete description of how researchers can justify their sample size using power analysis and other arguments.

• BLAIR et al. (2019) for a comprehensive way of thinking about power along with research designs and their implications much more generally.

References

Arel-Bundock, Vincent, Ryan C Briggs, Hristos Doucouliagos, Marco Mendoza Aviña, and T. D. Stanley. 2022. “Quantitative Political Science Research Is Greatly Underpowered.” http://dx.doi.org/10.31219/osf.io/7vy2f.

BLAIR, GRAEME, JASPER COOPER, ALEXANDER COPPOCK, and MACARTAN HUMPHREYS. 2019. “Declaring and Diagnosing Research Designs.” American Political Science Review 113 (3): 838–59. https://doi.org/10.1017/s0003055419000194.

Bloom, Howard S. 1995. “Minimum Detectable Effects.” Evaluation Review 19 (5): 547–56. https://doi.org/10.1177/0193841x9501900504.

Lakens, Daniël. 2022. “Sample Size Justification.” Collabra: Psychology 8 (1). https://doi.org/10.1525/collabra.33267.