While most hypotheses in political science suggest positive or negative effects, a substantial subset of important hypotheses suggest that potential explanatory variables should have no meaningful effect on the outcome of interest. Without testing these hypotheses, empirical evaluation of theoretical arguments remains incomplete. Current practice in political science in this situation is to take a statistically insignificant effect as evidence for the research hypothesis, with an occasional discussion of whether or not the test has sufficient power. I explain that this procedure does not align with hypothesis testing conventions in political science and thus is prone to misinterpretations. In particular, a statistically insignificant effect is neither necessary nor sufficient to show that a variable has no meaningful effect. As an alternative, I introduce to political science the intersection-union method for creating tests, which allows researchers to quickly construct tests of hypotheses of no effect using already-available software.

• Rainey, Carlisle. "Testing Hypotheses of No Meaningful Effect." Under Review. [Summary, Poster, Presentation Slides, Manuscript]

Related Work

• Esarey, Justin. "A Formal Test for Substantive Significance Using Bayesian Statistical Decision Theory." Working Paper.
• Esarey, Justin, and Nathan Danneman. "Formal Tests of Substantive Significance for Linear and Non-Linear Models." Working Paper.
• Gallistel, C. R. (2009). The importance of proving the null. Psychological Review 116(2): 439-53.
• Gill, Jeff. "The Insignificance of Null Hypothesis Significance Testing." Political Research Quarterly 52(3): 647-74.
• Rouder, J. N., Speckman, P. L., Sun, D., Morey, R. D., & Iverson, G. J. (2009). Bayesian t-tests for accepting and rejecting the null hypothesis. Pychonomic Bulletin & Review 16: 225-37.
• Wellek, Stefan. Testing Statistical Hypotheses of Equivalence and Noninferiority. Chapman and Hall.