An experimental data set; illustrates analysis of experimental data

Robbins et al. (2024) data set used to illustrate analysis of experimental data.

Usage

robbins_pilot

robbins_main

Format

An experimental data set

cong_overall: the outcome variable, on seven-point Likert scale from strongly approve to strongly disapprove. "Overall, do you approve or disapprove of how [Republicans/Democrats] in Congress handled the situation?
failure: experimental variable, indicates whether the operation failed or succeeded
amplify: experimental variable, indicates whether Congress amplified or ignored the whistleblower
pid_strength: control variable, measured pre-treatment. 0 means independent, 1 leaner, 2 weak partisan, and 3 strong partisan.
pid7: control variable, measured pre-treatment. Values are respondent's party identification on seven-point scale.

An object of class data.frame with 1462 rows and 5 columns.

Details

They use a 2 x 2 factorial vignette design, randomly assigning each respondent to read one of four vignettes. The vignette describes a hypothetical covert operation that ends in either success or failure. Then, the vignette describes a whistleblower coming forward and the legislature chooses to either amplify or ignore the whistleblower. The vignette always describes Congress as being controlled by the respondent's party and the president as being in the respondent's out-party. For example, for a Republican respondent, the vignette describes the legislature as "Republicans in Congress" and the president as the "Democratic president."

After the vignette, they ask the respondent whether they approve of Congress' actions.

They hypothesize that respondents will be more approving when Congress amplifies the whistleblower, but especially when the operation failed.

References

Robbin, Caroline, et al. 2024. "Overt Consequences of Covert Actions: Success, Failure, and Voters’ Preferences for Legislative Oversight." doi:10.31235/osf.io/9p5h8

Examples


# load Robbins et al.'s data
robbins_pilot <- crdata::robbins_pilot  # pilot data
robbins_main <- crdata::robbins_main    # main data

# example analysis

# center control variable at mean
robbins_pilot$rs_pid_strength <- robbins_pilot$pid_strength - mean(robbins_pilot$pid_strength)
robbins_main$rs_pid_strength <- robbins_main$pid_strength - mean(robbins_main$pid_strength)

# fit model to pilot data set
fit_pilot <- lm(cong_overall ~ failure*amplify*rs_pid_strength, data = robbins_pilot)

# fit model to main data set
fit_main <- lm(cong_overall ~ failure*amplify*rs_pid_strength, data = robbins_main)

# "predict" standard error from main data set using pilot data for interaction

# extract needed quantities
n_pilot <- nobs(fit_pilot)/4  # observations per condition
n_main <- nobs(fit_main)/4  # observations per condition
se_hat_pilot <- sqrt(diag(vcov(fit_pilot)))["failureFailure:amplifyAmplify"]  # se from pilot

# prediction
sqrt(n_pilot/n_main)*se_hat_pilot
#> failureFailure:amplifyAmplify 
#>                     0.1803322 

# conservative prediction
sqrt(n_pilot/n_main)*((sqrt(1/(2*n_pilot)) + 1)*se_hat_pilot)
#> failureFailure:amplifyAmplify 
#>                     0.1950809 

# actual
sqrt(diag(vcov(fit_pilot)))["failureFailure:amplifyAmplify"]   # se from main
#> failureFailure:amplifyAmplify 
#>                     0.3987599 


# creating a two-group example

# drop success condition
# - treatment: amplify
# - control:   ignore
# (operation always fails)

# drop success condition
robbins2 <- robbins_main |>
  subset(failure == "Failure")

# t test
t.test(cong_overall ~ amplify, data = robbins2)
#> 
#> 	Welch Two Sample t-test
#> 
#> data:  cong_overall by amplify
#> t = -16.582, df = 724.34, p-value < 2.2e-16
#> alternative hypothesis: true difference in means between group Ignore and group Amplify is not equal to 0
#> 95 percent confidence interval:
#>  -2.286581 -1.802466
#> sample estimates:
#>  mean in group Ignore mean in group Amplify 
#>            -1.2066116             0.8379121 
#>