The Front-End of Methods Training

Based on my own experience and interactions with other professors and students, most methods training in political science starts with a “baby stats” course, continues into a more detailed course on linear models, and finishes with a fairly rigorous course on the generalized linear model that includes a grab bag of the latest and greatest methods. In my experience the detail and breadth of these courses increases as the students goes along. Related to this, departments (with limited methods-oriented faculty) tend to devote their methodologists to the more advanced courses and, if necessary, use more substantive-oriented faculty for their introductory courses. My experience in the statistics department at Florida State suggests a slightly different approach might train students more effectively. While a course on the GLM is crucial (I think I’ve used logit in every paper I’ve ever written), a thorough course in probability seems just as important to me. So what are the key ideas that students should learn in an introductory methods course?

  1. point estimation
  2. interval estimation
  3. hypothesis testing

This could be done in the context of differences-in-means and a simple linear regression with a single explanatory variable (or even multiple regression). I’ve never used a Chi-square test in an actually application and I’ve certainly never done one by hand, so I don’t really see the point of doing several by hand as part of an applied methods class. Methods training in political science falls short of it’s potential because early methods classes fail to deal head on with these key concepts and then try to build on a nonexistent foundation. To really get a handle on the three key ideas of point estimation, interval estimation, and hypothesis testing, students need to be familiar with some basic principles of probability theory.

  1. probability distributions and random variables  (pdfs/pmfs, cdfs/cmfs, computer simulation)
  2. Bayes’ rule for discrete and continuous events
  3. mean and variance (of a random variable, not the sample mean and variance)
  4. conditional expectation
  5. central limit theorem
  6. sampling distributions

I’d start the class with a scatterplot of two theoretically related variables, such as the incumbent party’s presidential vote share and change in GDP.  I’d ask students to think about how these two things might be related. Based on simply inspecting the scatterplot, I’d ask them two specific questions.

  1. For every percentage point increase in the GDP growth rate, how many percentage points does the incumbent party’s vote share increase? Don’t worry about being exactly correct just come up with a “good estimate." Call this quantity the “effect."
  2. Choose two values that you are “fairly confident” lie above and below the actual effect.
  3. Are you "fairly confident” that that the actually effect is greater than zero?

I’d then set out to tackle these questions throughout the class. These imply other questions as well, such as what makes and estimate a “good estimate” and the technical meaning of “fairly confident.” I’d note that to answer these question, we need a statistical model, so I’d suggest \(y_i \sim N(\mu_i, \sigma^2)\), where \(\mu_i = \beta_{cons} + \beta_{x}x\). I could then note that this and similar models serve as powerful tools for answering these types of questions and that it’s really important to understand the details. I’d jump in with the normal distribution, expanding to other distributions, and working my way down the list, always coming back to the fundamental concepts of point estimation, interval estimation, and hypothesis testing, being vary careful with details and not shying away from the mathematical background. I don’t know what sort of textbook would be appropriate for this style of class. My favorite is Casella and Berger, but that’s much too advanced for an introductory class for political science graduate students. I haven’t spent a lot of time with it, but DeGroot and Schervish seems promising. These are just some initial ideas, so let me know what you think, especially if you disagree.


Advice for Teaching Undergraduate Methods

A friend writes:

I'm wondering if you have any materials and/or advice you'd be willing to share for teaching undergrad research methods.

I have four bits of advice.

  1. Use a textbook. I haven't found one that I love, but choose something and follow it closely. You don't necessarily need to assign readings from it, but you need something to follow. I once tried to teach a class "off the cuff" so that I could adjust to students' needs, progress, and interest. That was a terrible idea. Get a book and follow it. However, don't feel obligated to reach the end of the book.
  2. Do lots of examples. You cannot do too many examples, especially for the calculations. Depending on the amount of math you require them to do "by hand," this might take a whole class period. That's okay. Do lot's of examples. One or two is not enough.
  3. Do lots of in-class exercises. Introduce a concept, discuss it in some detail, and then let students work with it. There is much more opportunity for this in methods classes than in substantive political science classes. Three examples that might be helpful.
    1. I usually talk about concepts and measurement a lot throughout the class. I'll take three abstract political concepts such as war, income inequality, and partisanship and ask students to carefully define the underlying concepts and then develop a concrete, plausible way to measure these concepts. This usually leads to a long and interesting class discussion.
    2. Scatterplots and regression usually take up a substantial chunk as well. I always give students a scatterplot with just a few data points and ask them to draw the line that "best fits" the data. I tell them that we will see who can draw the best line.  We then find the slope and intercept and use that to compute the residuals and then the sum of squared errors. I award an honorary title of their choosing to whoever has the lowest sum of squares.
    3. I spend a lot of time talking about p-values as well. I usually reenact the lady tasting tea with Pepsi versus Pepsi Max. I think this is a nice example for working through the convoluted logic of hypothesis testing.
  4. Quiz often. Depending on your preferences, you may want to include graded quizzes as part of the class. If not, then I recommend doing self assessments at the beginning of each class, just so student can see if they don't quite understand the material. I think it will help you to do these often. It also probably makes sense to discuss these questions after the quiz with the students.

I am indifferent toward software--I've included it and I have excluded it. This semester, I'm going to try doing a little R.


Is OLS BLUE or JUNK?

My favorite pontificators in political science is Fernando Martel Garcia. I got to know him at replication panel ISA, where he quite vigorously opposed the APSR's policy of auto-rejecting replication papers. Fernando recently posted this gem to the PolMeth mailing list.

In the real world computers do not work alone but at the behest of the researcher operating them.  And the problem is that the latter are often trying to solve a different minimization problem. Namely, choosing regressors, samples, time periods, functional forms, measures, proxies, etc. that minimize the p-value of interest. Thus, in the context of research practice, or how scientists go about doing science, it might be more appropriate to say that most OLS estimates are JUNK rather than BLUE.  And so educators ought to do a much better job of teaching research practice and good research design, over and above OLS.

p-values get a lot of hate from many in the methodology community, but I actually like them. In fact, I'm growing more and more frequentist in my thinking. However, if researchers use p-values as their optimization criterion, then we are in rough shape. But what can we expect, since it seems that journal use p-values as a rejection criterion?


What I'm Up to at APSA

I'm doing a paper and a poster at APSA this year. These are a couple of projects that I'm excited about, so I'm looking forward to talking about them.

Friday, 8/29, 10:15-12:00, Marriott Exhibit Hall B North.

You should stop by and chat with me on Friday morning. I'm presenting a poster discussing the nuances of product terms, interaction, and logit models. The key point of the paper is that you need product terms in order to draw confident conclusions about interaction. If you like, you can go ahead and preview the poster, read the paper, and get the code and data from GitHub.

You can click here to add it to you calendar.

Friday, 8/29, 4:15-6:00, Hilton Columbia 4

On Friday afternoon, I'm presenting at a panel on Representation and Electoral Systems. We've got several interesting papers and a couple of great discussants, so it should be fun. You can go ahead and read my paper, get the code from GitHub, and preview my slides.

You can click here to add it to your calendar.


Thoughts on Giving a Great Conference Presentation

Political science conference presentations are typically boring. The presenter mumbles past their time limit about some vague experiment, the audience asks off-topic questions, and I'm engaging in the interesting discussion (that's happening simultaneously on Twitter and is unrelated to the panel). I care about efficiency more than most people. If I believe you are wasting my time, I will tune you out so fast. I carry all my stuff around in my pocket, so I can work. I am hardly a captive audience. Because I usually sit in the back and can see a number of laptop screens, I assure you that others feel similarly.

So, just in time for APSA, below are a list of suggestions to help you jump out of this tiresome, terrible mold, presented roughly in order of importance.

  1. Make your point early and often. I think a great way to start a presentation is "Today, I'm going to try to convince you that..." Be simple and direct from the very beginning. At no point after the first 30 seconds of the talk should anyone need to ask you what your point is.
  2. Never go over your allotted time. If the chair allots 12 minutes, finish in ten. Going over your allotted time is disrespectful to the audience and the other panelists.
  3. Practice, practice, practice. Own it. I think practicing about 10 times is a minimum. The first 30 seconds is the most important.
  4. Start with some sort of hook. You have 30 seconds to earn your audience's attention for 12 minutes. You can find plenty of suggestions for this using Google.
  5. Include little text on your presentation slides. You must recognize that your audience cannot read and listen at the same time. If you put a large chuck of text on the slides, you must give your audience time to read it, before talking. If you put all your thoughts in the slides, you might as well simply email them around and skip the talk--it is not doing anyone any good. Instead, use the slides for short statement to orient your audience in the direction of your talk and graphs. As an example, have a look at some slides I've used in the past.
  6. Pause, often, throughout the talk. Give your audience a chance to catchup. Periods, paragraphs, section heading, and chapters all signal readers that a transition is happening. You need to pause at the end of thoughts and give your audience a chance to digest the point, gather themselves, and get ready for the next point. What seems like an eternity to you as a presenter is like a cool summer breeze to your audience. Pauses are incredibly powerful. It sometimes takes people a while to wrap their head around something and collect their thoughts.
  7. Give pointers often throughout the talk. "Before jumping into why I think that [your point], let me explain why this is an import point to make." "Now that I've explained why I think that [your point] from a theoretical perspective, I'd like to show you some data that support my point as well." This goes along nicely with the pauses above.
  8. Have notes. Look at them--not your slides. No one will freak out if you stop talking and look at your notes. In fact, they'll appreciate the breather.
  9. Choose carefully what goes into your talk. Your job is not to go through everything in the paper. It is to state the main point of the paper and a brief argument for it. This might mean that you talk about only one of the twelve hypotheses. I might mean you talk only about the theoretical model or empirical results. It might mean that you skimp on one or the other.  For example, here's a 12 mintute presentation I'm giving about this paper at APSA. The paper has a formal model and an empirical analysis. I don't feel like 12 minutes is enough time for both, so the presentation only makes a passing mention of the formal model. Instead, I focus on (1) the theoretical intuition and (2) plots of the data.
  10. Never apologize to start a presentation--own it. Never start with administrative stuff, own it. Make your point. If you need to say something like please interrupt with questions, do it after getting the audience's attention. If you want people to hold questions until the end, at least don't tell them that.
  11. Connect with people. Look them in the eye. I struggle with this more than anything.

I have strong views on a lot of things. Feel free to take my views seriously or not. I hope, however, that you'll find them useful.


For more posts, see the Archives.