John Sides is associate professor of political science at George Washington University and the coauthor of Campaigns and Elections (Norton). He cofounded and contributes to The Monkey Cage, a politics blog. Lynn Vavreck is associate professor of political science and communications at the University of California, Los Angeles. Her books include The Message Matters: The Economy and Presidential Campaigns (Princeton). She cofounded and contributes to the Model Politics blog. Together they are the authors of The Gamble: Choice and Chance in the 2012 Presidential Election
In July 2013, they answered some questions about this ambitious writing project, their analysis of the outcomes and happenings of the 2012 Presidential Election, and what’s up next for each of them.
Q: Why did you write THE GAMBLE?
John Sides: During every election, there is an ongoing conversation among journalists, politicos, and others about what is happening, what it means, and, ultimately, why the winner won. Political scientists typically enter that conversation only much later, given how slowly we work. By that point the conventional wisdom is hardened and it is much harder for us to have any impact. Lynn and I wanted THE GAMBLE to be part of that conversation as it was actually happening.
Lynn Vavreck: Yes, too often the day to day reporting from the campaign trail is mistaken for assessments of what is pivotal in elections, when, in reality, it is just a reflection of what is going on on any given day in a particular place and time. We wanted to separate the “what are campaigns doing and saying today” reporting from a real analysis of what is likely to make a difference to voters on election day by demonstrating that much of what happens on the campaign trail will not be relevant to most voters.
This is not a new observation—political scientists have known this for more than 60 years; but saying it out loud during the campaign was new. THE GAMBLE ultimately gives political science a voice in the ongoing narrative about what mattered in 2012 and demonstrates that what we do as a field is valuable to understanding elections and partisan politics generally.
Q: What were some of the challenges you faced during the writing process?
LV: Science usually takes a lot of time—you test, retest, poke, and prod until you are sure of your findings. In this case, we were trying to do science in real time, which was really hard. We had an entire discipline of robust findings to draw upon, which gave us a pretty good sense of where we expected to find things that mattered as we looked at 2012 in real time, but still, the pressure to get the data, analyze it, understand it, try to break the results every way you can think of, and then write about the conclusions—on a weekly basis—was both stressful and exhausting. There were days, being on two separate coasts, that we literally worked every hour of the day because John would stay up working in DC until I got up in LA and then I would stay up until he woke up the next day. At times it seemed like an insane undertaking.
JMS: We were trying to do two things simultaneously. One was write a serialized account of the campaign, which allowed us to release e-chapters about the campaign and to finish the entire book in a timely fashion. The other was to follow the day’s events and use our data to write blog posts and op-eds. At times, saying something about what was happening at that moment made it hard to write chapters about what had happened six months beforehand.
Q: You mention the serialization of the book. Several of the book’s chapters were written as the election was unfolding and released as short e-books. When you were assembling the complete book, did you have to go back and revisit those chapters? Did subsequent events change what you wanted to say in those chapters?
JMS: I think we were pleased at how well the e-chapters held up after the election was over. I think one reason is that our analysis was guided by decades of political science research and the election ultimately ended up bearing out that research nicely.
LV: One of the challenges of writing a book like this, serially, is keeping the narrative consistent and whole when you don’t know how it is going to end. We had to do some very hard thinking early on about what we thought was going to happen and how we could show the key parts of that story with data and evidence; and then we had to hope that we were right, not only because we were out there writing about it in real time, but because the book would have been a little schizophrenic if the framework or arguments were changing week to week! We put in the hours in the beginning to develop a frame for the book that was flexible enough to encompass all the things we thought might happen, but specific enough to be interesting and distinctive.
Q: How did your models perform through the election? Were they accurate? Did you find any big surprises or disappointments?
JMS: Lynn and I had an early sense—informed in part by our trips to the Iowa caucus and New Hampshire primary—that Romney was the clear favorite to get the Republican nomination, and we said as much in blog posts in January. That proved correct. Then, together with Seth Hill, Lynn and I developed a simple forecasting model for the Washington Post’s Wonkblog, which we published in June. That model turned out to be very accurate as well, forecasting Obama’s share of the vote within a percentage point. In short, our message was that even in a slowly growing economy, the incumbent was favored to win—and that proved true.
Two things that did surprise me were, first, that Gingrich and Santorum managed to surge again after Romney’s victory in New Hampshire. I probably expected Romney to wrap up the nomination a little faster. A second thing was the first debate in October. I was surprised that Obama’s performance was judged so harshly by many in the media—even liberal commentators. This may have helped the polls shift so sharply in Romney’s favor.
Q: The media called the first debate a “game changer.” In fact, the media wrote about a lot of “game changers” throughout the election. Were there any actual “game changers”? What was the most over-hyped part of the Presidential Election?
JMS: There were two moments that did appear to move the polls appreciably: the Democratic National Convention and the first debate. I’d say those two moments were important, although “game changer” always strikes me as hyperbole. Perhaps the most overhyped moment was the release of the “47% video.” At most, this cost Romney a couple points—almost entirely among Republicans who quickly rallied to Romney again after the first debate. For an event that commentators were quick to call “devastating” to Romney’s campaign, the effects of the video seemed to wear off awfully quickly.
Q: You have a section of the book that talks about a misguided portrayal of the undecided voter. Can you describe your findings?
LV: Since the race was so stable for so long, the media picked up on the fact that about 3-5% of likely voters were undecided. It was actually a bit more than that in our data, but the focus of course turned to questions like, “Who ARE these people? Do they live under a rock?” and things suggesting they were dumb or in some way ill-equipped to be good citizens. We had more data on undecided voters than anyone and a lot of what we could say about them based on the data countered the caricatures that were being presented in the media.
Yes, undecided voters are less interested in politics, of course they are; and truthfully, about half of them end up staying home on Election Day. But, they are not so disengaged from politics that they don’t identify with a party—a lot of them do! Mainly, they seem to be people who are otherwise busy, somewhat interested in politics, but not yet ready to pay attention to how they will vote on Election Day. They were less enamored with their party’s nominees, to be sure, but they also just knew less about the nominees in general and had fewer opinions at all about policies. Not dumb. Just not as interested as political junkies, I would say. Politics is not a hobby for them the way it is for a lot of people who are glued to cable news and the Huffington Post. It also ended up not being true, as so many “analysts” insisted, that undecided voters always break for the challenger.
Q: Another running theme of the election coverage was the Obama camp’s flawless campaign. Is this substantiated by the data?
LV: Flawless? Probably not. But innovative? Absolutely. Innovative in terms of the way they fed all the information the campaign was gathering across efforts into one database every day. They leveraged information better than any presidential campaign in history, of that I am quite sure. On data infrastructure they were pioneers.
JMS: Too often what happens after the election is that the winner’s campaign gets too much credit and the loser’s campaign too much blame. Our data shows that the Obama campaign was able to gain a small advantage when it beat Romney on the airwaves and by having a more sophisticated operation on the ground. But it was unlikely this advantage was the determining factor in this race. Obama won by a margin larger than what his campaign alone could be expected to have produced.
LV: Counter to what a lot of people have written about the Romney campaign, John and I actually conclude in THE GAMBLE that he did just about as well as he could have done There were no major strategic errors, he just was disadvantaged by the structural conditions from the start and couldn’t overcome that disadvantage. To put this in perspective, only 4 presidential candidates in the last 60 years have been able to do this.
Q: So, let’s talk campaign strategies—ads, field offices, message. What does your analysis reveal about effectiveness of the core parts of the presidential campaigns?
LV: Political ads are always interesting in real time, and they provide so much material for journalists and analysts to talk about; but in terms of their impact, it is quite fleeting. Most of the effects of political ads decay within a day for presidential races and 3-4 days for down-ballot races. Some of the effect persists and accumulates over time, but not much. There is very little doubt in our minds that the ads Obama aired during the last week of the campaign had a much larger effect on his vote share than those ads he ran in the summer about Bain Capital.
JMS: We did find temporary, but noticeable shifts when one candidate was able to air more advertisements than the other. The problem, however, was that it was hard for either Obama or Romney to do so very often, or for very long. We also found that the number of field offices in a county was associated with increased vote share, though more for Obama than Romney. But, again, the apparent effects of field offices were not large enough to constitute the winning margin in the battleground states.
LV: From my perspective, what worked for Obama was sticking to his message about the economy—that he never wavered, never tried to disown the economy—that was his key strategic choice that led to victory. I know people think he should have lost because of the economy, but they are wrong. The economy was growing, albeit slowly, but it was growing. And incumbent party presidents in growing economies, even slowly growing ones, typically win—and more so when they talk about the economy in their campaigns. Obama did this—good strategic choice.
Q: It seems like there is a lot of unsubstantiated information in political and, in particular, election rhetoric. Where can voters find out what is really happening and what really matters?
LV: The problem is that reporters are not supposed to do this kind of digesting—they are meant to report. Opinion columnists are meant to share their thoughts, which is also different. This brings us to analysts. Analysts are the answer to your question, but the problem is that the major TV networks have turned to getting retired partisan consultants as “analysts” instead of people who are trained in quantitative analysis—and that means that what passes for analysis is stumping for their side. I think these types of guests have a role to play on 24 hour TV news—they are interesting and have done things few people have done; but to identify them as experts on elections writ large is a grave mistake.
JMS: The good news for voters is that there is an increasing amount of data-driven reporting and commentary during elections. I think about not only what Lynn and I were writing at The Monkey Cage, Model Politics, and elsewhere, but about the work of Nate Silver at 538, Mark Blumenthal and Simon Jackman at the Huffington Post, and Drew Linzer at Votamatic. If you followed what we were all writing, I think it gave you a very solid sense of which events were important and where the race ultimately stood along the way, including on the eve of the election.
Q: How have more traditional journalists responded to your work?
JMS: We have been quite pleased by the number of people who followed and responded to our work—even if they didn’t always agree. We had ongoing conversations with reporters during the campaign itself—people like Ryan Lizza, Sasha Issenberg, Dan Balz, and others—and were able both to learn from them and to convey our understanding to them.
LV: As a whole, the entire journalistic community was amazing. From day one they welcomed us in to their bunker—sharing drinks with us in Des Moines, Iowa, sitting down with us to talk at length about past campaigns and the lessons they bring to understanding 2012, and even unexpectedly giving us prime outlets for our work. Our ideas and our work ended up on cable news outlets, in major national newspapers, and in news magazines. We were overwhelmed by the response from the journalistic community and the seriousness with which they took our work. It vastly exceeded our expectations.
Q: One of the big stories following the election was the accuracy of data-crunchers like Nate Silver, Drew Linzer and Simon Jackman and the “rise of political data science.” Do you think this will change the way elections are covered or will we return to status quo for the next election?
JMS: I think they showed us that you can use data in ways that actually make the story of an election more interesting, not less interesting. You don’t have to be a reporter with great inside gossip to attract readers. Geeks can do it too. My hope is that when 2014 or 2016 rolls around, at least a few of the people who were so wrong about 2012 will be chastened, and in general there will be far less tolerance for hyperbole about game-changers or predictions that aren’t borne out by hard data.
LV: The data revolution is upon us—that battle is won. There is a huge appetite for numbers, not just in politics, but also in other places, like sports, dating and music preferences. The next step in political data analytics is something that we are actually both working on advancing right now. I think the future probably holds less pure forecasting or aggregating of polls like Nate Silver, Simon Jackman, and Drew Linzer did in 2012 and more analysis of the actual information in polls—but in a big data sort of way.
Q: This was an ambitious project. Do you think we’ll see more political scientists follow suit?
JMS: There were features of this project that would be hard to emulate. We were very fortunate to have access to incredible data—polling data from YouGov, media data from General Sentiment, advertising data via the Washington Post. It would be hard to put that together again. At the same time, writing this book has made me feel that there is real room for innovation in aspects of scholarly publishing. Thanks to the cooperation and leadership of Princeton University Press, I think THE GAMBLE shows that there is value in non-traditional forms of publishing, like the serialized e-chapters we produced. Ultimately, I think THE GAMBLE shows that scholars, and a scholarly press, can produce a book that is both rigorous and timely.
Q: The 2012 election is behind us. Will you attempt to do this type of research during the 2016 elections? Without an incumbent president, it seems like that might provide a terrific opportunity to crunch numbers and create new models.
JMS: I’m not sure that I’m ready to do this again in 2016, even though it is shaping up to be a fascinating election. I think Lynn and I will always be interested in elections and will continue to research and write about them, in addition to our work on other topics. But we’re both committed to the idea motivating THE GAMBLE—which was to bring academic ideas to a broader audience—and we will certainly continue to do that going forward.
LV: The project was so exhausting it is hard to imagine saying I would do it again; however, we accomplished so much and were taken so seriously by the people writing about the campaign and reporting on it in real time, that it would feel like an opportunity lost not to continue to have that role filled. So yes, I suspect you will see more from us in popular outlets over the next couple of years. As for 2016, who knows. I would love to think we could do it again, but that would depend on more than just our willingness to make sacrifices—all those generous data providers would have to line up again with us or someone would have to fund the data collection, and that is a pretty big ask. THE GAMBLE’s analysis relies on over a million dollars’ worth of survey data—I’m not sure we’re ever going to be able to do that again.
Q: So, what’s next for both of you? What are you currently working on?
JMS: Lynn and I have a paper under review about the effects of field offices in 2012, and another in the early stages questioning the conventional wisdom that voters in primary elections are much more ideologically extreme than voters as a whole. I’m contemplating a second book project on a completely different topic: the federal budget—how the public reasons about budgets and deficits, how budgets get made in Congress, and what could make the budgeting process better.
LV: I have spent the weeks since THE GAMBLE shipped finishing up a round of papers on a basic science project I started in 2011 that is funded by the National Science Foundation. The work examines the possibilities of moving federally funded survey work away from in-person, in the home interviews and to a computer-assisted interview mode. There is an innovative sampling framework that I tested and an experiment done in Las Vegas, Nevada on how the mode of interview affects the answers people give. Hopefully, the work will help the federal government save tens of millions of dollars on survey research.
Other than that, I’m thinking generally about how to advance the analysis of political data in the public space. John and I have a few irons in the fire and hopefully one or two of them will pan out. Stay tuned for our next adventure.