Tuesday's midterm election results seemed to catch many people by surprise. Democrats swept races across the country, wresting control of Congress from Republicans and flipping several states in their favor. But did the polls see this coming? Local Edition spoke with AJ Simmons, research director for the Center for State Policy and Leadership at UIS, to answer that question and look at what they said about a few Illinois races.
—
Ben Singson
We're a few days removed from the so-called "blue wave" that shook up Congress, flipped several historically red states, and handed control of both the House and the Senate to Democrats. Many people were taken by surprise on election nights. They either didn't expect it to happen or didn't expect it to be this strong. But how well did the polls predict this? For more, we turn to AJ Simmons, research director for the Center for State Policy and Leadership at the University of Illinois Springfield. AJ, thanks for taking some time to talk with me.
AJ Simmons
Thank you so much for having me on.
Ben Singson
All right, so Tuesday was a big night for Democrats. What did the polls have to say about this blue wave heading into the election?
AJ Simmons
So, polls suggested a blue wave, but not the blue tsunami that we got, right? National polling range somewhere between like a plus 3 to a plus 14 democratic environment, with the average being something like a plus 10. We're at plus 14 as of I think current count. So some of those more optimistic pollsters who were getting some grief before the election ended up being correct. And I think the more interesting stories when we look at the states, right, looking at the Senate, Polls pretty consistently showed wins for the Dems in Maine, Michigan, Alaska, North Carolina, Texas, along with all the safe seats that we would expect like here in Illinois. Alaska's not been called yet, but the Democrats have a lead currently there in Alaska. I'd even argue the polls suggested a clear Dem victory in Kansas in the Senate race, which is why we end up with like a bunch of people predicting 53-47 Dem controlled Senate maps, because polling all seemed pretty confidently in the direction of the Dems there. We got some mixed signals for Nebraska and Iowa where there was a lot more uncertainty about those races. And then as we got closer to election day, we started to get some polls showing a close race in South Carolina and in Mississippi, ones that had kind of the race really within the margin of error, which is why it's going to be a reoccurring theme in this conversation that people should always keep in mind that margin of error. It doesn't necessarily mean the race is a toss up if that shows up in a poll, but What it does suggest is we don't have strong evidence of like a meaningful difference in the candidates at that point in time. And it could point to a close race. And then the House, the House is really weird because we'd have some seats that had a lot of polling and then other seats had like none, right? Nothing in the House polling suggested this like 250 to 260 Democratic majority that it looks like we're going to end up with. I think polling would have suggested like a 235. Democratic controlled house. So overall, polls suggested a good night for Dems and they ended up with a great night.
Ben Singson
Gotcha, gotcha. Let's zoom in on two races here in Illinois. The first that I wanted to look at was JB Pritzker versus Darren Bailey. That race was called fairly quickly after polls closed. Were the polls expecting such a, I guess, decisive victory over Bailey?
AJ Simmons
So, we didn't actually have a ton of polling here in Illinois, but what we did get consistently pointed to a pretty healthy Pritzker victory. And he ultimately won by 23 points. And that result is kind of particularly interesting for me and NPR Illinois because myself and Nick Waterbury, our assistant research director, published a write-up for you in NPR Illinois where we looked at different ways of weighing the demographics of our survey respondents and how that impacted different results in the governor's race, kind of what the survey said. And what that showed is anywhere from like a 15 point victory to a 30 point victory for Governor Pritzker. So no matter how we weighed the data, it pointed to this big victory for Governor Pritzker. And one of the interesting things that jumps out to me is kind of how when we're doing surveys, we're trying to predict who's going to turn out to vote. And we did one version of our survey where we kind of had this really high likely voter turnout. And then we did a version that we thought was more realistic. And it looks like the realistic one that we did, which is the very last model in that write up for NPR Illinois, actually tracks to what turnout is going to end up being here, which is about 55, 56% of the vote and predicted a 22 point victory for Governor Pritzker. Gotcha. So that's a pretty, pretty, pretty good prediction from that survey. But I think this is a really good example of why we should be public about kind of the assumptions we put in our polling and that sort of stuff. Because we showed like, if we weight it this way, which is a completely defensible way to show Governor Chris Cruz had to win by 15% according to this poll. If we weight it this way, which is also defensible, it's 30 points. Well, okay. And we show all these different combinations aimed at trying to just help voters understand like need to lift up the hood on polling. But polling did suggest a really healthy victory for Governor Pritzker here, and that's what we ended up getting.
Ben Singson
Gotcha, gotcha. The other race that I wanted to look at was Juliana Stratton versus Don Tracy. Stratton, another race called pretty quickly, I think. She will be the replacement for Dick Durbin in the Senate. What did the polls have to say about that race?
AJ Simmons
So again, there were not a ton of polls for this race. There were in fact, I think even fewer polls for this race than the governor's race. Like we didn't even conduct one, for example. But the poll suggested somewhere between a 12 to 17 point victory for Lieutenant Governor Stratton. She ended up winning by, I think, 24 points. So, the polling definitely underestimated her margin. And that's kind of interesting, but there's a number of possible explanations that we could, you know, go into. And I think we might talk a bit about later, but like, what did the models of those polls look like, what kind of were they expecting for the electorate's demographics to be? Random sampling variability, some combination of that sort of stuff could be. I think the bigger surprise for me was one pollster released a poll that had Pritzker winning by 20, 21 points, something like that. And then the same data in the same poll had Stratton winning by only 12 points using the same sample and the same weights, right? That's pretty unusual considering how yoked together Pritzker and Stratton are politically in Illinois. And I want to be clear, I'm not suggesting anything nefarious at all. It's just a good illustration of like the variation that you can get with survey data, particularly when you're dealing with kind of relatively small differences between candidates from the same party, relative few polls, demographic modeling decisions, and like the good old margin of error.
Ben Singson
Gotcha, gotcha. These polls aren't 100% accurate, rarely if ever. Sometimes they underestimate things like with this past Tuesday. Sometimes they're completely out of step, like in the 2016 and 2024 presidential elections. Why do polls get things wrong sometimes?
AJ Simmons
So, I want to be clear about a couple things. When we talk about polls getting it wrong, I think I view it as a little bit more abstract than perhaps say the media does, right? Like there's getting the raw number wrong or the percent wrong. And there's also getting who wins or loses wrong, right? And so, you know, you're right, you know, in 2016 and in 2024, polling had a miss there, right? But we're talking in 2016 by national averages being off by a percentage point. And actually getting the national vote right, it's just a weird election where at the state level there is enough variation. We sort of, I guess, assume that the state level will match the national level often, but we know that that's not true and we have an electoral college. And so the polls were not that far off nationally. Now statewide there was some, and that's a problem of like not doing enough polling at the state level. But I tend to view the polling of the direction. Are they getting the direction of where the race is currently at and where the race is going right? And there have been, even with that more broader definition of wrong, where polls have gotten things wrong. But I think sometimes this narrative of like polling misses are wrong somewhat gets overblown because like Yeah, it's not going to be exact. And so the question becomes like, why do these things get wrong? Well, some is a margin of error. Again, in 2016 and in 2024, the result of the national average being like one or two percentage points off is technically wrong. But the margin of error of those surveys was 3 or 4%, meaning that for each candidate, their range was 3 or 4% of what their average was. But races in that margin of error, we can't really say anything about what the outcome of the race is going to be or even the current pulse of the race because it's within the margin of error and it doesn't give us enough information. We also know that there's things like sampling error where one out of every 20 polls or five out of 100 polls, everything's done right, still leads to an incorrect result. And we have to account for the fact that folks can change their mind. So those are all some big concepts and I want to unpack a little bit more. But that margin of error, let me give an example of this. If we have polls showing candidate A is in the lead, and I'm putting that in quotes, with 52% of the vote, candidate B has got 48% of the vote, but the margin of error is 4 percentage points, we don't really have enough information to confidently say that 52% candidate is in the lead, right? And so having multiple polls pointing in the same direction can give us a little bit more confidence that we're seeing a real pattern rather than just some like random variation. But even then, some of those, these other challenges still matter. And so what are some of these other challenges? One is again, making sure that the polls voters actually reflect the people that are going to be at the polls, right? And that is way harder than it sounds because so much of our likely voter models rely, at least on part, on who has voted in the past, who says they're going to vote, these sort of things. But what happens when a candidate activates people who don't normally vote? If we're relying too heavily on historic voting patterns or people telling us whether they're going to vote, we can miss those people. And then the timing of voter decision making matters too. because people can change their minds. Again, let's give an example of this. You have a race polling at 49% to 45%, and 11% of voters are still undecided. What those undecided voters ultimately decide to do is going to matter quite a bit, right? And then there's also, again, random sampling error that I mentioned earlier. Even when you do everything right, you're still surveying just a sample of people rather than every single voter. We're not doing a census, we're doing a sample, right? And of statistical approaches, help us understand and quantify some of that uncertainty, right, margin of error. But even at a margin of error in calculating that things that like being 95% confident, again, that goes back to this idea of five out of 100 polls are going to fall outside of that margin of error just by random chance. And I just gave all the reasonable, thoughtful explanations for why polls might get things wrong, but there's another possibility that we shouldn't overlook, and that is bad actors intentionally releasing misleading polls. Sometimes you'll see completely fabricated polls, which we saw this election. Sometimes you'll see campaigns release polls that are designed in a way to favor their candidate. They'll ask questions of, "Do you support freedom or do you hate America?" And then they'll be like, yeah, that's the result of our poll. No, that's an actual valuable poll, right? And they do that sort of stuff for all sorts of reasons. Attention, media coverage, fundraising, and some of the darker arts of political campaigns to try and influence results. So, when it comes to campaigns and polls, sometimes it's less about measuring public opinion and more about trying to shape it in some way. Whether that's through putting out a skewed poll or keeping a cold poll quiet because the result is so important, right? So that's a bit of a longer answer, but that's why I ultimately think that it's important to remember that there's a difference between polls being wrong and polls being bad.
Ben Singson
All right. AJ Simmons, Research Director for the Center for State Policy and Leadership. Thank you very much.
AJ Simmons
Thanks so much for having me.
