Recent polls in the seven key states show a surprisingly close presidential race: 124 of the last 321 polls conducted in those states – nearly 39% – show margins of 1 percentage point or less.

In fact, state polls reveal not only a surprisingly close race, but an improbably close race. Even in truly tied elections, the randomness inherent in polling would generate more varied and less clustered results – unless state polls and polling averages are artificially close due to decisions made by the sounders.

The results of a poll depend on the opinions of voters and the decisions of pollsters. Decisions on how to weight polls to match the expected composition of the electorate can move poll results up to 8 points. This is true even if pollsters make entirely reasonable decisions about how to weight their survey data, because survey researchers have been forced to consider new methods and ideas for weighting and remedying falling rates response following missed surveys in 2016 and 2020.

But the fact that so many polls report the exact same margins and results raises a troubling possibility: that some pollsters are making adjustments so similarly that these choices result in results clustering together, creating a potential illusion of certainty – or that Some pollsters even draw inspiration from the results of others to guide their own (i.e. “aggregate”).

If so, the artificial similarity of the polls could create a false impression that might not occur on Election Day. We could well find ourselves in a very close election. But there’s also a good chance that one candidate or the other will win all the key states and win the presidency quite comfortably, at least relative to the balanced polling picture.

What should we see in a perfect polling world due to chance?

In a perfect polling world – a researcher’s paradise in which every voter can be contacted and every contacted voter responds – we can use mathematics to calculate how much variation there should be, because voters are randomly selected to participate in a survey.

If a race in this world were truly tied 50%-50%, not all polls would produce results split 50%-50%. Imagine if the pollsters of this world conducted 100 identical surveys of 863 randomly selected voters (that’s the average sample size of this year’s swing state polls). Results from 95 of these polls would show the candidates with support between 46.7% and 53.3% – even though we know in this fantasy world that the race is actually tied at 50%. The other five polls would show the candidates winning something even larger or smaller outside of that range.

This variation is known as the “margin of error” in a poll – that is, how much the random selection of voters who always respond can affect a poll’s estimate for a candidate.

Because each candidate’s support varies randomly, these polls predict a tie margin that ranges from -6.6 to +6.6 for 95 out of 100 polls (with even larger margins for the other five ).

It’s important to point out that the range of margins we can expect in a tied race (and in a perfect polling world) is much wider than the margins in swing states in 2020. Even in ideal circumstances for a surveyit is difficult, if not impossible, for a poll to be very informative about who is leading a close race. And this is arguably a lower bound than what we should observe in a more complicated real world, where polls vary in how respondents are selected, contacted, and weighted to match the electorate that they pollsters estimate it to be in 2024.

We can also calculate what proportion of polls of 863 people should show varying margins in a truly tied race. Rounded to the nearest percentage point, about 11% of polls in a tied race are expected to show a tie.

This means that almost 9 out of 10 polls in a tied race should not actually yield a tied result, due to randomness and margin of error.

About 32% of polls are expected to have a margin of 1 point or more, 55% are expected to have a margin of 2 points or more, and 69% are expected to have a margin of 3 points or more. Even in a 50-50 race, about 10% of polls should have a margin of more than 5 points due to the inherent randomness – almost the same percentage that shows a (rounded) tie!

With enough polls, the predicted margin should also resemble a normal “bell curve” distribution – with a similar number of polls showing either candidate in the lead.

What do we see in the swing state polls?

Actual polls from swing states show much less variation than the benchmarks we would expect in a perfect polling world. Of the 321 polls conducted in the seven swing states, only 9 polls (3%) show a margin greater than 5 points. Even if all races were tied – which they are not – we would still expect to see about 32 of the 321 polls with a margin of more than 5 points due to randomness.

Visualizing how reported polling margins compare to what we would expect in a perfect polling world strongly suggests “clustering” of swing state polling margins around statewide polling averages . In these 321 state polls, 69 of them (21%) indicate an exact tie and 124 polls (39%) indicate a margin of 1 percentage point or less. Both of these numbers are roughly double what we would expect in a perfect polling world where the only source of variation is the random selection of responding voters.

Pennsylvania is perhaps the most troubling state. In total, 20 of 59 polls (34%) show an exact tie and 26 (44%) show a margin of 1 point or less. And even though there is a 15% chance that a truly tied race could produce a poll with more than a 5-point margin due to chance, we see only 2 of Pennsylvania’s 59 polls (3.3%) with a margin greater than 5 points.

Even where polling results are not as closely clustered, such as in Arizona, Michigan, and Wisconsin, there are still many more polls than one would expect around the polling average and too few polls with large margins.

What is happening?

The concentrated margins we see in swing state polls likely reflect one of two possibilities.

Pollsters may sometimes adjust poll results that seem “weird” to them by choosing a weighting system that produces results closer to those of other polls. It appears that risk-averse pollsters have a strong incentive to do so. Unless a pollster conducts many polls and can be sure that the impact of chance is average, he may fear reputational and financial costs if he gets an erroneous result due to chance, since the Pollsters are rated on the accuracy of their polls.

A risk-averse pollster who has a 5-point margin in a race he thinks is tied may choose to “adjust” the results to bring them closer to what other polls show, lest his poll aberrant negatively affects its reputation compared to its competitors.

Another more likely possibility is that some of the tools pollsters use in 2024 to address 2020 polling problems, such as weighting based on partisanship, past votes or other factors, could iron out differences and reduce variation in reported survey results. The effect of such decisions is subtle, but important, because it means that poll similarity depends on the decisions of pollsters rather than voters.

And if these assumptions are wrong, which can only be known after the election, then the risk of a potentially large polling error increases as the variation between different polls decreases.

Why it matters

The fact that so many swing state polls report similar margins is a problem because it raises the question of whether the polls are tied in these races because of voters or pollsters. Will 2024 be as close as 2020 because our politics are stable, or will 2024 polls only resemble 2020 results because of decisions made by state pollsters? The fact that the polls appear closer than one would expect in a perfect polling world raises serious questions about the second scenario.

Published polls and polling averages create a consensus that the race will be very close and that we will likely see an outcome similar to 2020. That may be true. It would be wonderful if the polls managed to address the concerns of 2016 and 2020 in 2024.

But the fact that the polls all report such similar margins doesn’t necessarily make it more likely that those margins represent the end result. In fact, it raises the possibility that the results of the election could be strikingly different from the very close narrative that overall state polls and polling averages suggest.