Research2000 published political tracking polls, where they ask the same question, week after week. The results of such polls can be compared over time:
From this, we can look at the change from week to week, and then count how many 1% changes occurred, 2% changes, etc:
What is strange about this data? Look at the sharp dip at 0% change. Let’s compare this to the changes in a similar Gallup poll:
Ok, so it’s weird, but could it come from actual changes in opinions of the populace? What if there are weekly oscillations such that the public opinion never remains the same week to week?
Well, even if that were the case, this data would be very strange. Because there are LOTS of +/- 1% changes. 1% is very close to 0%, and with a margin of error of 2%, a 1 point change in public opinion will often show up as a 0 point change in the poll.
To illustrate this, let’s pretend the underlying opinion actually did have weekly changes of only +/- 1%, and then look at what a weekly poll of that opinion might look like:
When we measure this through polling, we will only be asking a randomly chosen subset of the population, and therefore, there won’t be the precise changes you see in the underlying population opinion. To Illustrate this further, let’s look at the histograms of weekly change for the underlying opinion and the simulated polls:
Notice how despite the fact that the underlying opinion has no 0 point changes, the poll of that opinion has lots. What are the odds of it having as few as R2K’s polls have had in the last 74 weeks?
One in 10000000000000000
NOTE: Please see our recent report for more detail, or come back here for future technical write ups. The figures in this posts are just illustrations, check out our data section for the real data