You don't have to be a fantasy
baseball team manager to understand the role of statistics in baseball. With a
quick review of a player's batting average, on base percentage or earned run
average, one can determine how effective a player has been over the course of
his career. These statistics even provide a way to compare players who never
met on the diamond — or even lived at the same time. The exceptions are players
who excelled in the steroids era and justifiably many in baseball put an
asterisk (*) next to those numbers to note that the statistics may be out of
whack because of the players alleged use of performance enhancing drugs.
Statistics — in the form of public opinion polls — are critical in politics too. While I am not asserting that
anyone at the esteemed Taubman Center at Brown University is on drugs, their
polling data is in dire need of an asterisk because each and every time they
conduct a poll, their methodology is so out of whack with accepted industry
practices that their results cannot possibly be accurate. They recently
released a poll that covered the presidential primary as well as several state
political issues ranging from legalization of marijuana to something they
called a “toll tax on large trucks.” While I cannot disagree with their results
because I have no better data to compare them to, their methodology (i.e. how
they go about gathering the data) is so suspect that I can’t put any credence
in their findings.
While there are many problems
with the most recent Brown poll, the first is that they called “registered
voters.” A good political pollster will only call “likely” voters. The
difference? People who are “likely” voters have usually voted in at least 2 of
the last 4 elections, making them more likely to show up again. A registered
voter may or may not have ever voted in his or her life, but he or she has most
certainly been to the DMV (where they were registered). When you ask registered
voters about how they may vote in an election, your results will not be as
accurate because they may never actually participate in the election (note that
only 40% of RI’s eligible voters showed up to vote in 2014). When asked if they
will vote, many registered voters will say yes even if they do not plan to show
up, so asking them directly is not helpful.
The “poll” put Hillary Clinton
up 9% over Bernie Sanders with 16% “undecided.” Chances are good that the
“undecided” number actually represents people who have no idea that there is a
presidential primary this week because people who are likely to vote this week
know who they prefer. In fact, Brown could only scrounge up 164 people who said
they were going to vote in the Republican primary — creating a huge margin of
error in the Republican results. Even worse: Brown put out a statement last
week saying that they were delaying the release of their poll because they were
struggling to complete enough interviews due to “voter fatigue.” A
well-executed poll is like “Fight Club” (there is no fight club) and a good
pollster would not poison his or her sample by letting potential interviewees
know that they might get a call.
Why does it matter? As a
relatively small media market, there are very few public opinion polls released
each year, so when one is released, it takes on an outsized significance. Just
like hearing “9 out of 10 dentists recommend Crest” on TV, people are
influenced by polls. If their candidate is out in front or very far behind,
they may choose to stay home on election day. Polls are also used by campaigns
to fundraise and build momentum and it truly unfair to have false information
floating around in the ether. I do blame the RI news media for promoting it —
the poor quality of Brown polls is well-worn topic among reporters, but the
results still land on the front page of the Journal,
making the state’s paper of record complicit in putting bad information out to
the public.
The hard part is that there’s so
rarely a “good” poll that runs concurrently for an apples to apples comparison,
so political geeks are left to bemoan the methodology while putting an asterisk
next to the numbers and reminding people to vote regardless of what they hear. So, * VOTE!