by Charles Platt
I don’t think the Harris polling company has violated any laws, here. But it does look to me as if they have been, perhaps, unethical.
A widely quoted Harris poll claims that 57% of sampled Republicans think that Barack Obama is a muslim. This has been a great gift for liberal commentators who are predisposed to fear and loathe the Tea Party and its supposed influence. But did this gift just materialize from dispassionate and objective data, as if by accident? Or was it a premeditated attempt to create a misleading picture that would demonize a segment of the American public?
First take a look at the actual poll. Here’s the apparent source:
Browsing through it, we find that Republicans are not the only ones who doubt the president’s religion. Even among Democracts, 15 percent supposedly think that he is a Muslim. That translates as more than 10 million people, if we’re sampling the national population of around 70 million registered democrats. This doesn’t make sense to me. It cannot be true–and in fact, it isn’t true.
To find out why, we need to browse a little further.
Here, buried on page 4, is the methodology. It states that the entire poll was conducted online. In other words, this poll was not done by telephoning a random sample of the population and asking them to respond.
The online population is not representative of the general population. Therefore, we are starting with a non-representative sample, probably weighted toward those who have strong political opinions, both left and right, since political extremists seem more numerous online.
But the sampling problem has only just begun. There are millions of Internet users. How were the respondents chosen? Were they a random sample? No, apparently not. They seem to have been allowed to choose themselves. It says in the Methodology: “Respondents for this survey were selected from among those who have agreed to participate in Harris Interactive surveys.” In other words, the data are based on the views of volunteers.
I suggest it is extremely likely that people who hold strong opinions are more likely to want to talk about them than people who have moderate opinions or are simply not interested.
Worse still, Harris does not even reveal the process by which the final 2,320 people were chosen from among all the respondents. Harris simply says they “were selected.”
To me, this looks like a travesty of polling. It makes a final selection by means that are not revealed, from a group who just happened to respond to this poll, who are not necessarily representive of all the people who participate in Harris polls, who are not representative of Internet users, who are not representative of the general public.
If you conclude that the poll therefore has a very high probability of error–well, no problem! Harris agrees with you! In the Methodology section we find this artfully worded disclaimer: “Because the sample is based on those who agreed to participate in the Harris Interactive panel, no estimates of theoretical sampling error can be calculated.”
I take this to mean that there is no accuracy. This is not, and was never intended to be, a picture of American political opinion.
But it gets even worse, because when the people at Harris ended up with the numbers, they changed them.
The numbers do not reflect the population as a whole, but have been changed as if they do. The Methodology section states: “Figures for age, sex, race/ethnicity, education, region and household income were weighted where necessary to bring them into line with their actual proportions in the population.” So, for instance, if the selection of respondents didn’t happen to include many women, the prevalence of their views was apparently multiplied by a suitable factor–which of course is not revealed.
The press release does sneak in a reference in its third paragraph stating that the poll was of “2,320 adults surveyed online,” but the real meaning of this reference is never spelled out, and is outweighed by the document’s first paragraph, which draws attention to “the large numbers of Americans who hold extreme views of President Obama.” This implies that everyday Americans are the real subject of inquiry. Certainly that is how the polling data have been presented by commentators, and I have not seen any press release from Harris correcting this misapprehension.
Why did Harris do it this way? Well, “internet polling” is cheaper than telephoning people, but is that the only reason to do it? Surely the people at Harris must know that they will harvest more extreme views via this methodology, and since extremism was the focus of this poll (note the term “wingnuts” in its title), the methodology was virtually guaranteed to give the most sensationalistic results that anyone could hope for.
I don’t believe the people at Harris are stupid. I think that they knew what they were doing. Sensationalism sells, and thus the results were sure to be quoted widely. Does this mean the objective was just to get media coverage? Or could it be even worse than this? Did Harris specifically *want* to portray Americans this way, feeding the agenda of commentators who are eager to demonize anti-Obama groups such as the Tea Party?
I don’t want to think that a respected national polling organization has perpetrated such a piece of gross trickery. And indeed I would welcome any rebuttal disproving my conclusion. But given the disclosures in the poll itself, what other conclusion is possible?
By Email
By RSS Feed
I think Harris knew they could count on a lazy media to report the cheap headline before moving on to the latest dead blonde.
The carelessness with which media ill-informs the nation is worthy of our concern. And I don’t know the solution to that unless it’s a more skeptical public.
While I don’t even attempt to know how many Americans (falsely) believe that President Obama is a Muslim just to put it in perspective. I was listening to talk radio this afternoon while in the car and the pundit who was on was talking about how little we know about the President, where he’s from, or what documents we have about him such as a birth certificate, school records, etc. He continued to ask, don’t these documents have religion on them—certainly (at the least) insinuating that the President is a Muslim (as well as being foreign born).
Sadly, I do not find these figures incredibly out there. I’ve had conversations with my own family where they expressed the belief that the President was not Christian—and wouldn’t believe me when I tried to persuade them otherwise.
Regardless of the exactness of the poll data, there is a segment of this country that are willing to believe anything about a person who does not agree with them politically.
Well, Charles, that stinks.
But then, you’re no prize either.
Always interesting (although not very) to find that I have some persistent negative aura to a person I have never heard of. Try to be a bit more creative, though, harold haroldharold. The mere fact that you login using one dorky name not just once but three times suggests a lack of imagination. And the pseudonymous status suggests the usual cowardice.
at this point its pretty clear the tea party is merely ignorance fueled by fear, or vice versa. the signs like grandmas not shovel ready or keep the government out of medicare are proof,
Whereas your comment of course is a model of carefully reasoned argument based on solid observation? Pretty, clear, yes–clear that someone is exhibiting mindless reflexes to something he saw on TV.
So, you’re playing the victim, claiming that liberal media are demonizing anti-Obama groups? That’s rich. I can’t count how many times a day I’m called a faggot-loving, Marxist, anti-American, devil-worshipping, filthy liberal scumbag. And I used to be considered right-of-center.
Besides, the Tea Party is comprised of Rousseau-liberal Jacobins, not disenfranchised conservatives. There is nothing that ties them to paleo-conservatism (the mere fact that I must use a prefix to distinguish it says a lot.)
One of the interesting and depressing aspects of the political polarization that seems to have occurred is that people seem to react instead of reading. All I did was to show, beyond reasonable doubt, I think, that a polling organization may have been less than ethical, and the poll may have been rigged.
And now I’m “playing the victim”? What does that possibly mean, and what does it have to do with what I wrote, which has nothing to do with my political views anyway? And surely the primary issue is the ethics of the organization, not which party or group they favored.
Oh, I didn’t read? So you weren’t asserting that it was “…a premeditated attempt to create a misleading picture that would demonize a segment of the American public?” Or that the liberal media would consider it a “…great gift for liberal commentators who are predisposed to fear and loathe the Tea Party and its supposed influence?” Forgive me for misunderstanding.
I completely agree with your argument that the statistics of the Harris poll are scientifically worthless. But the fact that you were able to ascertain that from their published methodology diffuses any untoward motivations. It seems more a case of caveat emptor than anything. Does it surprise me that an organization would try to add sexy spin to its product? No, but in the case of a polling organization that *might* have negative consequences. That is the deal with their devil.
Besides, the numbers really aren’t the lede here. The fact is that a sizable group of Tea Party members believe Obama is Kenyan and/or Muslim. Leaders of the Tea Party movement and the GOP are milking that for political purposes. That is the real story. There can’t be real debate – skepticism or not – as long as most of the polity is being shamelessly manipulated by political leaders.
I very carefully used conjecture, rather than statements of fact, because I don’t know, for a fact, what the motives at Harris were. But either way, it’s moot. I am mainly concerned with the idea that a polling organization could rig its system to favor any political agenda, whatever the agenda may be.
You write, “The fact is that a sizable group of Tea Party members believe Obama is Kenyan and/or Muslim.” Please could you define “sizable” and provide a reliable source?
And surely the shameless manipulation of public opinion is standard procedure for any political party.
Polling companies can’t spin their data once they have it. The can only bend it going in by less than rigorous methodology which a reading of Charles’ post suggests that is exactly what they did.
THEY can only . . .
They apparently could have quoted better polling sources and gotten similar results. I wouldn’t pay attention to anything with the words “Harris” and “internet” but would this one work for you?
http://www.newsweek.com/2010/08/27/newsweek-poll-democrats-may-not-be-headed-for-midterm-bloodbath.html
57% vs 10% of an online population with (allegedly) better access to information. hmmmmmm…sorry…I’m not buying into the “this is bad data” concept. Such a huge swing is still gonna mean something.
Those numbers are a result of $$$ being spent to give that impression. That’s newsworthy by itself. The fact that 57% of one group believed it and only 10% of the other believed it just shows who they were preaching to.
it’s no different from the “he was born in africa make him prove he was born here” riff during the election.