The Elephant in the Room

The Burnham, Lafta, Doocy, and Roberts report of Iraqi mortality, recently published in the Lancet, has stirred up a hornet’s nest of controversy, not just among the various political camps, but among survey statisticians as well. I was a bit dubious of their point estimate (654 965), but I didn’t think too much about it in light of the large confidence interval (392 979, 942 636) and knowing how point estimates can vary widely based on sampling plans and estimation models. You can get up to speed on the whole story at Wikipedia.

Now there are two statistical areas of concern about the quality of the estimates. One addresses sample size, and is laid out convincingly in this opinion piece in the 18 October Wall Street Journal. Nice to see some good examples for cluster sampling; I can use them in my lectures.

The second area is response rate; this is discussed, with confounding, outrage, and name-calling over at the Social Science Statistics Blog, operated by Harvard University’s Institute for Quantitative Social Science. So far the combatants haven’t gotten to the really useful research question: how do survey researchers get such high response rates overseas, while pollsters and market researchers get dismal response rates here in the US? My intuition prompts the glib explanation of survey fatigue–folks overseas, especially in the developing world, are happy that someone (anyone) thinks they are important enough to survey, while people in the US find almost any question to be annoying and intrusive. A less charitable explanation–the one that started the SSSBlog fight–is the possibility of curbstoning. Not too likely, if the results at the Demographic and Health Surveys site are believable (and I can’t imagine a Vast Survey Statistician Response Inflation Conspiracy). If I were studying someplace like the University of Nebraska, I’d see my faculty advisor about making this into a dissertation topic.

Update (18 October) The post on the Social Science Statistics Blog referenced above has been removed. As has the post announcing that it has been removed. But there’s an explanation, sort of. You can still sneak up on it indirectly through Kieran Healy’s Weblog.

Update (19 October)  More outrage and name calling over at Deltoid.  Tip from Winds of Change.