Iraq Family Health Survey
On January 9, 2008 the World Health Organization reported the results of the "Iraq Family Health Survey" published in the New England Journal of Medicine. The study surveyed 9,345 households across Iraq and was carried out in 2006 and 2007. It estimated 151,000 deaths due to violence from March 2003 through June 2006.
The study was done by the "Iraq Family Health Survey Study Group", a collaborative effort of six organizations: the Federal Ministry of Health, Baghdad; Kurdistan Ministry of Planning, Erbil; Kurdistan Ministry of Health, Erbil; Central Organization for Statistics and Information Technology, Baghdad; World Health Organization Iraq office, Amman, Jordan; World Health Organization, Geneva.
Methods
The Iraq Family Health Survey was a cross-sectional, nationally representative survey of 9345 households that was conducted by relevant federal and regional ministries in Iraq in collaboration with the World Health Organization. The sampling frame that was used in the southern and central provinces was derived from the 1997 Iraq census, which had been updated for the 2004 Iraq Living Conditions Survey. The sampling frame used in Kurdistan was based on information provided by the Statistical Offices in the region. Population estimates for Iraq for the survey period were projected by Iraq's Central Organization for Statistics and Information Technology.Training of central and local supervisors from all 18 governorates was conducted in Amman, Jordan. Training of interviewers was done separately in each governorate for one week during May and June 2006 and a one-day refresher training session was conducted the day before the start of the survey in each governorate. Following interviewer training, the survey instruments and procedures were pilot tested in all governorates. The survey fieldwork was conducted during August and September 2006 in the 14 South/Centre governorates. In Anbar governorate, the fieldwork was conducted in October and November 2006, while fieldwork for the Kurdistan region was conducted during February and March 2007.
Overall, 407 personnel participated in the implementation of the survey, consisting of 100 central, local and field supervisors, 224 interviewers evenly split between males and females, and 83 central editors and data entry personnel.
The survey had an original target sample size of 10,080 households. Interviewers visited 89.4% of 1086 household clusters during the study period and the household response rate was 96.2%. 115 clusters were not visited due to security problems. The IFHS argues that past mortality is likely to be higher in these missed clusters. Ratios derived from comparing their surveyed clusters to corresponding data from Iraq Body Count were used to impute elevated mortality rates to these missed clusters. Using this method, the IFHS derives death rates for the missed clusters in Baghdad that are 4.0 times as high as the clusters visited by the survey in Baghdad. The same procedure in Anbar derived rates that are 1.7 times as high as clusters visited in Anbar.
The IFHS also argues that underreporting is likely to be common in household surveys, particularly due to household dissolution after the death of a household member. On this basis the authors used several demographic assessments such as the growth balance method to derive an adjustment for reporting bias. This adjustment raised the violence-related death rate by roughly 50%, from 1.09 to 1.67.
Results
The number of violent deaths derived from the household survey, plus the adjustments for missing clusters and reporting bias, was estimated to be 151,000 from March 2003 through June 2006. This estimate suggests that violence was a leading cause of death for Iraqi adults and was the main cause of death in men between the ages of 15 and 59 during the reporting period.The results were also compared to the 2006 survey by Burnham et al. reported in the Lancet, and to the Iraq Body Count project. Naeema Al Gaseer, the WHO Representative to Iraq noted, "Our survey estimate is three times higher than the death toll detected through careful screening of media reports by the Iraq Body Count project and about four times lower than a smaller-scale household survey conducted earlier in 2006," referring to the survey by Burnham et al. reported in the Lancet journal in October 2006.
The NEJM article on the survey states that both the IFHS and the IBC "indicate that the 2006 study by Burnham et al. considerably overestimated the number of violent deaths. For example, to reach the 925 violent deaths per day reported by Burnham et al. for June 2005 through June 2006, as many as 87% of violent deaths would have been missed in the IFHS and more than 90% in the Iraq Body Count. This level of underreporting is highly improbable, given the internal and external consistency of the data and the much larger sample size and quality-control measures taken in the implementation of the IFHS." The article also notes that the IFHS and IBC are consistent with each other on both the distribution of violent deaths by province and on the trend in levels of violent deaths over time, while both sources are inconsistent with the results of the Burnham et al. survey on these issues.
The IFHS also collected data on health issues beyond mortality. One such notable finding of the survey was that a worryingly low 57% of the women surveyed said they had heard of AIDS, as compared with 84% of women in Turkey and Egypt, 91% in Morocco and 97% in Jordan.
Reactions and Criticisms
An accompanying commentary appeared in the editorial 'Perspectives' section of the New England Journal of Medicine issue that contained the study. The authors commend the IFHS study group for its attempt to capture the highest-quality results, but also discuss "substantial limitations" of the IFHS and more generally of the use of household surveys to estimate mortality in circumstances such as Iraq, saying:The sampling frame was based on a 2004 count, but the population has been changing rapidly and dramatically because of sectarian violence, the flight of refugees, and overall population migration. Another source of bias in household surveys is underreporting due to the dissolution of some households after a death, so that no one remains to tell the former inhabitants' story. Mortality estimates that are derived from surveying deaths of siblings were also calculated, but this method may also be subject to such underreporting.... Under the current conditions in Iraq, it is difficult to envision a study that would not have substantial limitations. The circumstances that are required to produce high-quality public health statistics contrast starkly with those under which the IFHS study group worked. Indeed, it must be mentioned that one of the authors of the survey was shot and killed on his way to work.
Paul Spiegel, a medical epidemiologist at the United Nations High Commission on Refugees in Geneva, commented, "Overall, this is a very good study," adding that "this does seem more believable to me" than the earlier Lancet survey, which estimated 601,000 deaths from violence over the same period.
Officials in the Iraqi government had differing reactions to the report. The Iraqi Health Minister Saleh al-Hasnawi described the survey as "very sound" and said the survey indicated "a massive death toll since the beginning of the conflict," and "I believe in these numbers". However, a senior official in the Iraq Health Ministry's inspector general's office cast doubt on the findings, saying 151,000 was far too high because the numbers cited by the study were much larger than figures tracked by the ministry.
On the other hand, Jalil Hadi al-Shimmari, who oversees the western Baghdad health department,
said the 151,000 total seems roughly accurate but is probably a "modest"
one and that "the real number might be bigger than this."
Many criticisms leveled at the IFHS estimate have originated from authors of the Lancet survey or others associated with that survey. Some of these criticisms relate to an idea that respondents would have a fear of giving information about violence-related deaths to government interviewers who represented "one side of the conflict" and the reliance on Iraq Body Count's data in dangerous areas.
In a January 11, 2008 article Les Roberts, co-author of the Lancet study, is quoted as saying:
They roughly found a steady rate of violence from 2003 to 2006. Baghdad morgue data, Najaf burial data, Pentagon attack data, and our data all show a dramatic increase over 2005 and 2006.
On February 24, 2009 Morning Edition discussed what a Baghdad central morgue statistics office worker reported to them:
the number of deaths the morgue registers never corresponds with numbers from the Ministry of Health or the Ministry of Interior. "They do it on purpose," he says. "I would go home and look at the news. The ministry would say 10 people got killed all over Iraq, while I had received in that day more than 50 dead bodies just in Baghdad. It's always been like that — they would say one thing but the reality was much worse."
400,000 excess deaths?
A July 2008 article in MedPage Today asserts that the IFHS survey actually estimated 400,000 excess Iraqi deaths as a result of the invasion, with 151,000 being violence-related deaths and 249,000 from non-violent causes. However, the Q&A published on the IFHS by the World Health Organization states that it did not estimate excess deaths:The main NEJM article only briefly discusses its measured increase in non-violent mortality rates from the pre-invasion to post-invasion period:
Overall mortality from nonviolent causes was about 60% higher in the post-invasion period than in the pre-invasion period. Although recall bias may contribute to the increase, since deaths before 2003 were less likely to be reported than more recent deaths, this finding warrants further analysis.
The Q&A elaborates:
John Tirman, who commissioned and directed the funding for the 2nd Lancet study, stated in a January 21, 2008 AlterNet article:
Les Roberts said Friday, January 10, 2008:
John Tirman wrote on in Editor and Publisher:
Timothy R. Gulden, Ph.D., of the University of Maryland School of Public Policy in College Park, asserted that the Iraq Family Health Survey authors "acknowledge and attempt to correct for underreporting of deaths from nonviolent causes, but they make no allowance for the more serious underreporting of violence-related deaths to government-affiliated survey takers."
Earlier Iraqi Health Minister estimate in November 2006
When the Iraq Family Health Survey was published in January 2008 the Iraqi health minister was Dr Salih al-Hasnawi.The previous Iraqi health minister, Ali al-Shemari, in early November 2006 estimated between 100,000 and 150,000 people had been killed since the March 2003 U.S.-led invasion. The Taipei Times reported on his methodology: "Al-Shemari said on Thursday that he based his figure on an estimate of 100 bodies per day brought to morgues and hospitals -- though such a calculation would come out closer to 130,000 in total."
The Washington Post reported: "As al-Shemari issued the startling new estimate, the head of the Baghdad central morgue said Thursday he was receiving as many as 60 violent death victims each day at his facility alone. Dr. Abdul-Razzaq al-Obaidi said those deaths did not include victims of violence whose bodies were taken to the city's many hospital morgues or those who were removed from attack scenes by relatives and quickly buried according to Muslim custom."
From a November 9, 2006 International Herald Tribune article:
"Each day we lost 100 persons, that means per month 3,000, per year it's 36,000, plus or minus 10 percent," al-Shemari said. "So by three years, 120,000, half year 20,000, that means 140,000, plus or minus 10 percent," he said, explaining how he came to the figures. "This includes all Iraqis killed — police, ordinary people, children," he said, adding that people who were kidnapped and later found dead were also included in his estimate. He said the figures were compiled by counting bodies brought to "forensic institutes" or hospitals.
From the November 11, 2006 Taipei Times article:
An official with the ministry also confirmed the figure yesterday , but later said that the estimated deaths ranged between 100,000 and 150,000. "The minister was misquoted. He said between 100,000-150,000 people were killed in three-and-a-half years," the official said.