In this post, I used IMF estimated data for predicting the counterfactual mortality levels, necessary to calculate excess deaths. It is important to note that by not taking into account the recent population decline, one is probably biasing down estimates of excess deaths.
For instance, here are three estimates:
Figure 1: Estimated mid-year population for Puerto Rico, in millions, from IMF World Economic Outlook (blue), World Bank World Development Indicators (green), and CIA Handbook (red square), all plotted on a log scale. Bold dashed orange line at 2016, when Harvard School of Public Health team based counterfactual. Thin dashed orange line at 2010, start period (2010-2016) for Santos-Lozada and Howard used for counterfactual.
If the population is actually smaller in September 2017 onward than what it was in mid-2016, then the “normal” level of mortality is overstated. Excess deaths = recorded minus expected will then be biased down. If we use 2010-2016, the bias will be further exacerbated. That is why both Kishore et al. (2018) and Santos-Lozada and Howard (2018) indicated their estimates were conservative.
A more general picture of estimates here, from the DRAFT Economic and Disaster Recovery Plan for Puerto Rico:
So, when you read:
Thus, the year-end excess death toll of 1,400 may be treated as a firm number in practice. It is nowhere near the 4,600 central estimate of the MPR study and relayed by virtually the entire mass market media in the US. The Harvard study was wrong, and by a wide margin. Therefore, as we wrote earlier, unless the Study authors can point to where 3,000 bodies undiscovered through March 2018 may literally be buried, we are left to conclude that they simply do not exist, and the Study must be judged as wildly inaccurate and a gross exaggeration of the true impact of Hurricane Maria in Puerto Rico.
Understand, in addition to the final administrative data likely not being “firm”, the counterfactual is constructed. Assumptions made implicitly can have a big impact on the inferences, even when those assumptions seem innocuous.
My estimates indicate 740-2705, depending on how to treat statistical significance, assumptions regarding population size, nature of error terms in regressions, and not taking into account specification error, measurement error in population statistics, and demographic attributes of the population.
Update, 5:30PM Pacific: Reader TDM suggests that demographic effects are important in calculating the counterfactual. I estimate:
(3) mt = β0 + β1popt + δ(OCT14) + monthly dummies + θ(pop65) + ut
Where pop65 is the (log) estimated population over 65 according to the World Bank and pop is sourced from the World Bank as well (instead of IMF). The estimate of θ is 0.150, t-stat of 0.137. This suggests, but is not completely dispositive, of the idea that the results are highly biased due to the omission of demographic factors.
Update, 8/14 10:30AM Pacific:
Notice that estimated excess mortality is barely changed for the 2010-16 sample.
Figure 2: Estimated excess fatalities using OLS log-log specification including IMF WEO population (blue), and using OLS log-log specification including World Bank population and over-60 population (red). Source: Author’s calculations.
Page 5 of Kishore et al. (2018) also casts doubt on the demographics hypothesis.
Update, 8/18/2018, 5:30PM Pacific: Data here [XLSX].