Monday, May 18, 2009

Error bands for accountability 

A number of people have been commenting on the new graph that's shows how the outturn of the unemployment rate compares to what the Obama Administration forecasted back when they were pushing for passage of the stimulus bill. (For a different way to tell the same story, look at Jim Hamilton's post this weekend on industrial production.) I like how Greg Mankiw tells this story best:
What does this mean? One interpretation is that the fiscal stimulus has failed to achieve what Team Obama thought it would. Another interpretation is that the baseline was worse than they believed at the time. I am confident the report authors would adopt the second interpretation. If so, that fact is consistent with what I said in a previous post: In light of the shifting baseline, it is impossible to hold the administration accountable for whether its policies are achieving their intended effects.

To be clear, this lack of accountability is not a feature on this specific administration but is, instead, a reflection of the inherent uncertainties associated with macroeconomics. The administration, however, has not been particularly forthright in admitting to this lack of accountability. Indeed, the act of releasing quarterly reports on how many jobs have been "created or saved" gives the illusion of accountability without the reality.
As somebody famously said, it's difficult to make predictions, particularly about the future. Anyone who has done economic forecasting would tell you how many pitfalls there are in providing precision from which to make a "created or saved" statement. Ed Leamer in his EconTalk interview with Russ Roberts made the case forcefully that the most important part of the forecast isn't the point estimate but the error band around the forecast. And it's even more difficult to define that 95% confidence interval when the size of the stimulus dwarfs by 3 or 5 sigmas any stimulus within the sample for which you estimated your multiplier of 1.56 or whatever. The Administration's estimates said only that there was "substantial" variability about the forecast, and yet it continues to use a $93,333 of government spending = one job calculation as if jobs were something you could buy off a shelf.

(And speaking of shelves, next time I teach graduate forecasting, I will take off the shelf and the class will read Leamer's new book. I'm only up to page 120 or so, but it's tremendous.)