Monday, June 15, 2009
Not all forecasters are alike
"No one realized how bad the economy was. The projections, in fact, turned out to be worse. But we took the mainstream model as to what we thought -- and everyone else thought -- the unemployment rate would be."
"Everyone guessed wrong at the time the estimate was made about what the state of the economy was at the moment this was passed."Two things about that: First, we were forecasting in a period where there was little good past experience to work form. Almost all forecasting involves taking a set of data that you think represents the state of the economy, looking at the most recent experience and finding "comparable" periods of history where the same conditions essentially occurred. Now of course the data is never identical -- we don't live in test tubes. I refer to macroeconomic data as "in the wild", impure, contaminated by hundreds of other influences that we never measure. You assume the stuff you're not measuring isn't important. Good emphasis on this comes from Ed Leamer, whose book "Macroeconomic Patterns and Stories" includes this paragraph:
You may want to substitute the more familiar scientific words �theory and evidence� for �patterns and stories.� Do not do that. With the phrase �theory and evidence� come hidden stow-away after-the-fact myths about how we learn and how much we can learn. The words �theory and evidence� suggest an incessant march toward a level of scientific certitude that cannot be attained in the study of the com plex self-organizing human system that we call the economy. The words �patterns and stories� much more accurately convey our level of knowledge, now, and in the future as well. It is literature, not science.This was the essence of what I was trying to say Thursday. We try to express that lack of certitude by providing a confidence interval, which when done properly makes most of the so-called "created or saved" jobs into something that could well be just noise. �But even confidence intervals give the forecast a patina of scientism that has no business in economic forecasting. �I'm not confident of confidence intervals (in short, here's a one sentence explanation: I have to have some degree of knowledge of the shape of the distribution of the errors I make in forecasting ... and I question whether or not I can possibly know that.) �When Biden also said�""The bottom line is that jobs are being created that would not have been there before" or that it "clearly has had an impact", we have to respond that we don't know that.
Labels: Biden, economics, forecasting, Obama