Beware the unknown unknowns

There are known knowns. There are things we know that we know. There are known unknowns. That is to say, there are things that we now know we don’t know. But there are also unknown unknowns. There are things we do not know we don’t know.

Donald Rumsfeld, 2/12/2002


A successful investor needs to not only be intelligent and insightful, but also pragmatic. Know what you don't know. But one also needs to allow for events and outcomes that simply can't be anticipated-- the "unknown unknowns."

I thought about this when I read an article in today's Chicago Tribune . It was a "year ahead" piece on the economy and financial markets, with opinions from various experts. Professor Erik Hurst, of the University of Chicago business school, on housing prices:

Housing prices will fall another 15percent to 20 percent in the next couple of years, he predicts. "We've got a long way to go."

His study of market data stretching back decades gives him 100 percent confidence in his prediction, he said. "A big increase in price movements is followed by big declines. Take it to the bank."


Hmmmm.... "100% confidence"? based on his study "stretching back decades"?

I can only hope that he was severely misquoted. If not, I think he's an idiot. Let's think about this. First, what's his data set? "Stretching back decades" could mean as little as 20 or 30 years. But let's assume that he has gone back 190 years (otherwise he would have said centuries, no?). I know that Schiller and others have looked at housing prices back to the 1800s, but how good is that data? Do you really think that they have good records for a broad range of housing transactions dating back that far? Sure, there may be history for some houses on Beacon Hill dating back to the 1600s. But what about the sod huts in Nebraska in the 1870s?
I'd be willing to bet that many home sales went unrecorded or otherwise lack good records at least as recently as the 1930s. And many of the meticulous details that we have today, dutifully reported on CNBC like sports scores, date back to the 1970s or '80s.

But even if we have good data back to the early 1800s, Prof. Hearst is telling us about a study of cycles: a big increase is followed by big declines. How many times has that happened? I can assure you that one of the most basic assumptions about home prices prior to 2007 was that they never go down. This is what got us into the present mess. On a national basis they had virtually never declined. So how many cycles could have been captured in his research? I'll assume that he has looked at regional cycles and other data. Maybe he has captured 20 such events? I'd think that's a generous assumption.

One of the my earliest and best lessons in investing involved my only commodity trade. We were just out of college, and a good friend had taken a job as a commodities broker. He found an investment opportunity in lumber futures: it seemed that there was a seasonal pattern that had worked in 16 of the past 17 years. He put together a group of our friends to open an account and take advantage of this pattern. I think that I invested $2000.

Guess what? By the time that our contract had expired, that lumber trade's record was 16 out of 18. I lost it all. But for $2000, I learned a very valuable lesson about correlation.

One of my favorite quotes from Fooled By Randomness author Nassim Taleb:

My classical metaphor: A Turkey is fed for a 1000 days—every days confirms to its statistical department that the human race cares about its welfare "with increased statistical significance". On the 1001st day, the turkey has a surprise.

Perhaps housing prices will fall 15 to 20 percent in the next couple of years. Or maybe they'll fall 50 percent, or maybe they'll go up. Far too many unknown unknowns here. Wanna bet that if we were to interview the Professor in a few years, and it turns out that his 100% confidence prediction was wrong, he'd cite some unforseen events to explain his error? Unprecedented government intervention, currency crisis, runaway inflation, change in tax laws, etc. etc. The point is that we can rarely if ever be certain about anything. The successful investor attempts to know what he doesn't know, and makes allowances for the unknown unknowns.