Robert Shiller had a piece in the Sunday NYT noting that the S&P 500 was unusually high relative to his measure of trailing earnings. He calculated a ratio above 25, far above the historic average of 15. Shiller said that in the past, each time this ratio crossed 25 the market took a plunge shortly thereafter. He concludes his piece by seeing it as a mystery that the market remains as high as it does.
Brad DeLong picks up on Shiller's analysis and points out that in most cases in the past where Shiller's ratio had exceeded 25, people who held onto their stock over the next decade would still have seen a positive real return. He notes the examples in the 1960s when investors would have seen a negative tenyear return, even though Shiller's ratio was below the critical 25 level. He therefore concludes there is no issue.
I would argue there is an issue, although not quite as much as Shiller suggests. To get at the problem, we have to recognize that stock returns, at least over a long period, are not just random numbers. Both Shiller and DeLong treat this as a question of guessing whether an egg will turn into a lizard or chicken based on the distribution of past hatchings that we have witnessed.
That would be a reasonable strategy if that is the only information we have. But if we saw that one of the eggs was laid by a hen, then we may want to up our probability estimate that it will hatch into a chicken.
In the case of stock returns we can generate projections based on projections of GDP growth, profit growth, and future price to earnings ratios. For example, we may note that the ratio of stock prices to aftertax corporate profits for the economy as a whole was 22.3 at the end of 2013. (This takes the value of stock from the Financial Accounts of the United States, Table L.213, lines 2 plus 4 for market valuation. Aftertax corporate profits are from the National Income and Product Accounts, Table 1.10, Line 17).
This means that earnings are roughly 4.5 percent of the share price. If companies pay out 70 percent of their earnings as dividends or share buybacks (roughly the average), this translates into a 3.1 percent real return in the current year.
The next question is where we would expect share prices to be in ten years. While we don't know exactly what the economy will do over the next decade, there are few analysts who would project an average growth rate of much below 2.0 percent nor above 2.6 percent. This means that if the profit share of GDP were to remain constant and the price to earnings ratio were to remain constant then we can expect real returns to be between 5.1 percent and 5.7 percent.
Of course neither the profit share nor the price to earnings ratio need remain constant. Suppose the profit share drops from its current 7.2 percent share of GDP to its postwar average of 5.1 percent. In that case, total profits ten years from now would be 13.7 percent lower in the 2.0 percent GDP growth scenario and 8.4 percent lower in the 2.6 percent growth scenario. If the price to earnings ratio stayed constant, the slow growth scenario would imply a real decline in prices of 1.5 percent annually, leading to a total real return of 1.6 percent (3.1 percent minus 1.5 percent). The more rapid growth scenario implies a decline in real prices of 0.9 percent annually for a total real return of 2.2 percent (3.2 percent minus 0.9 percent).
It is also possible that the price to earnings ratio will fall back toward its historic average of 15 to 1. This makes the picture worse, but not by quite as much as it might first appear. In the slow growth case we would see a decline in real share prices that averaged 5.3 percent annually. In the latter case the decline in real share prices would average 4.7 percent annually. However it is important to keep in mind that a drop in the price to earnings ratio will lead to a rise in the dividend yield. If our endpoint is a price to earnings ratio of 15, it implies that in year ten the dividend yield will be 4.7 percent. To simplify matters, if we say the average yield over the ten year period is 3.9 percent (the average of the endpoints) then the average real return is minus 1.4 percent in the slow growth scenario and minus 0.8 percent in the fast growth scenario.
To decide whether this would be a disaster it is necessary to consider the alternative investment opportunities. At the moment, shortterm money pays a real interest rate of around 1.5 percent. Thirtyyear Treasury bonds pay a real interest rate of around 1.0 percent. At this point, Treasury bonds also carry a substantial risk of a capital loss if interest rates rise, as is generally predicted, so this is not a risk free return.(The late 1990s gave a very different picture since the real return on longterm Treasury bonds was over 3.0 percent at the time.)
Taking these comparisons into account, stocks still look like a pretty good deal since even if there is some decline in the profit share of income and also some reversion toward longterm trends in price to earnings ratios (my bet is that the ratio stays above 20), then the real returns are still likely to be well above 3.0 percent. In short, I can't see the basis for Shiller's big fears, on the other hand, the high price to earnings ratios in the stock market means returns will almost certainly be lower in the next decade or so than their longterm average.
(There actually is a very good paper on this topic, see Baker, Delong, and Krugman, 2005.)
(Only one link allowed per comment)

Brad has been contemplating Bayesian probabilities lately, and I think the place where he went off track is to assume that all events are random. Probabilities are only valid if the thing being measured is random. I think a lot of people get really confused by this, and here is what I think is behind it:
Probabilities are only valid for betting on the next occurrence if the event is random in nature. So what is the event? Is it the move of the market? Or is it your bet?
Probabilities come from nature through quantum mechanics. Probabilities work here. When we start looking at the probabilities of betting on humancreated events, if probabilities work at all it is only because our bests are random, not the underlying events. The underlying events that drive a market are not remotely random.
If you can have success betting on investments using probabilities, it is because you are trowing darts. Your bets are random, therefore probabilities work. I would not invest with somebody that throws darts.
Probabilities are way over used by most people. I've come to realize that the reason is that people WANT probabilities to work for everything even when they aren't appropriate for the situation.