The Economics of Insider Trading: A Puzzle

The standard story on insider trading is straightforward. I have knowledge that implies that a stock is going to go up, so I buy it. The seller sells because, absent that knowledge, my offer is more than he, and presumably most of the rest of the market, thinks the stock is worth.There is a problem for this story if we assume that all the players are rational. If there are a significant number of insider traders in the market, the fact that I offered more for the stock than the owner thought it was worth should be evidence to him that I know something about it he doesn't, and so a reason not to sell.To put the point differently, if the insider traders are making a higher return on their invested capital than the market average, which is the point of insider trading, everyone else must be making a lower return than the market average. But anyone who wants can get the market average by buying the whole market, or an index fund, or a random selection of stocks. All he has to do to protect himself against insider traders is to avoid the strategy of estimating stock values for himself and selling when the price he is offered is above his estimate, buying when below.One possible explanation is that some traders are insiders and some think they are, believe they have better than average knowledge or judgement but are wrong. But I am assuming, as economists often do, that all the players in the game are rational.This puzzle first struck me when I thought I had an ingenious explanation of why, on average, stocks outperform bonds, as it is commonly asserted that they do. If buying stock and holding for ten years is practically guaranteed to give you a better return than buying bonds and holding for ten years, one would think that everyone investing for the moderately long term would buy stock and their doing so would drive the price of bonds down and their yield up until the two investments were equally attractive. One could avoid that by assuming that there was enough risk averse money looking for short term investments to drive bond prices up and the yield down to a point where the yield was below the return on stocks. That conjecture could lead to interesting empirical work testing it. But I am by nature a theorist not an empiricist, so I was looking for a different answer and thought I might have found it.Imagine a market where everybody is an inside trader on a small scale. I know more than the market about the firm I work for. You know more than the market about a firm you have repeatedly had dealings with. He knows more than the market about a technology he has worked with extensively that will affect the performance of several firms.Each person invests some of his money where he has an edge and, by the standard analysis of insider trading, makes an above market return. But because he is risk averse he wants to diversify his investment, so part of his money goes into other investments. His marginal return is the return he can get without inside knowledge, his average return is a suitably weighted average of that and his insider return. Bond investments only have to do as well as his marginal investments, so the return on bonds is lower than the average return on stocks.I thought it was a lovely theory–until it was pointed out to me that an investor who wanted to get the average return on stocks could do so with no inside information by simply buying the whole market, or an index fund, or ... . And, if the average return on stocks was higher than on bonds, he would.Which brings me back to my puzzle. Does anyone have a good solution? I have wondered if it has some deep connection with Robin Hanson's old puzzle of why each of us prefers his own opinion to that of others even if he has no reason to think he is better informed on the subject than they are.