Even if you are a die-hard believer in the efficient market hypothesis, that doesn’t mean you have to invest only in index funds. If the efficient market hypothesis is correct, then you won’t do any worse (on average) with a random collection of stocks within an index than you will by holding the index. If stock picking doesn’t matter, then you are free to pick any collection of stocks within the index that vaguely represents the index.
Some index funds perform poorly against the index not because they have high fees, but because they are trying to track the index too closely. When a stock is added to S&P 500, millions of dollars invested in S&P 500 Index funds must all buy that stock in order to track the index exactly. Stocks added to the S&P 500 do very poorly the year after this surge of automated buying. Funds that delay purchasing these stocks can perform better than those who purchase them immediately and pay a premium. Similarly, stocks that are being removed from the S&P 500 will out perform the index over the next year because all the index funds dumping the stock drive the price down needlessly. Delaying the sale of this stock until it has had a chance to recover produces superior returns.
Source: How to Blend Index Funds
Here's my situation. I suspect that buying individual stocks yourself can be cheaper than investing in index mutual funds or ETFs because holding individual stocks doesn't eat up any management fees. You have to pay brokerage fees to buy the stocks, but you only pay once rather than annually, so that over the long run (30 or 40 years) your costs are virtually zero.
There are exception to the rule here. Many foreign, high-cost investments should be accessed using ETFs, such as emerging markets or frontier markets. However, if your discount online brokerage offers $20 trades to buy stock in domestic companies, I think that replicating the index yourself can save you money.
One problem is trying to constantly sell and buy once companies go off and on the index. This problem comes about because indexers set an arbitrary line between big and small companies. Many choose the S&P500. But why the top 500 companies? William Bernstein in The 15-Stock Diversification Myth claims the following: "Fifteen stocks is not enough. Thirty is not enough. Even 200 is not enough. The only way to truly minimize the risks of stock ownership is by owning the whole market." The whole market means the whole market, not the S&P500. It means everything, from small caps to large caps.
What I suggest then is using ETFs to cover the high-cost areas and for the low-cost areas, randomly sample from a population of all stocks using market cap as weight. Over time, as you buy, your cumulative sample will converge towards the market.
DIY (do-it-yourself) indexing will give you identical expected returns to an index mutual fund but because DIY indexing carries significantly lower costs, it follows that DIY indexing will beat index mutual fund investing over the long run in the same way that index mutual fund investing will beat active mutual fund investing in the long run (mainly because of fees).