Hi jcl,

yes, I am coming round to it. I think my unease has been partly with some of the systems that I come across, rather than the procedure itself; these do not provide any "model" for the market, but put up degrees of freedom which end up somehow working with a backtest set. The absence of clear rationale makes this appear rather fickle and contingent (so I need a fast MA period of 9 for 2006 but 17 for 2007, yeah, that will do...). Obvious "quantitative" algorithms like cointegrated pairs trading or Markov switching time series models do attempt a direct representation of what is happening, which somehow makes me trust them more. This may be wrong in the first place as some of the classical models postulated in finance are really pretty rubbish (Markowitz: we consider normal returns and investor utilities only, we neglect estimation and model risk... -- Black-Scholes: normal increments, no transaction costs, static vols in time and across strikes, complete markets meaning the options being priced are effectively redundant) and a strong intuition that has held true in the past (as a "stylised fact") is likely worth at least as much as a starting point, objectively speaking.

In fact, thinking about Markowitz in particular, I don't really see in what ways the standard process is any more rigorous that that described in the various books we have mentioned. I do think that the boundary between working on a system / "optimising it" and "throwing a lot of mud at a wall in the hope that some of it will stick" is fluid, there will be no magic statistic telling us when we cross it, and I suspect that always remaining aware of this is the main skill to develop. I am writing this after rereading the section on Luxor MAE/MFE in J+T which I think is absolutely superb, but which could easy turn into one such overfitting exercise if tackled from the wrong angle or in the wrong way.

Anyway, I am sort of rambling now so I'll stop; thanks for the insights so far.