To me, that sounds as if your configuration were perfectly okay... well, not perfect maybe, but it should pretty well do the job.
Since FPS only drop when looking at the guards, it seems to be a problem related to entities somewhat. Could you probably post the different rendering stats as shown on the debug panel when looking at the guard models?
I'd especially be interested in the "ref" value...