i think the z-buffer must be larger than your actual screen resolution. i'm not quite sure about the numbers behind d3d_antialias, but for 2xfsaa you'd probably need twice the size for the backbuffer (either 1.414... or 2, i'm not sure).