Well you got it almost right.
FF shaders do use your graphics card, but they dont use the shader hardware. Almost all GPUs, new and most old ones, have the same basic fundamentals, and an FF shader uses these instead of the vertex/pixel shader hardware.
However the performance *may* drop when you use FF shaders, because its not using the hardware boosted shader modules. Vertex/Pixel shader hardware takes some strain off the GPU and will run faster.