Is it possible to create a dedicated server that does not do any drawing? It seems that even with the -sv flag without the -cl flag, there is still some drawing, and that slows down the CPU.

Any ideas?

If you set the -cl flag on a server, there will still be a check to see if sounds and graphics are active. However if they are not used, this will not take away from system performance as the check will go right over these steps.