Stencil shadows follow the polygons of your model, thats normal. For better looking, first switch off the ugly self shadows, for this set the CAST flag of all models. Then take care of some better shading in your level, for this increase the sun brightness and decrease the ambient brightness, this will give you smooth gouraud shading. You can also play with material values of mat_model.