I just found out that I used wrong texcoords for the transparency check.
The shadows shape is much better now, but you also can see the pixels better...