I got it to work, and I don't really know how
The Isometric flag suddenly started working and now I can use the sprites' Z-coordinates to layer them.
No more floatiness and no was d3d_entsort needed

If I figure out what I did differently, I will add a post!