Nah, just think it through.
I'll try to explain though, its not easy.
[sticking to 'square' scaling for simplicity]

If scales are at 1.0, all pixels are 1x1 in size.
If scales are at 2.0, all pixels are 2x2 in size.
If scales are at 3.0, all pixels are 3x3 in size.
etc...

BUT!!!
If scales are at 1.5, SOME pixels are 1x1, and some are 2X2,
because you cant make a pixel 1.5x1.5

SO.
If you lock your scales to ONLY INTEGERS, the problem will go away.
But the higher your scale gets, the 'chunkier' each step will become...

Get what I mean?