did you try to set a constant lerp factor?