Move all of your sticks for a little bit and the spinning should stop.

I used to have this problem with a joystick that had a throttle on it. If I didn't flick the throttle around after I started my game, my ship would hit full speed instantly.