Some devices need a reboot of windows to be fully 'off' and out of the system (it's a Windows thing), you can't unplug everything just by pulling out the plug. Perhaps restart your computer without the joystick connected and try again? Also, in case you have a wireless gamepad, disconnect that too, or any device that controls input. I once had my Wacom Tablet causing trouble in a game, eventhough at first sight i thought it could have nothing to do with it.
Your optical mouse should be fine, because I use one all the time,
