you have to use mouse_mode = 0, or mouse_mode = 1; if mouse_mode = 2 mouse_force is not changed by mouse movement.

the function for rotating the cam for example:

if(mouse_middle == 1)
{
camera.pan -= mouse_force.x * 5 * time_step; //turn left and right
camera.tilt += mouse_force.y * 5 * time_step; // turn up and down
}