How often do you send it? You could wait on the server til the pan changes significant and then send the pan to the clients. Then you wait til it changes again significant. About 5 or 10 degree, you have to try different values. But make sure to do smooth rotations on the clients, or they will notice.
