I dont really see a problem here o.0
Just update the playerstructs on all clients after loading them on the server.
Im using ANet btw so I cant really help you with the built in thingy.