Just build an own wrs with all the textures, and build another wrs with very small dummy textures, that you put in the server's folders.
But the server and client are using the same folders! It's the same binary, as I said above.
If we use different folders for the two, then we need to reship all .wrs files again a second time, which will almost double the file size.
Whats about removing the skin after loading. this should solve in someway the memory issue, but would slowdown additionally the loading process.
this would happen every round-restart and would slow down the loading process even more instead of speeding it up, I believe.