Yes that would be possible.
Shematic network example:
[client1]----[router1]---------------------------[router3]----------[server]
Port 2300 Port 8947 | Port 2300 Port 2300
|
[client2]----[router2]----------------
Port 2300 Port 38495
The problem is, that the routers change the port number of client 1 and 2 because of NAT/PAT as you can see in the example above. Thus if the server wants to send a packet to client1 and uses port 2300 it won't work.
Solution (NAT punsh-through): The server is reachable through port 2300 because port forwarding was done here. Now all clients contact the server by sending a packet. Thus the server can store the destination ports of the packets (which are 8947 and 38495). If the server then sends a packet to these ports, the clients will receive the packet.
And that's it. It's very simple.
