Its no fault to share your thoughts and feelings here, other already did so too.

I have a son who is already 15 years old, I left his mother, when he was 2.
There were no others involved for a long time she had been single for some years and me too. I simply couldn't stand to stay with her and share the everyday life.
But, we absolutely agreed that our son shouldn't suffer from this situation, he met me regularly, stayed one or more days a week at my home. We stayed in the same town, it was no problem to organize the things that related to him.
It isn't always a matter of law, sometimes it is simply a matter of the will to give the children the best, but I know that it isn't always as easy as I experienced it, and I know of other couples where it got a mess of conflicts and misbehavior and an emotionally worse time for them and the children.

I wish you and your children the best, don't fight for them as a property, think about what is the best for them, and that is mainly not to be with the farther or with the mother, but having not to worry about their parents, because they involve them into their fights, because of their hurt dignity, because that's what tears their souls apart, not the change of their home and the change of the daily agreements, but getting involved in a fight that they can only loose.

I wish you strength and wisdom, friends who can help, and that your wife is willing and able to make agreements that are good for the children.