I think your idea could work if it's done right and if those very unlikely situations of points changing direction just when they're crossing, are not messing things up.
That combined with a color recognition should get results that are reliable enough

Since in my case I'm only using points that will never cross their path, I could fix my problem by always taking the right point as point one, instead of the bright one.
This works perfectly, I can know get the distance between the light sources and the camera. So if the observer has two light sources left and right of the head, then I know if he's moving closer or further away.

Now the next problem is that if he's turning the head, then the points are moving closer together and it looks like if he's moving closer.
I have to find a solution for that yet...