Myabe it's return(target.x); instead of return(target);

We need to copy the returned vector to earth.x, that is what vec_set does. You cant just copy a vector to anyother by using earth.x=mouseTo3D(), because that's for single values, not for arrays (which is what a vector is, var vector[3]).