here's the theory in 2d.
using counter-clockwise (ccw) angles, your condition would be something like:
angle(vx) - angle(v2) < angle(v1) - angle(v2) => ccw order: v2, vx, v1
to calculate the angle of each vector* just use atan2(dy, dx) with dx/dy = the x & y components of each vector
(* the angle between the positive x axis and the vector, like in mathematics)keep in mind that special cases can cause the area between v1 and v2 to exceed the [0° - 360°] range. to compensate for that, a simple modulo-operation should do the trick:
ang_diff_2x = (angle(vx) - angle(v2)) % 360
ang_diff_21 = (angle(v1) - angle(v2)) % 360
if(ang_diff_2x < ang_diff_21)
...
