this is just off the top of my head, but...

i think you need a third vertex position for roll. using two vertex positions gives you a line, and something oriented along that line can have any roll. a third vertex position gives you a triangle -- there are no unknowns if you do it right.

don't change the way you're finding pan and tilt at the moment, coz that appears to be working just fine. sorry i can't go into specifics, coz i just can't think of what you'd actually do with the third vertex position, and i have homework due tomorrow that i have to do in a couple of minutes.

good luck

julz