Definitely use XNA. You can easily incorporate any Windows forms...
Yes. I agree. XNA is one solution here. But you can also choose pure OpenGL. There are a lot of tutorials describing how to put it into a MS Windows environment. There are even platform independent window / gui libraries for OpenGL.