I have a windows style menu in the following code, but I need to make a pop up window similar to the "Options->Preferences" menu in SED.
I think it has something to do with CreatePopupMenu() but I can't seem to figure it out.

'CreatePopupMenu' is used for a context menu (shortcut menu) like the one that apperas when you right-click and you get 'copy','cut','paste' etc.

What you're describing is a 'modal' window. You just create a window the standard win32 way, with WNDCLASSEX/CreateWindow()/RegisterWindow(), but to make sure that the window is modal you have to do a little bit more tweaking.

You can either temporary disable the parent window with eg, EnableWindow(), or block messages in the message loop, while the modal window is open.

There may be a specialised function for modal windows, something like Dialog() or DialogBox().

I haven't worked with modal windows in native win32 myself, so this is just my theory, but it should work. Google 'modeless vs modal' and similar terms to find some standard C++.

I tried looking at a few samples and I can't seem to get it to work with 3DGS. I'm pretty green with the windows api stuff.

Ok, i'll have a go later on, I'm messing about quite a bit with win32 and 3DGS.
I think building a Dialog template and then using DialogBox() is the way...