Mother's Den was made about 3 years ago and at the time not many people had widescreen resolutions. I had to force 1280x1024 when you played Mother's Den because Intense X at the time had a few problems with re-scaling panels to work for other resolutions (it works fine now). So, if your monitor+gfx card supports 1280x1024 then the installer should let you go through..

I should really update that Mother's Den demo at some point.. But anyway, if you can't play the demo here's a video showing many scenes from it: