I´ve similar thoughts, but I want to use Fmod for this task.
HeelX has done a nice wrapper for it.

To do a real synthesizer, you´ll need at least:

-A controllable oscillator - this can also be a sampler for arbitrary Waveforms (VCO)

-A controllable filter, usually a low pass filter (VCF)

-A controllable amplifier (VCA)

-At least one envelope generator (AD / ADSR)

This sounds more complicated as it should be.
I think the greatest challenge would be to implement full MIDI support.

Why choose Lite-C?
Well...
...
To prove an alibi, we could do some fancy visual effects and user interfaces with the engine.

Edit:
http://www.synthedit.com/