Do you use media_play()?
As far as I know, you can't preload sounds for media_play().

You could use snd_play() as an alternative, though. It plays wav and ogg files. There shouldn't be any delay when using this function.
You can also create your sound objects to be played back by snd_play() during runtime (snd_create) and destroy them, when you don't need them anymore (ptr_remove).