In my design process, I've enabled NumSampleCycles at one stage but I noticed it can have a dramatic effect depending if I enable it during Training or Testing or both. What does this mean?

For example:

--> WFO rolling Train, no oversampling
--> Test, no oversampling
--> good result

--> WFO rolling Train with oversampling on
--> Test with oversampling on
--> bad result

--> WFO rolling Train with no oversampling
--> Test with oversampling on
--> good result

It seems that oversampling is "bad" to use during Training, but what does that mean?

Since a WFO rolling train includes several OOS cycles for optimizing, why would oversampling cause it to produce such poor optimal parameters?

I just want to understand better.
Thanks

It could mean that either the system is very noise sensitive, or maybe it depends on some seasonal effect.

Dusktrader, how do you switch oversampling for training? I am only able to test with it.

If I set NumSampleCycles=## while Training (same as you do for Test) then it seems to also run through the cycles during training.