Make it a habit to always test out of sample, the fastest way is horizontal splitting. In sample tests give too optimistic results, even when you only manually optimized the parameters.

For checking the consistency you can test and train also with different split combinations, f.i. SKIP1 for training and SKIP2+SKIP3 for testing.