Keras predict_proba() call time

I have been playing around with the deep learning demo script, DeepLearn.c. I started with deepnet and then moved to keras. First tried CPU then GPU training, now back to CPU. However consistently the predict_proba() function in keras is very slow. Training is quick and testing takes forever. I timed the predict_proba calls in R with tictoc. I am getting 30-33ms per prediction. A couple per bar and the test time is very long. Does this seem appropriate and/or normal? If not, any ideas on what might be going on? The ml files are small 10-20 kb.

By the way I have tried directly in the R console and get the same result. The R bridge does not seem to be causing any issues.

You may want to try predict_on_batch() instead of predict_proba(). I am using Keras with Tensorflow on Python, not R, but had the same issue with prediction time, and predict_on_batch() turned out to be more than an order of magnitude faster, in my setup.

Did you see the improvement even when using just one value? I will give it a try either way, thanks for the tip!

Yes, significant speedup both with one value (vector) as well as with matrices...

I did see some improvement on this with a single data point. It went from about 30-33ms down to 19ms with just one row of data. It is a bit odd, every 14th call or so is the 30ms.