Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
3 registered members (AndrewAMD, Nymphodora, Quad), 919 guests, and 6 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
R bridge running very slow #470150
12/29/17 08:54
12/29/17 08:54
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
I have a weird issue where my R bridge seems to be running very slow. I'm training a DNN using Keras for R with tensorflow backend. Training takes some time, which is expected (three layers with 20 neurons in each).

My issue is that during testing, it takes 5-10 seconds of time to call the neural.load-function.
With R debugging enabled, the console will output something like
"Load KerasDeepLearn_USDJPY_1" and then after 10 seconds or so
"neural.load('....KerasDeepLearn_USDJPY_1')

I'm not that familiar with the bridge, but the long wait between the Load message from Zorro and the subsequent neural.load message from R seems strange to me. I would expect the log from R milliseconds after the call from Zorro.

I've attached my files (.R extensions are apparently not allowed by the file manager, so I renamed it to .txt)

I'm running Zorro 1.74.2 with R 3.4.1

Attached Files
KerasDeepLearn.c (555 downloads)
KerasDeepLearn.txt (481 downloads)
Last edited by Dalla; 12/29/17 08:55.
Re: R bridge running very slow [Re: Dalla] #470183
01/01/18 17:11
01/01/18 17:11
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
We have not observed bad slowness with Keras so far, but you could look into it by adding timer calls to the neural() function.

Re: R bridge running very slow [Re: jcl] #470189
01/01/18 21:47
01/01/18 21:47
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
I added a timer to the neural load function like this
Code:
if(mode == NEURAL_LOAD) {
		printf("nLoad %s",strrchr(Data,'')+1);
		timer();
		int neuralLoadReturnCode = Rx(strf("neural.load('%s')",slash(Data)),3); 		
		printf("nNEURAL_LOAD call time = %.3f ms",timer()); 
		return neuralLoadReturnCode;
 	}



This is the output (only included first five model calls)
Code:
Load KerasDeepLearn_USDJPY_1.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_1.ml')
NEURAL_LOAD call time = 14981.734 ms
Load KerasDeepLearn_USDJPY_2.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_2.ml')
NEURAL_LOAD call time = 11214.417 ms
Load KerasDeepLearn_USDJPY_3.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_3.ml')
NEURAL_LOAD call time = 13523.271 ms
Load KerasDeepLearn_USDJPY_4.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_4.ml')
NEURAL_LOAD call time = 17167.255 ms
Load KerasDeepLearn_USDJPY_5.ml
neural.load('Y:/Zorro/Data/KerasDeepLearn_USDJPY_5.ml')
NEURAL_LOAD call time = 19805.827 ms



As you can see the calls take 10 to 20 seconds

I should add the the models are small, only ~70kb in size
I also tried setting up a completly new environment on AWS and ran the same script, but the issue remains

Last edited by Dalla; 01/01/18 21:51.
Re: R bridge running very slow [Re: Dalla] #470192
01/02/18 08:56
01/02/18 08:56
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
Hmm, when I run the Deeplearn script with a 2-layer Keras net, I'm also getting about 500 ms loading time. That's indeed a bit slow. I don't know what takes so long.

Maybe you can get some hints about how to speed up loading on Keras or Tensorflow forums - otherwise you have to live with it.

Re: R bridge running very slow [Re: jcl] #470196
01/02/18 10:06
01/02/18 10:06
Joined: Feb 2017
Posts: 369
D
Dalla Offline OP
Senior Member
Dalla  Offline OP
Senior Member
D

Joined: Feb 2017
Posts: 369
Thanks, good to know it's not an issue on my side at least. I´ve found others having similar issues, but no solution yet.

Re: R bridge running very slow [Re: Dalla] #473115
06/14/18 10:33
06/14/18 10:33
Joined: Oct 2017
Posts: 17
J
JRA Offline
Newbie
JRA  Offline
Newbie
J

Joined: Oct 2017
Posts: 17
Hi Dalla:

Have you tested the GPU version? I´m running it but at the moment it´s not an improvement over CPU.


Moderated by  Petra 

Powered by UBB.threads™ PHP Forum Software 7.7.1