|
3 registered members (Quad, Ayumi, AndrewAMD),
1,092
guests, and 1
spider. |
|
Key:
Admin,
Global Mod,
Mod
|
|
|
adviseLong(NEURAL) in an asset loop
#457103
12/26/15 04:32
12/26/15 04:32
|
Joined: Apr 2014
Posts: 482
Sydney, Australia
boatman
OP
Senior Member
|
OP
Senior Member
Joined: Apr 2014
Posts: 482
Sydney, Australia
|
I'd like to save a separate machine learning model file for each asset in an asset loop using the NEURAL framework in the r.h file. I have some questions that I think would help my understanding of how to achieve this. If I call adviseLong(NEURAL...) in an asset loop, the framework in the r.h file over writes the .ml file with each iteration of the loop. I'd like to save a separate .ml file for each asset. Here is the NEURAL_SAVE and NEURAL_LOAD functions from r.h:
if(mode == NEURAL_SAVE)
return Rx(strf("save(Models,file='%s')",slash(Data)),2);
if(mode == NEURAL_LOAD)
return Rx(strf("load('%s')",slash(Data)),2);
return 1;
The line
Rx(strf("save(Models,file='%s')",slash(Data)),2);
saves a .ml file with the name of the script. But how is the name of the script being accessed here since it is not explicitly specified? If I try
Rx(strf("save(Models,file='%s_%s')", strx(Asset, "/", "_"), slash(Data)),2);
the script tries to save a file with the path Asset_Data/SCRIPT_NAME_1.ml'. Obviously that makes no sense. Likewise if I try
file='%s_%s')", slash(Data)),strx(Asset, "/", "_"),2);
the file path to be saved is Data/SCRIPT_NAME_1.mlASSET'. Again, that makes not much sense. What am I missing? How can I save a model to a file of the format Data/SCRIPT_NAME_ASSET_1.ml using the r.h framework?
|
|
|
Re: adviseLong(NEURAL) in an asset loop
[Re: jcl]
#457315
01/12/16 23:15
01/12/16 23:15
|
Joined: Apr 2014
Posts: 482
Sydney, Australia
boatman
OP
Senior Member
|
OP
Senior Member
Joined: Apr 2014
Posts: 482
Sydney, Australia
|
Thank you, that's clear now. In order to train different models for each asset, I've done this:
if(mode == NEURAL_TRAIN) {
file_write("Data\\signals.csv",Data,0); // export batch training data to R
Rx(strf("Data <- read.csv('%sData/signals.csv',header=F)",slash(ZorroFolder)));
if(!Rx(strf("Model_%s <- <<machine learning algorithm>>)", strx(Asset,"/","")))) return 0;
which works as I expected. I've verified in R that each model is actually trained on the correct data too. I'll load the models in the NEURAL_LOAD function via a string that is dynamically created to accommodate each asset in the loop. One final comment: the manual states that NEURAL_TRAIN is called at the end of any WFO cycle, however I have verified that it is in fact called at the end of any loop() cycle, as you stated. That misleading statement in the manual was a source of some confusion for me. If that could be updated, I'd really appreciate it. Thanks for looking into this for me, jcl.
|
|
|
Re: adviseLong(NEURAL) in an asset loop
[Re: jcl]
#457331
01/13/16 23:10
01/13/16 23:10
|
Joined: Apr 2014
Posts: 482
Sydney, Australia
boatman
OP
Senior Member
|
OP
Senior Member
Joined: Apr 2014
Posts: 482
Sydney, Australia
|
I don't understand how the "model" parameter is incremented within the function. Where should I specify the models that I want to use? And how is that related to the "model" parameter? Let me explain my confusion through my current approach: At the moment, I'm doing the following to train a knn model and a random forest model on the same signals for each asset in a loop (although its not relevant, the 'train' function is from the caret package in R):
if(mode == NEURAL_TRAIN) {
file_write("Data\\signals.csv",Data,0); // export batch training data to R
Rx(strf("Data <- read.csv('%sData/signals.csv',header=F)",slash(ZorroFolder)));
if(!Rx(strf("knn_%s <- train(Data[, c(1,2)], Data[, 3], method = 'knn')", strx(Asset,"/","")))) return 0;
if(!Rx(strf("rf1_%s <- train(Data[, c(1,2)], Data[, 3], method = 'rf')", strx(Asset,"/","")))) return 0;
which seems to work as expected. However, saving and loading the correct models quickly becomes difficult since using this method, I need to specify each model expclicitly:
if(mode == NEURAL_SAVE) {// save the trained model in the Zorro Data folder
return Rx(strf("save(knn_AUDUSD, knn_EURUSD, rf_AUDUSD, rf_AUDUSD, file='%s')", slash(Data)),2);
With many machine learning algorithm types and many assets, this approach would quickly lead to disarray. So my question is, how exactly would I specify the above training and saving cycles using the list approach you suggested? Maybe if you could describe how I interact with the "model" parameter, I would understand better. At the moment, its not clear to me how that parameter is used or specified - I have an inkling that I need to specify it in the adviseLong() call? Thanks for your help, jcl.
Last edited by boatman; 01/13/16 23:45.
|
|
|
Re: adviseLong(NEURAL) in an asset loop
[Re: jcl]
#457338
01/14/16 09:33
01/14/16 09:33
|
Joined: Apr 2014
Posts: 482
Sydney, Australia
boatman
OP
Senior Member
|
OP
Senior Member
Joined: Apr 2014
Posts: 482
Sydney, Australia
|
Thanks, I think I'm catching on. If I understood you correctly, something like this would be the 'official' way to use the NEURAL function (obviously the code is truncated to show only the relevant functions):
{
if(mode == NEURAL_TRAIN) {
file_write("Data\\signals.csv",Data,0); // export batch training data to R
Rx(strf("Data <- read.csv('%sData/signals.csv',header=F)",slash(ZorroFolder)));
if(!Rx("modellist[[model+1]] <- train(Data[, c(1,2)], Data[, 20], method = 'knn')) return 0;
if(!Rx("modellist[[model+1]] <- train(Data[, c(1,2)], Data[, 20], method = 'rf')) return 0;
...
}
while(asset(loop("AUD/USD", "EUR/USD")))
while(algo(loop("knn", "rf"))) {
if (Algo == 'knn') {
adviseLong(NEURAL....);
}
if (Algo == 'rf') {
adviseLong(NEURAL....);
}
}
Will the two calls to the adviseLong() function work as I expect them to in the case above? That is, will the "model" parameter be incremented in both function calls?
Last edited by boatman; 01/14/16 09:33.
|
|
|
Re: adviseLong(NEURAL) in an asset loop
[Re: boatman]
#457341
01/14/16 10:42
01/14/16 10:42
|
Joined: Jul 2000
Posts: 27,986
Frankfurt
jcl
Chief Engineer
|
Chief Engineer
Joined: Jul 2000
Posts: 27,986
Frankfurt
|
No, model is incremented by asset, algo, and long/short. In your example above it is not incremented.
For training two different methods 'knn' and 'rf', normally you would just use two different algos and then get two different model numbers.
if(mode == NEURAL_TRAIN) {
...
if(Algo == "KNN")
Rx("modellist[[model+1]] <- train(Data[, c(1,2)], Data[, 20], method = 'knn');
else if (Algo == "RF")
Rx("modellist[[model+1]] <- train(Data[, c(1,2)], Data[, 20], method = 'rf');
...
|
|
|
Re: adviseLong(NEURAL) in an asset loop
[Re: jcl]
#457363
01/14/16 23:22
01/14/16 23:22
|
Joined: Apr 2014
Posts: 482
Sydney, Australia
boatman
OP
Senior Member
|
OP
Senior Member
Joined: Apr 2014
Posts: 482
Sydney, Australia
|
Thank you, jcl. The correct use of the NEURAL function is clear now. For completeness, I used string formatting to access the "model" parameter as follows:
if(!Rx(strf("modellist[[%d+1]] <- train(Data[, c(1,2)], Data[, 20], method = 'knn')", model)) return 0;
A simple hack for using the model training capacity of the caret package is to name the items in the algo loop so that they correspond with methods in caret and then access those methods via string formatting and the "Algo" variable. For example, the following trains a k-nearest neighbours model and a random forest model and stores them in the list "modellist":
if(mode == NEURAL_TRAIN) {
...
if(!Rx(strf("modellist[[%d+1]] <- train(Data[, c(1,2)], Data[, 20], method = '%s')",model, Algo)) return 0;
...
}
...
while(algo(loop("knn", "rf"))) {
...
|
|
|
|
