'neural' function

Posted By: gtell

'neural' function - 09/12/16 21:55

Dear JCL,
dear all,

I am using adviceLong and I need to use:

Code:
prepr<-preProcess(X, method = "spatialSign")


in neural.train and:

Code:
x.ts<-predict(prepr, X)


in neural.predict.

Both in neural.train and neural.predict use the same "prepr" object.
How can I save/load it?
Could you give me an idea?
What I want to achieve is to pre-process data in the same way both in training and test.
Thanks.
Cheers.
Posted By: jcl

Re: 'neural' function - 09/13/16 06:56

Save a list of prepr objects in the model file together with the models list, like this:

save(Models,Preprs,file=name)
Posted By: gtell

Re: 'neural' function - 09/13/16 10:18

Thanks, good idea.
Unfortunatelly I get an error in the testing phase after training.

The following is my neural script:

Code:
library('deepnet', quietly = T) 
library('caret', quietly = T)


neural.train = function(model,XY) 
{
  XY <- as.matrix(XY)
  X <- XY[,-ncol(XY)]
  Y <- XY[,ncol(XY)]  
  Y <- ifelse(Y > 0,1,0)
  prepr<-preProcess(X, method = "spatialSign")
  x.tr<-predict(prepr, X)   
  Preprs[[model]] <<- prepr
  Models[[model]] <<- sae.dnn.train(x.tr,Y,
      hidden = c(50,50,50), 
      activationfun = "tanh", 
      learningrate = 0.7, 
      momentum = 0.5, 
      learningrate_scale = 1.0, 
      output = "sigm", 
      sae_output = "linear", 
      #numepochs = 300, 
      batchsize = 100,
      hidden_dropout = 0, 
      visible_dropout = 0)
}

neural.predict = function(model,X) 
{
  if(is.vector(X)) X <- t(X)
  x.ts<-predict(Preprs[[model]], X)
  return(nn.predict(Models[[model]], x.ts))
}

neural.init = function()
{
  set.seed(365)
  Models <<- vector("list")
  Preprs <<- vector("list")
}

neural.save = function(name)
{
  save(Models,Preprs,file=name)  
}

neural.load = function(name)
{
  load(Models,Preprs,file=name)  
}


even if I remove "neural.load" (not sure if I need it) I always get the following error:

Code:
Load DeepLearnSAE2.4_EURUSD_2.ml
R error -  
Error in newdata[, object$method$center, drop = FALSE] : 
  attribute 'dimnames' missing for array
Calls: neural.predict -> predict -> predict.preProcess -> sweep
Execution stopped
 
Profit 8$  MI 2$  DD 23$  Capital 100$
Trades 596  Win 52.2%  Avg +0.1p  Bars 2
AR 27%  PF 1.06  SR 0.64  UI 38%  R2 0.00
Quit
Time 00:01:16


If I run a TestOOS() from R, it works.
I am not sure where it is the error.
What do you think?
Posted By: jcl

Re: 'neural' function - 09/13/16 10:41

neural.load is not used, since the R load function loads every object from the file anyway. But I do not see the reason of the error immediately in the code - you must debug it. The predict function seems to expect either the X array or the Preprs object in a different format, so the first step would be comparing the original object with the object read back from the file, and also checking the array.
Posted By: gtell

Re: 'neural' function - 09/13/16 19:34

I could troubleshoot it and it was small mistake.
The preProcess requires the column names of the dataframe you are passing in the predict are the same of what you used in the train.
So this is solved! Thanks for your help.
However with Zorro I am getting 10% less of winning trades then with R. I do not understand why and this drive me crazy...
Posted By: gtell

Re: 'neural' function - 09/14/16 16:07

So, my question is very simple. If running the TestOOS function you get the following accurancy:

(found in your article: http://www.financial-hacker.com/build-better-strategies-part-5-developing-a-machine-learning-system/ )

Code:
Confusion Matrix and Statistics

          Reference
Prediction    0    1
         0 1231  808
         1  512  934
                                          
               Accuracy : 0.6212          
                 95% CI : (0.6049, 0.6374)
    No Information Rate : 0.5001          
    P-Value [Acc > NIR] : < 2.2e-16


which is 62%, shall I expect the almost same rate of winning trades in Zorro?
Because I am getting about 10-15% less.
I do not understand what is the reason.
Thanks.
Cheers.
Posted By: jcl

Re: 'neural' function - 09/15/16 05:49

The accuracy is the ratio of right to wrong predictions. Since this depends on the data, you will get a different accuracy with different data. This is perfectly normal.

The TestOOS accuracy is based on a single test and not very relevant. The WFA accuracy is relevant. But if you get consistently a better TestOOS accuracy, the WFA period might be bad choosen.
© 2024 lite-C Forums