Gamestudio Links
Zorro Links
Newest Posts
Blobsculptor tools and objects download here
by NeoDumont. 03/28/24 03:01
Issue with Multi-Core WFO Training
by aliswee. 03/24/24 20:20
Why Zorro supports up to 72 cores?
by Edgar_Herrera. 03/23/24 21:41
Zorro Trader GPT
by TipmyPip. 03/06/24 09:27
VSCode instead of SED
by 3run. 03/01/24 19:06
AUM Magazine
Latest Screens
The Bible Game
A psychological thriller game
SHADOW (2014)
DEAD TASTE
Who's Online Now
5 registered members (Nymphodora, AndrewAMD, TipmyPip, Quad, Imhotep), 847 guests, and 4 spiders.
Key: Admin, Global Mod, Mod
Newest Members
sakolin, rajesh7827, juergen_wue, NITRO_FOREVER, jack0roses
19043 Registered Users
Previous Thread
Next Thread
Print Thread
Rate Thread
Help with Neural predict -> "subscript out of bounds" error #477030
05/03/19 21:59
05/03/19 21:59
Joined: Nov 2016
Posts: 103
NSW
V
vinsom Offline OP
Member
vinsom  Offline OP
Member
V

Joined: Nov 2016
Posts: 103
NSW
Hi,
Does anyone has any idea why this is happening during the NEURAL testing after training ?
Getting "subscript out of bounds" from the R script when there are multiple assets or NumWFOCycles >60.
If I remove 1 of the 2 asset, or set NumWFOCycles=50 , this issue is not happening.


/////////////////////////// Zorro script
#include <r.h>

var change(int n)
{
return scale((priceClose(0) - priceClose(n))/priceClose(0),100)/100;
}

function run()
{
NumCores = -1;
Script = "DeepLearnMXDEBUG";
Verbose = 31|DIAG;
StartDate = 2016;//2016;
BarPeriod = 1440; // 1 hour
LookBack = 100;
Weekend=0;

NumWFOCycles = 60;

assetList("History\AssetsBitmex.csv");
while(loop(Assets))
{
asset(Loop1);
set(RULES);

LifeTime = 3;
if(Train) Hedge = 2;
var Threshold = 0.5;

set(LOGFILE|PLOTNOW);
if(adviseLong(NEURAL+BALANCED,0,
change(1),change(2),change(3),change(4)) > 0.5)
enterLong();
if(adviseShort() > 0.5)
enterShort();

PlotWidth = 800;
PlotHeight1 = 340;

}
}

////////////////////////// Zorro assets

Name,Price,Spread,RollLong,RollShort,PIP,PIPCost,MarginCost,Leverage,LotAmount,Commission,Symbol,Type
ETH/BTC,1,0,0,0,0.00000001,0.00000001,0,1,1,-0.25,ETH/BTC,
BCH/BTC,1,0,0,0,0.00000001,0.00000001,0,1,1,-0.25,BCH/BTC,

/////////////////////////// R File

# how to install
#cran <- getOption("repos")
#cran["dmlc"] <- "https://s3-us-west-2.amazonaws.com/apache-mxnet/R/CRAN/"
#options(repos = cran)
#install.packages('mxnet')
# - or (if not yet available for current version) -
#install.packages("https://s3.ca-central-1.amazonaws.com/jeremiedb/share/mxnet/CPU/mxnet.zip", repos = NULL)
library('mxnet', quietly = T)
library('caret', quietly = T)

neural.train = function(model,XY)
{
X <- data.matrix(XY[,-ncol(XY)])
Y <- XY[,ncol(XY)]
Y <- ifelse(Y > 0,1,0)
Models[[model]] <<- mx.mlp(X,Y,
hidden_node = c(30),
out_node = 2,
activation = "sigmoid",
out_activation = "softmax",
num.round = 20,
array.batch.size = 20,
learning.rate = 0.05,
momentum = 0.9,
eval.metric = mx.metric.accuracy)
}

neural.predict = function(model,X)
{
if(is.vector(X)) X <- t(X)
X <- data.matrix(X)
Y <- predict(Models[[model]],X)
return(ifelse(Y[1,] > Y[2,],0,1))
}

neural.save = function(name)
{
for(i in c(1:length(Models)))
Models[[i]] <<- mx.serialize(Models[[i]])
save(Models,file=name)
}

neural.load <- function(name)
{
load(name,.GlobalEnv)
for(i in c(1:length(Models)))
Models[[i]] <<- mx.unserialize(Models[[i]])
}

neural.init = function()
{
mx.set.seed(365)
Models <<- vector("list")
}

neural.test = function()
{
neural.init()
XY <<- read.csv('MyZorroPath/Data/DeepLearnMX.csv',header = F)
splits <- nrow(XY)*0.8
XY.tr <<- head(XY,splits)
XY.ts <<- tail(XY,-splits)
neural.train(1,XY.tr)

X <<- XY.ts[,-ncol(XY.ts)]
Y <<- XY.ts[,ncol(XY.ts)]
Y.ob <<- ifelse(Y > 0,1,0)
Y.pr <<- neural.predict(1,X)
confusionMatrix(as.factor(Y.pr),as.factor(Y.ob))
}

///////////////////////////// Zorro output

Error in Models[[model]] : subscript out of bounds
Calls: neural.predict -> predict
Execution halted

Attached Files
DeepLearnDEBUG.c (1 downloads)
Re: Help with Neural predict -> "subscript out of bounds" error [Re: vinsom] #477032
05/04/19 07:14
05/04/19 07:14
Joined: Jul 2000
Posts: 27,977
Frankfurt
jcl Offline

Chief Engineer
jcl  Offline

Chief Engineer

Joined: Jul 2000
Posts: 27,977
Frankfurt
I would interpret this error as if not all models are trained, so the Models vector is too small.

Look into the training log: has it error messages?

Re: Help with Neural predict -> "subscript out of bounds" error [Re: vinsom] #477034
05/04/19 20:18
05/04/19 20:18
Joined: Nov 2016
Posts: 103
NSW
V
vinsom Offline OP
Member
vinsom  Offline OP
Member
V

Joined: Nov 2016
Posts: 103
NSW
Hi jcl,
No errors in the training log.

I run the Debug View, and there I can see all input/output to R.
All logs attached (training/testing/diag/debugview), if can help.

Thanks

Attached Files
Log.zip (0 downloads)
Re: Help with Neural predict -> "subscript out of bounds" error [Re: vinsom] #477035
05/04/19 20:34
05/04/19 20:34
Joined: Nov 2016
Posts: 103
NSW
V
vinsom Offline OP
Member
vinsom  Offline OP
Member
V

Joined: Nov 2016
Posts: 103
NSW
Just want to add that the error is in the debug-view log when testing (attached previour post).
Input/Output to R looks normal in this file.

The traininig looks like might be incomplete from the screenshot attached, as the purple bar didn't go to the end,and 57 runs instead of 60, something might be happening during training, that actually causes the error when testing.

But there are no errors during training at all...

Attaching a screenshot, the debug-view training log (the previous one was for testing) and the train log again if can help

Attached Files
Log.zip (0 downloads)
Last edited by vinsom; 05/04/19 20:59.
Re: Help with Neural predict -> "subscript out of bounds" error [Re: vinsom] #477036
05/05/19 06:26
05/05/19 06:26
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
Hi, you have day bars and only 3 years, so thats far too few bars for 60 wfo cycles. I think you cannot train a cycle that has only a few bars, maybe you get no result for some cycles and thats the reason of the error. Try 20 years instead of 3, or 10 wfo cycles instead of 60.

Re: Help with Neural predict -> "subscript out of bounds" error [Re: Petra] #477039
05/05/19 07:26
05/05/19 07:26
Joined: Nov 2016
Posts: 103
NSW
V
vinsom Offline OP
Member
vinsom  Offline OP
Member
V

Joined: Nov 2016
Posts: 103
NSW
Yes I thought so, but then if remove either one of the 2 asset, then there are no errors and training complete successfully and testing afterwards has no issue

Re: Help with Neural predict -> "subscript out of bounds" error [Re: vinsom] #477040
05/05/19 10:35
05/05/19 10:35
Joined: Apr 2008
Posts: 585
Austria
Petra Offline
Support
Petra  Offline
Support

Joined: Apr 2008
Posts: 585
Austria
Thats exactly the problem, the first asset determines the bars, so you probably have some cycles with no candles of the second asset where they dont overlap. Especially with cryptos that have lots of gaps anyway.

Re: Help with Neural predict -> "subscript out of bounds" error [Re: Petra] #477047
05/05/19 21:47
05/05/19 21:47
Joined: Nov 2016
Posts: 103
NSW
V
vinsom Offline OP
Member
vinsom  Offline OP
Member
V

Joined: Nov 2016
Posts: 103
NSW
I think the issue in training has been fixed with adding array.layout = "rowmajor" in the training function of the R script.
There is still a warning in the predict function as from the log below, but not sure how to add the same code to the predict function.
Anyone has any idea ?

Thanks

log:
[27312] <2> ExecuteCode: in >>> Y <- neural.predict(2,X)
[27312] <2> ExecuteCode: out <<< arning message:
[27312] In mx.model.select.layout.predict(X, model) :
[27312] Auto detect layout of input matrix, use rowmajor..


neural.train = function(model,XY)
{
X <- data.matrix(XY[,-ncol(XY)])
Y <- XY[,ncol(XY)]
Y <- ifelse(Y > 0,1,0)
Models[[model]] <<- mx.mlp(X,Y,
hidden_node = c(60),
out_node = 2,
activation = "sigmoid",
out_activation = "softmax",
num.round = 10,
array.batch.size = 40,
learning.rate = 0.02,
momentum = 1.5,
eval.metric = mx.metric.accuracy,
array.layout = "rowmajor")
}


Moderated by  Petra 

Gamestudio download | chip programmers | Zorro platform | shop | Data Protection Policy

oP group Germany GmbH | Birkenstr. 25-27 | 63549 Ronneburg / Germany | info (at) opgroup.de

Powered by UBB.threads™ PHP Forum Software 7.7.1