matlab - Issues with neural network -
i having issues using neural network. using non linear activation function hidden layer , linear function output layer. adding more neurons in hidden layer should have increased capability of nn , made fit training data more/have less error on training data.
however, seeing different phenomena. adding more neurons decreasing accuracy of neural network on training set. 
here graph of mean absolute error increasing number of neurons. accuracy on training data decreasing. cause of this?
is nntool using of matlab splits data randomly training,test , validation set checking generalization instead of using cross validation.
also see lots of -ve output values adding neurons while targets supposed positives. issues?
i not able explain behavior of nn here. suggestions? here link data consisting of covariates , targets
i unfamiliar nntool suspect problem related selection of initial weights. poor initial weight selection can lead slow convergence or failure converge @ all.
for instance, notice number of neurons in hidden layer increases, number of inputs each neuron in visible layer increases (one each hidden unit). using logit in hidden layer (always positive) , pick initial weights random uniform distribution between fixed interval. number of hidden units increases, inputs each neuron in visible layer increase because there more incoming connections. large number of hidden units, initial solution may become large , result in poor convergence.
of course, how behaves depends on activation functions , distributio of data , how normalized. recommend looking @ efficient backprop yann lecun excellent advice on normalizing data , selecting initial weights , activation functions.
Comments
Post a Comment