multi-layer perceptron (MLP) architecture: criteria for choosing number of hidden layers and size of the hidden layer?

Question

If we have 10 eigenvectors then we can have 10 neural nodes in input layer.If we have 5 output classes then we can have 5 nodes in output layer.But what is the criteria for choosing number of hidden layer in a MLP and how many neural nodes in 1 hidden layer?

Answer 1

There is no rule to find out how many hidden layers you need. In many cases, one hidden layer works well, but in order to justify this for a specific problem, you have to apply a heuristic method such as cross-validation. Using cross-validation you divide your data into two parts namely training set and validation set (also called test set).

Assuming your data does require separation by a non-linear technique, then always start with one hidden layer. Almost certainly that's all you will need. If your data is separable using an MLP, then that MLP probably only needs a single hidden layer.

For getting the number of neurons in the hidden layer, there is a technique which I generally follow:

• During the building of your model, test your model thorough testing will reveal the signatures of "incorrect" network architecture. For instance, if you begin with an MLP having a hidden layer comprised of a small number of nodes (which you will gradually increase as needed, based on test results) your training and generalization error will both be high caused by bias and underfitting.

• Then increase the number of nodes in the hidden layer, one at a time, until the generalization error begins to increase, this time due to overfitting and high variance.

Performing the above steps you can easily get the number of nodes in the hidden layer.

Hope this helps!