Number of parameters in an artificial neural network for AIC

How can I calculate the number of parameters in an artificial neural network in order to calculate its AIC?

Answer

Every connection that is learned in a feedforward network is a parameter. Here is an image of a generic network from Wikipedia:

enter image description here

This network is fully connected, although networks don’t have to be (e.g., designing a network with receptive fields improves edge detection in images). With a fully connected ANN, the number of connections is simply the sum of the product of the numbers of nodes in connected layers. In the image above, that is $(3\times 4) + (4\times 2) = 20$. That image does not show any bias nodes, but many ANNs do have them; if so, include the bias node in the total for that layer. More generally (e.g., if your ANN isn’t fully connected), you can simply count the connections.

Attribution
Source : Link , Question Author : Funkwecker , Answer Author : gung – Reinstate Monica

Leave a Comment