# What if all the nodes are dropped when using dropout?

When implementing dropout (or drop connect) – do you need to account for the case that every node in a layer is dropped?

Even though this is a very small chance, what is the correct approach to take in this scenario? Pick a new random set to drop out or set all the inputs to the next layer to be zero?

Does anyone know what the popular libraries (TensorFlow, Keras, etc.) do in this situation?

This is a concern which will very rarely every be realized. For a moderately sized neural network whose hidden layers each have $1000$ units, if the dropout probability is set to $p=0.5$ (the high end of what’s typically used) then the probability of all $1000$ units being zero is $0.5^{1000} = 9.3\times10^{-302}$ which is a mind-bogglingly tiny value. Even for a very small neural network with only $50$ units in the hidden layer, the probability of all units being zero is $.5^{50}=8.9\times10^{-16}$, or less than $\frac{1}{1\ \text{thousand trillion}}$