What guidelines should be followed for using Neural Networks with sparse inputs

I have extremely sparse inputs, e.g. locations of certain features in an input image. Further each feature can have multiple detections (not sure if this will have a bearing on the design of the system). This I shall be presenting as an k channel ‘binary image’ with ON pixels representing presence of that feature, and vice versa. We can see that such an input is bound to be very sparse.

So, are there any recommendations when using sparse data with neural nets, specifically data that representative of detections/locations?

Answer

You can try using feature embeddings to reduce the dimension of the input space. Sort of the word2vec approach in NLP, it seems like it could apply in your case since your features are binary (On/Off).

Attribution
Source : Link , Question Author : user3246971 , Answer Author : Miguel

Leave a Comment