# Inverse word embedding: vector to word

I’m building a generative text model, and the output of one of the final layers is a word embedding (vector) of the generated word. I’m left with the task of converting this vector back to the actual word.

Is there a good algorithm for doing this inversion? I’m thinking of using a fully-connected/dense layer, but then it’s decoupled from the original (forward) embedding layer. Ideally, I’d think it’s better to make use of parameters of the embedding layer somehow for the inversion.

## Answer

There is no one ‘right’ way to turn wordvectors back into words.
The issue is that the words themselves form a discrete set of points in the embedding space, and so the output of a model is very unlikely to be exactly equal to the location of any word.

Typically if your model emits a vector $$vv$$ then interpreting it as a word is done by finding a word $$ww$$ with embedding $$vwv_w$$ such that $$d(v,vw)d(v, v_w)$$ is small, i.e. $$vv$$ is ‘close’ to the embedding of $$ww$$. Choosing the distance function $$dd$$ is up to you, although typically the cosine similarity is used. Depending on the application, you could also consider showing the top-$$kk$$ similar words to your wordvector, which could offer a bit more diversity.

Attribution
Source : Link , Question Author : rishai , Answer Author : Chris Cundy