- I enjoy flying.
- I like NLP.
- I like deep learning
Now we can apply Singular Value Decomposition to this matrix to get $X = U \Sigma V^T$ where $U$ and $V$ are orthogonal and $\Sigma$ is diagonal and the singular values are sorted (I believe) so $\sigma_1 \geq \sigma_2 \geq \ldots \geq \sigma_r$.
My question is what do the matrices $U$, $\Sigma$, $V$ represent in terms of the words and/or sentences and the relationship between words.
If I think about matrices as linear transformations essentially SVD is taking some matrix $X$ and decomposing it into a (rotation)(stretch)(rotation).
I am ultimately looking for a way to think about the SVD decomposition of a co-occurrence matrix more intuitively.