  We discuss how to formulate lattice gauge theories in the Tensor Network
language. In this way we obtain both a consistent truncation scheme of the
Kogut-Susskind lattice gauge theories and a Tensor Network variational ansatz
for gauge invariant states that can be used in actual numerical computation.
Our construction is also applied to the simplest realization of the quantum
link models/gauge magnets and provides a clear way to understand their
microscopic relation with Kogut-Susskind lattice gauge theories. We also
introduce a new set of gauge invariant operators that modify continuously
Rokshar-Kivelson wave functions and can be used to extend the phase diagram of
known models. As an example we characterize the transition between the
deconfined phase of the $Z_2$ lattice gauge theory and the Rokshar-Kivelson
point of the U(1) gauge magnet in 2D in terms of entanglement entropy. The
topological entropy serves as an order parameter for the transition but not the
Schmidt gap.
