  As is the case of many signals produced by complex systems, language presents
a statistical structure that is balanced between order and disorder. Here we
review and extend recent results from quantitative characterisations of the
degree of order in linguistic sequences that give insights into two relevant
aspects of language: the presence of statistical universals in word ordering,
and the link between semantic information and the statistical linguistic
structure. We first analyse a measure of relative entropy that assesses how
much the ordering of words contributes to the overall statistical structure of
language. This measure presents an almost constant value close to 3.5 bits/word
across several linguistic families. Then, we show that a direct application of
information theory leads to an entropy measure that can quantify and extract
semantic structures from linguistic samples, even without prior knowledge of
the underlying language.
