  Broadly speaking Information theory (IT) assumes no structure of the
underlying states. But what about contexts where states do have a clear
structure - how should IT cope with such situations? And if such coping is at
all possible then - how should structure be expressed so that it can be coped
with? A possible answer to these questions is presented here. Noting that IT
can cope well with a structure expressed as an accurate clustering (by shifting
to the implied reduced alphabet), a generalization is suggested in which
structure is expressed as a measure on reduced alphabets. Given such structure
an extension of IT is presented where the reduced alphabets are treated
simultaneously. This structure-sensitive IT, called ITs, extends traditional IT
in the sense that: a)there are structure-sensitive analogs to the notions of
traditional IT and b)translating a theorem in IT by replacing its notions with
their structure-sensitive counterparts, yields a (provable) theorem of ITs.
Seemingly paradoxically, ITs extends IT but it's completely within the
framework of IT. The richness of the suggested structures is demonstrated by
two disparate families studied in more detail: the family of hierarchical
structures and the family of linear structures. The formal findings extend the
scope of cases to which a rigorous application of IT can be applied (with
implications on quantization, for example). The implications on the foundations
of IT are that the assumption regarding no underlying structure of states is
not mandatory and that there is a framework for expressing such underlying
structure.
