  A key issue in the control of distributed discrete systems modeled as Markov
decisions processes, is that often the state of the system is not directly
observable at any single location in the system. The participants in the
control scheme must share information with one another regarding the state of
the system in order to collectively make informed control decisions, but this
information sharing can be costly. Harnessing recent results from information
theory regarding distributed function computation, in this paper we derive, for
several information sharing model structures, the minimum amount of control
information that must be exchanged to enable local participants to derive the
same control decisions as an imaginary omniscient controller having full
knowledge of the global state. Incorporating consideration for this amount of
information that must be exchanged into the reward enables one to trade the
competing objectives of minimizing this control information exchange and
maximizing the performance of the controller. An alternating optimization
framework is then provided to help find the efficient controllers and messaging
schemes. A series of running examples from wireless resource allocation
illustrate the ideas and design tradeoffs.
