  In this paper, we investigate discrete-time decision-making problems in
uncertain systems with partially observed states. We consider a non-stochastic
model, where uncontrolled disturbances acting on the system take values in
bounded sets with unknown distributions. We present a general framework for
decision-making in such problems by using the notion of the information state
and approximate information state, and introduce conditions to identify an
uncertain variable that can be used to compute an optimal strategy through a
dynamic program (DP). Next, we relax these conditions and define approximate
information states that can be learned from output data without knowledge of
system dynamics. We use approximate information states to formulate a DP that
yields a strategy with a bounded performance loss. Finally, we illustrate the
application of our results in control and reinforcement learning using
numerical examples.
