  This work proposes a compositional data-driven technique for the construction
of finite Markov decision processes (MDPs) for large-scale stochastic networks
with unknown mathematical models. Our proposed framework leverages
dissipativity properties of subsystems and their finite MDPs using a notion of
stochastic storage functions (SStF). In our data-driven scheme, we first build
an SStF between each unknown subsystem and its data-driven finite MDP with a
certified probabilistic confidence. We then derive dissipativity-type
compositional conditions to construct a stochastic bisimulation function (SBF)
between an interconnected network and its finite MDP using data-driven SStF of
subsystems. Accordingly, we formally quantify the probabilistic distance
between trajectories of an unknown large-scale stochastic network and those of
its finite MDP with a guaranteed confidence. We illustrate the efficacy of our
data-driven results over a room temperature network composing 100 rooms with
unknown models.
