  Public health organizations face the problem of dispensing treatments (i.e.,
vaccines, antibiotics, and others) to groups of affected populations through
"points-of-dispensing" (PODs) during emergency situations, typically in the
presence of complexities like demand stochasticity, heterogenous utilities
(e.g., for vaccine distribution, certain segments of the population may need to
be prioritized), and limited storage. We formulate a hierarchical Markov
decision process (MDP) model with two levels of decisions (and
decision-makers): the upper-level decisions come from an inventory planner that
"controls" a lower-level dynamic problem, which optimizes dispensing decisions
that take into consideration the heterogeneous utility functions of the random
set of PODs. We then derive structural properties of the MDP model and propose
an approximate dynamic programming (ADP) algorithm that leverages structure in
both the policy and the value space (state-dependent basestocks and concavity,
respectively). The algorithm can be considered an actor-critic method; to our
knowledge, this paper is the first to jointly exploit policy and value
structure within an actor-critic framework. We prove that the policy and value
function approximations each converge to their optimal counterparts with
probability one and provide a comprehensive numerical analysis showing improved
empirical convergence rates when compared to other ADP techniques. Finally, we
show how an aggregation-based version of our algorithm can be applied in a
realistic case study for the problem of dispensing naloxone (an overdose
reversal drug) via first responders amidst the ongoing opioid crisis.
