  We study a multi-objective model on the allocation of reusable resources
under model uncertainty. Heterogeneous customers arrive sequentially according
to a latent stochastic process, request for certain amounts of resources, and
occupy them for random durations of time. The decision maker's goal is to
simultaneously maximize multiple types of rewards generated by the customers,
while satisfying the resource capacity constraints in each time step. We
develop models and algorithms for deciding on the allocation actions. We show
that when the usage duration is relatively small compared with the length of
the planning horizon, our policy achieves $1-O(\epsilon)$ fraction of the
optimal expected rewards, where $\epsilon$ decays to zero at a near optimal
rate as the resource capacities grow.
