  We study a general problem of allocating limited resources to heterogeneous
customers over time under model uncertainty. Each type of customer can be
serviced using different actions, each of which stochastically consumes some
combination of resources, and returns different rewards for the resources
consumed. We consider a general model where the resource consumption
distribution associated with each (customer type, action)-combination is not
known, but is consistent and can be learned over time. In addition, the
sequence of customer types to arrive over time is arbitrary and completely
unknown.
  We overcome both the challenges of model uncertainty and customer
heterogeneity by judiciously synthesizing two algorithmic frameworks from the
literature: inventory balancing, which "reserves" a portion of each resource
for high-reward customer types which could later arrive, and online learning,
which shows how to "explore" the resource consumption distributions of each
customer type under different actions. We define an auxiliary problem, which
allows for existing competitive ratio and regret bounds to be seamlessly
integrated. Furthermore, we show that the performance guarantee generated by
our framework is tight, that is, we provide an information-theoretic lower
bound which shows that both the loss from competitive ratio and the loss for
regret are relevant in the combined problem.
  Finally, we demonstrate the efficacy of our algorithms on a publicly
available hotel data set. Our framework is highly practical in that it requires
no historical data (no fitted customer choice models, nor forecasting of
customer arrival patterns) and can be used to initialize allocation strategies
in fast-changing environments.
