  Agents that assist people need to have well-initialized policies that can
adapt quickly to align with their partners' reward functions. Initializing
policies to maximize performance with unknown partners can be achieved by
bootstrapping nonlinear models using imitation learning over large, offline
datasets. Such policies can require prohibitive computation to fine-tune
in-situ and therefore may miss critical run-time information about a partner's
reward function as expressed through their immediate behavior. In contrast,
online logistic regression using low-capacity models performs rapid inference
and fine-tuning updates and thus can make effective use of immediate in-task
behavior for reward function alignment. However, these low-capacity models
cannot be bootstrapped as effectively by offline datasets and thus have poor
initializations. We propose BLR-HAC, Bootstrapped Logistic Regression for Human
Agent Collaboration, which bootstraps large nonlinear models to learn the
parameters of a low-capacity model which then uses online logistic regression
for updates during collaboration. We test BLR-HAC in a simulated surface
rearrangement task and demonstrate that it achieves higher zero-shot accuracy
than shallow methods and takes far less computation to adapt online while still
achieving similar performance to fine-tuned, large nonlinear models. For code,
please see our project page https://sites.google.com/view/blr-hac.
