  The development of ethical AI systems is currently geared toward setting
objective functions that align with human objectives. However, finding such
functions remains a research challenge, while in RL, setting rewards by hand is
a fairly standard approach. We present a methodology for dynamic value
alignment, where the values that are to be aligned with are dynamically
changing, using a multiple-objective approach. We apply this approach to extend
Deep $Q$-Learning to accommodate multiple objectives and evaluate this method
on a simplified two-leg intersection controlled by a switching agent.Our
approach dynamically accommodates the preferences of drivers on the system and
achieves better overall performance across three metrics (speeds, stops, and
waits) while integrating objectives that have competing or conflicting actions.
