  Misalignment between model predictions and intended usage can be detrimental
for the deployment of computer vision models. The issue is exacerbated when the
task involves complex structured outputs, as it becomes harder to design
procedures which address this misalignment. In natural language processing,
this is often addressed using reinforcement learning techniques that align
models with a task reward. We adopt this approach and show its surprising
effectiveness across multiple computer vision tasks, such as object detection,
panoptic segmentation, colorization and image captioning. We believe this
approach has the potential to be widely useful for better aligning models with
a diverse range of computer vision tasks.
