  Discriminative latent variable models (LVM) are frequently applied to various
visual recognition tasks. In these systems the latent (hidden) variables
provide a formalism for modeling structured variation of visual features.
Conventionally, latent variables are de- fined on the variation of the
foreground (positive) class. In this work we augment LVMs to include negative
latent variables corresponding to the background class. We formalize the
scoring function of such a generalized LVM (GLVM). Then we discuss a framework
for learning a model based on the GLVM scoring function. We theoretically
showcase how some of the current visual recognition methods can benefit from
this generalization. Finally, we experiment on a generalized form of Deformable
Part Models with negative latent variables and show significant improvements on
two different detection tasks.
