  Joint Energy-based Model (JEM) is a recently proposed hybrid model that
retains strong discriminative power of modern CNN classifiers, while generating
samples rivaling the quality of GAN-based approaches. In this paper, we propose
a variety of new training procedures and architecture features to improve JEM's
accuracy, training stability, and speed altogether. 1) We propose a proximal
SGLD to generate samples in the proximity of samples from the previous step,
which improves the stability. 2) We further treat the approximate maximum
likelihood learning of EBM as a multi-step differential game, and extend the
YOPO framework to cut out redundant calculations during backpropagation, which
accelerates the training substantially. 3) Rather than initializing SGLD chain
from random noise, we introduce a new informative initialization that samples
from a distribution estimated from training data. 4) This informative
initialization allows us to enable batch normalization in JEM, which further
releases the power of modern CNN architectures for hybrid modeling. Code:
https://github.com/sndnyang/JEMPP
