  Modal regression is aimed at estimating the global mode (i.e., global
maximum) of the conditional density function of the output variable given input
variables, and has led to regression methods robust against heavy-tailed or
skewed noises. The conditional mode is often estimated through maximization of
the modal regression risk (MRR). In order to apply a gradient method for the
maximization, the fundamental challenge is accurate approximation of the
gradient of MRR, not MRR itself. To overcome this challenge, in this paper, we
take a novel approach of directly approximating the gradient of MRR. To
approximate the gradient, we develop kernelized and neural-network-based
versions of the least-squares log-density derivative estimator, which directly
approximates the derivative of the log-density without density estimation. With
direct approximation of the MRR gradient, we first propose a modal regression
method with kernels, and derive a new parameter update rule based on a
fixed-point method. Then, the derived update rule is theoretically proved to
have a monotonic hill-climbing property towards the conditional mode.
Furthermore, we indicate that our approach of directly approximating the
gradient is compatible with recent sophisticated stochastic gradient methods
(e.g., Adam), and then propose another modal regression method based on neural
networks. Finally, the superior performance of the proposed methods is
demonstrated on various artificial and benchmark datasets.
