  Penalized empirical risk minimization with a surrogate loss function is often
used to derive a high-dimensional linear decision rule in classification
problems. Although much of the literature focuses on the generalization error,
there is a lack of valid inference procedures to identify the driving factors
of the estimated decision rule, especially when the surrogate loss is
non-differentiable. In this work, we propose a kernel-smoothed decorrelated
score to construct hypothesis testing and interval estimations for the linear
decision rule estimated using a piece-wise linear surrogate loss, which has a
discontinuous gradient and non-regular Hessian. Specifically, we adopt kernel
approximations to smooth the discontinuous gradient near discontinuity points
and approximate the non-regular Hessian of the surrogate loss. In applications
where additional nuisance parameters are involved, we propose a novel
cross-fitted version to accommodate flexible nuisance estimates and kernel
approximations. We establish the limiting distribution of the kernel-smoothed
decorrelated score and its cross-fitted version in a high-dimensional setup.
Simulation and real data analysis are conducted to demonstrate the validity and
superiority of the proposed method.
