  Ridge regularized linear models (RRLMs), such as ridge regression and the
SVM, are a popular group of methods that are used in conjunction with
coefficient hypothesis testing to discover explanatory variables with a
significant multivariate association to a response. However, many investigators
are reluctant to draw causal interpretations of the selected variables due to
the incomplete knowledge of the capabilities of RRLMs in causal inference.
Under reasonable assumptions, we show that a modified form of RRLMs can get
very close to identifying a subset of the Markov boundary by providing a
worst-case bound on the space of possible solutions. The results hold for any
convex loss, even when the underlying functional relationship is nonlinear, and
the solution is not unique. Our approach combines ideas in Markov boundary and
sufficient dimension reduction theory. Experimental results show that the
modified RRLMs are competitive against state-of-the-art algorithms in
discovering part of the Markov boundary from gene expression data.
