  Machine learning models often pose a threat to the privacy of individuals
whose data is part of the training set. Several recent attacks have been able
to infer sensitive information from trained models, including model inversion
or attribute inference attacks. These attacks are able to reveal the values of
certain sensitive features of individuals who participated in training the
model. It has also been shown that several factors can contribute to an
increased risk of model inversion, including feature influence. We observe that
not all features necessarily share the same level of privacy or sensitivity. In
many cases, certain features used to train a model are considered especially
sensitive and therefore propitious candidates for inversion. We present a
solution for countering model inversion attacks in tree-based models, by
reducing the influence of sensitive features in these models. This is an avenue
that has not yet been thoroughly investigated, with only very nascent previous
attempts at using this as a countermeasure against attribute inference. Our
work shows that, in many cases, it is possible to train a model in different
ways, resulting in different influence levels of the various features, without
necessarily harming the model's accuracy. We are able to utilize this fact to
train models in a manner that reduces the model's reliance on the most
sensitive features, while increasing the importance of less sensitive features.
Our evaluation confirms that training models in this manner reduces the risk of
inference for those features, as demonstrated through several black-box and
white-box attacks.
