  In response to legislation mandating companies to honor the \textit{right to
be forgotten} by erasing user data, it has become imperative to enable data
removal in Vertical Federated Learning (VFL) where multiple parties provide
private features for model training. In VFL, data removal, i.e.,
\textit{machine unlearning}, often requires removing specific features across
all samples under privacy guarentee in federated learning. To address this
challenge, we propose \methname, a novel Gradient Boosting Decision Tree (GBDT)
framework that effectively enables both \textit{instance unlearning} and
\textit{feature unlearning} without the need for retraining from scratch.
Leveraging a robust GBDT structure, we enable effective data deletion while
reducing degradation of model performance. Extensive experimental results on
popular datasets demonstrate that our method achieves superior model utility
and forgetfulness compared to \textit{state-of-the-art} methods. To our best
knowledge, this is the first work that investigates machine unlearning in VFL
scenarios.
