  Deep learning models are susceptible to adversarial attacks, where slight
perturbations to input data lead to misclassification. Adversarial attacks
become increasingly effective with access to information about the targeted
classifier. In the context of multi-task learning, where a single model learns
multiple tasks simultaneously, attackers may aim to exploit vulnerabilities in
specific tasks with limited information. This paper investigates the
feasibility of attacking hidden tasks within multi-task classifiers, where
model access regarding the hidden target task and labeled data for the hidden
target task are not available, but model access regarding the non-target tasks
is available. We propose a novel adversarial attack method that leverages
knowledge from non-target tasks and the shared backbone network of the
multi-task model to force the model to forget knowledge related to the target
task. Experimental results on CelebA and DeepFashion datasets demonstrate the
effectiveness of our method in degrading the accuracy of hidden tasks while
preserving the performance of visible tasks, contributing to the understanding
of adversarial vulnerabilities in multi-task classifiers.
