  Deep learning models have been incorporated into high-stakes sectors,
including healthcare diagnosis, loan approvals, and candidate recruitment,
among others. Consequently, any bias or unfairness in these models can harm
those who depend on such models. In response, many algorithms have emerged to
ensure fairness in deep learning. However, while the potential for harm is
substantial, the resilience of these fair deep learning models against
malicious attacks has never been thoroughly explored, especially in the context
of emerging Trojan attacks. Moving beyond prior research, we aim to fill this
void by introducing \textit{TrojFair}, a Trojan fairness attack. Unlike
existing attacks, TrojFair is model-agnostic and crafts a Trojaned model that
functions accurately and equitably for clean inputs. However, it displays
discriminatory behaviors \text{-} producing both incorrect and unfair results
\text{-} for specific groups with tainted inputs containing a trigger. TrojFair
is a stealthy Fairness attack that is resilient to existing model fairness
audition detectors since the model for clean inputs is fair. TrojFair achieves
a target group attack success rate exceeding $88.77\%$, with an average
accuracy loss less than $0.44\%$. It also maintains a high discriminative score
between the target and non-target groups across various datasets and models.
