  Adopting Convolutional Neural Networks (CNNs) in the daily routine of primary
diagnosis requires not only near-perfect precision, but also a sufficient
degree of generalization to data acquisition shifts and transparency. Existing
CNN models act as black boxes, not ensuring to the physicians that important
diagnostic features are used by the model. Building on top of successfully
existing techniques such as multi-task learning, domain adversarial training
and concept-based interpretability, this paper addresses the challenge of
introducing diagnostic factors in the training objectives. Here we show that
our architecture, by learning end-to-end an uncertainty-based weighting
combination of multi-task and adversarial losses, is encouraged to focus on
pathology features such as density and pleomorphism of nuclei, e.g. variations
in size and appearance, while discarding misleading features such as staining
differences. Our results on breast lymph node tissue show significantly
improved generalization in the detection of tumorous tissue, with best average
AUC 0.89 (0.01) against the baseline AUC 0.86 (0.005). By applying the
interpretability technique of linearly probing intermediate representations, we
also demonstrate that interpretable pathology features such as nuclei density
are learned by the proposed CNN architecture, confirming the increased
transparency of this model. This result is a starting point towards building
interpretable multi-task architectures that are robust to data heterogeneity.
Our code is available at https://github.com/maragraziani/multitask_adversarial
