  Panoramic image segmentation in computational pathology presents a remarkable
challenge due to the morphologically complex and variably scaled anatomy. For
instance, the intricate organization in kidney pathology spans multiple layers,
from regions like the cortex and medulla to functional units such as glomeruli,
tubules, and vessels, down to various cell types. In this paper, we propose a
novel Hierarchical Adaptive Taxonomy Segmentation (HATs) method, which is
designed to thoroughly segment panoramic views of kidney structures by
leveraging detailed anatomical insights. Our approach entails (1) the
innovative HATs technique which translates spatial relationships among 15
distinct object classes into a versatile "plug-and-play" loss function that
spans across regions, functional units, and cells, (2) the incorporation of
anatomical hierarchies and scale considerations into a unified simple matrix
representation for all panoramic entities, (3) the adoption of the latest AI
foundation model (EfficientSAM) as a feature extraction tool to boost the
model's adaptability, yet eliminating the need for manual prompt generation in
conventional segment anything model (SAM). Experimental findings demonstrate
that the HATs method offers an efficient and effective strategy for integrating
clinical insights and imaging precedents into a unified segmentation model
across more than 15 categories. The official implementation is publicly
available at https://github.com/hrlblab/HATs.
