  As Artificial Intelligence (AI) making advancements in medical
decision-making, there is a growing need to ensure doctors develop appropriate
reliance on AI to avoid adverse outcomes. However, existing methods in enabling
appropriate AI reliance might encounter challenges while being applied in the
medical domain. With this regard, this work employs and provides the validation
of an alternative approach -- majority voting -- to facilitate appropriate
reliance on AI in medical decision-making. This is achieved by a
multi-institutional user study involving 32 medical professionals with various
backgrounds, focusing on the pathology task of visually detecting a pattern,
mitoses, in tumor images. Here, the majority voting process was conducted by
synthesizing decisions under AI assistance from a group of pathology doctors
(pathologists). Two metrics were used to evaluate the appropriateness of AI
reliance: Relative AI Reliance (RAIR) and Relative Self-Reliance (RSR). Results
showed that even with groups of three pathologists, majority-voted decisions
significantly increased both RAIR and RSR -- by approximately 9% and 31%,
respectively -- compared to decisions made by one pathologist collaborating
with AI. This increased appropriateness resulted in better precision and recall
in the detection of mitoses. While our study is centered on pathology, we
believe these insights can be extended to general high-stakes decision-making
processes involving similar visual tasks.
