  Recent advancements in language models (LMs) have led to the emergence of
powerful models such as Small LMs (e.g., T5) and Large LMs (e.g., GPT-4). These
models have demonstrated exceptional capabilities across a wide range of tasks,
such as name entity recognition (NER) in the general domain. (We define SLMs as
pre-trained models with fewer parameters compared to models like GPT-3/3.5/4,
such as T5, BERT, and others.) Nevertheless, their efficacy in the medical
section remains uncertain and the performance of medical NER always needs high
accuracy because of the particularity of the field. This paper aims to provide
a thorough investigation to compare the performance of LMs in medical few-shot
NER and answer How far is LMs from 100\% Few-shot NER in Medical Domain, and
moreover to explore an effective entity recognizer to help improve the NER
performance. Based on our extensive experiments conducted on 16 NER models
spanning from 2018 to 2023, our findings clearly indicate that LLMs outperform
SLMs in few-shot medical NER tasks, given the presence of suitable examples and
appropriate logical frameworks. Despite the overall superiority of LLMs in
few-shot medical NER tasks, it is important to note that they still encounter
some challenges, such as misidentification, wrong template prediction, etc.
Building on previous findings, we introduce a simple and effective method
called \textsc{RT} (Retrieving and Thinking), which serves as retrievers,
finding relevant examples, and as thinkers, employing a step-by-step reasoning
process. Experimental results show that our proposed \textsc{RT} framework
significantly outperforms the strong open baselines on the two open medical
benchmark datasets
