  Large language models (LLMs) have shown amazing capabilities in knowledge
memorization and the present. However, when it comes to domain-specific
knowledge and downstream tasks like medical, general LLMs are often unable to
give precise answers. In addition, when people want LLMs to answer
classification questions, they usually go through instruction tuning first.
However, LLMs do not always give a direct index of the categorization after
instruction tuning. In this paper, we proposed LlamaCare, a fine-tuned medical
language model, and Extended Classification Integration(ECI), a module to
handle classification problems of LLMs. Our contributions are : (i) We
fine-tuned a large language model of medical knowledge with very low carbon
emissions and achieved similar performance with ChatGPT by a 24G GPU. (ii) We
solved the problem of redundant categorical answers and improved the
performance of LLMs by proposing a new module called Extended Classification
Integration. (iii) We released our processed data for one-shot and few-shot
training for some benchmarks such as PubMedQA and USMLE 1-3 step. Our method
achieves a close performance comparable to some state-of-the-art models with
the same quantity of parameters on benchmarks, while being more environmentally
friendly by using less GPU computation time. Our models, codes, and datasets
can be found at \url{https://github.com/Stephen-SMJ/LLamaCare}.
