  Large Language Models (LLMs) have become the go-to solution for many Natural
Language Processing (NLP) tasks due to their ability to tackle various problems
and produce high-quality results. Specifically, they are increasingly used to
automatically generate code, easing the burden on developers by handling
repetitive tasks. However, this improvement in quality has led to high
computational and memory demands, making LLMs inaccessible to users with
limited resources. In this paper, we focus on Central Processing Unit
(CPU)-compatible models and conduct a thorough semi-manual evaluation of their
strengths and weaknesses in generating Python code. We enhance their
performance by introducing a Chain-of-Thought prompt that guides the model in
problem-solving. Additionally, we propose a dataset of 60 programming problems
with varying difficulty levels for evaluation purposes. Our assessment also
includes testing these models on two state-of-the-art datasets: HumanEval and
EvalPlus. We commit to sharing our dataset and experimental results publicly to
ensure transparency.
