  In this paper, we focus on methods to reduce the size and improve the quality
of the prompt context required for question-answering systems. Attempts to
increase the number of retrieved chunked documents and thereby enlarge the
context related to the query can significantly complicate the processing and
decrease the performance of a Large Language Model (LLM) when generating
responses to queries. It is well known that a large set of documents retrieved
from a database in response to a query may contain irrelevant information,
which often leads to hallucinations in the resulting answers. Our goal is to
select the most semantically relevant documents, treating the discarded ones as
outliers. We propose and evaluate several methods for identifying outliers by
creating features that utilize the distances of embedding vectors, retrieved
from the vector database, to both the centroid and the query vectors. The
methods were evaluated by comparing the similarities of the retrieved LLM
responses to ground-truth answers obtained using the OpenAI GPT-4o model. It
was found that the greatest improvements were achieved with increasing
complexity of the questions and answers.
