  Search engines have traditionally served as primary tools for information
seeking. However, the new Large Language Models (LLMs) have recently
demonstrated remarkable capabilities in multiple tasks and, specifically, their
adoption as question answering systems is becoming increasingly prevalent. It
is expected that LLM-based conversational systems and traditional web engines
will continue to coexist in the future, supporting end users in various ways.
But there is a need for more scientific research on the effectiveness of both
types of systems in facilitating accurate information seeking. In this study,
we focus on their merits in answering health questions. We conducted an
extensive study comparing different web search engines, LLMs and
retrieval-augmented (RAG) approaches. Our research reveals intriguing
conclusions. For example, we observed that the quality of webpages potentially
responding to a health question does not decline as we navigate further down
the ranked lists. However, according to our evaluation, web engines are less
accurate than LLMs in finding correct answers to health questions. On the other
hand, LLMs are quite sensitive to the input prompts, and we also found out that
RAG leads to highly effective information seeking methods.
