  Major search engine providers are rapidly incorporating Large Language Model
(LLM)-generated content in response to user queries. These conversational
search engines operate by loading retrieved website text into the LLM context
for summarization and interpretation. Recent research demonstrates that LLMs
are highly vulnerable to jailbreaking and prompt injection attacks, which
disrupt the safety and quality goals of LLMs using adversarial strings. This
work investigates the impact of prompt injections on the ranking order of
sources referenced by conversational search engines. To this end, we introduce
a focused dataset of real-world consumer product websites and formalize
conversational search ranking as an adversarial problem. Experimentally, we
analyze conversational search rankings in the absence of adversarial injections
and show that different LLMs vary significantly in prioritizing product name,
document content, and context position. We then present a tree-of-attacks-based
jailbreaking technique which reliably promotes low-ranked products.
Importantly, these attacks transfer effectively to state-of-the-art
conversational search engines such as perplexity.ai. Given the strong financial
incentive for website owners to boost their search ranking, we argue that our
problem formulation is of critical importance for future robustness work.
