  The integration of Large Language Models (LLMs) like GPT-4o into robotic
systems represents a significant advancement in embodied artificial
intelligence. These models can process multi-modal prompts, enabling them to
generate more context-aware responses. However, this integration is not without
challenges. One of the primary concerns is the potential security risks
associated with using LLMs in robotic navigation tasks. These tasks require
precise and reliable responses to ensure safe and effective operation.
Multi-modal prompts, while enhancing the robot's understanding, also introduce
complexities that can be exploited maliciously. For instance, adversarial
inputs designed to mislead the model can lead to incorrect or dangerous
navigational decisions. This study investigates the impact of prompt injections
on mobile robot performance in LLM-integrated systems and explores secure
prompt strategies to mitigate these risks. Our findings demonstrate a
substantial overall improvement of approximately 30.8% in both attack detection
and system performance with the implementation of robust defence mechanisms,
highlighting their critical role in enhancing security and reliability in
mission-oriented tasks.
