  Vision-driven autonomous flight and obstacle avoidance of Unmanned Aerial
Vehicles (UAVs) along complex riverine environments for tasks like rescue and
surveillance requires a robust control policy, which is yet difficult to obtain
due to the shortage of trainable riverine environment simulators. To easily
verify the vision-based navigation controller performance for the river
following task before real-world deployment, we developed a trainable
photo-realistic dynamics-free riverine simulation environment using Unity. In
this paper, we address the shortcomings that vanilla Reinforcement Learning
(RL) algorithm encounters in learning a navigation policy within this partially
observable, non-Markovian environment. We propose a synergistic approach that
integrates RL and Imitation Learning (IL). Initially, an IL expert is trained
on manually collected demonstrations, which then guides the RL policy training
process. Concurrently, experiences generated by the RL agent are utilized to
re-train the IL expert, enhancing its ability to generalize to unseen data. By
leveraging the strengths of both RL and IL, this framework achieves a faster
convergence rate and higher performance compared to pure RL, pure IL, and RL
combined with static IL algorithms. The results validate the efficacy of the
proposed method in terms of both task completion and efficiency. The code and
trainable environments are available.
