  Deep Reinforcement Learning (DRL) is hugely successful due to the
availability of realistic simulated environments. However, performance
degradation during simulation to real-world transfer still remains a
challenging problem for the policies trained in simulated environments. To
close this sim-to-real gap, we present a novel hybrid architecture that
utilizes an intermediate output from a fully trained attention DRL policy as a
navigation cost map for outdoor navigation. Our attention DRL network
incorporates a robot-centric elevation map, IMU data, the robot's pose,
previous actions, and goal information as inputs to compute a navigation
cost-map that highlights non-traversable regions. We compute least-cost
waypoints on the cost map and utilize the Dynamic Window Approach (DWA) with
velocity constraints on high cost regions to follow the waypoints in highly
uneven outdoor environments. Our formulation generates dynamically feasible
velocities along stable, traversable regions to reach the robot's goals. We
observe an increase of 5% in terms of success rate, 13.09% of the decrease in
average robot vibration, and a 19.33% reduction in average velocity compared to
end-to-end DRL method and state-of-the-art methods in complex outdoor
environments. We evaluate the benefits of our method using a Clearpath Husky
robot in both simulated and real-world uneven environments.
