Payton Clem successfully defended her M.S. thesis Autonomous Target Tracking of Hostile Ground Target under Wind Disturbance and Sun Concealment using Deep Reinforcement Learning on 12 December. Payton has been with VSCL since the first semester of her senior year and is highly engaged in AI and flight testing.
Intelligence, surveillance, and reconnaissance (ISR) missions benefit from the use of unmanned aircraft systems (UAS) capable of maintaining visual contact with ground targets, referred to here as target tracking. For practical deployment, it is valuable for tracking to be autonomous and function without detailed knowledge of the surrounding environment. The task becomes more complex when additional objectives, such as concealment or avoiding a hostile target, are introduced. To address this problem, a Soft Actor-Critic (SAC) reinforcement learning controller is developed that uses only the target’s location in the image frame. The agent controls a multirotor UAS equipped with a fixed optical sensor, requiring the agent to adjust vehicle attitude to keep the target in view while accounting for wind, varying target behaviors, altitude-based concealment constraints, and sun-related concealment. Previous work on fixed-camera target tracking has shown that RL-based algorithms can produce unstable behaviors such as control oscillations and large altitude changes. This work focuses on reward shaping to mitigate these issues and encourage stable, consistent tracking. In addition, the influence of including solar concealment information in the reward function is examined to assess its effect on vehicle behavior. The results demonstrate that the proposed reward structure effectively reduces unwanted behaviors such as diving and pitch and yaw ringing. The reward structure enables stable, long-duration tracking, despite the incorporation of constraints associated with sun concealment strategies. The resulting policy achieves reliable tracking across the evaluated conditions.
Payton’s research is supported by the Army Research Laboratory on the project Robust Threat Detection for Ground Combat Vehicles with Multi-Domain Surveillance in Hostile Environments
