Payton Clem successfully defended her M.S. thesis Autonomous Target Tracking of Hostile Ground Target under Wind Disturbance and Sun Concealment using Deep Reinforcement Learning on 12 December. Payton has been with VSCL since the first semester of her senior year and is highly engaged in AI and flight testing.
Intelligence, surveillance, and reconnaissance (ISR) missions benefit from the use of unmanned aircraft systems (UAS) capable of maintaining visual contact with ground targets, referred to here as target tracking. For practical deployment, it is valuable for tracking to be autonomous and function without detailed knowledge of the surrounding environment. The task becomes more complex when additional objectives, such as concealment or avoiding a hostile target, are introduced. To address this problem, a Soft Actor-Critic (SAC) reinforcement learning controller is developed that uses only the target’s location in the image frame. The agent controls a multirotor UAS equipped with a fixed optical sensor, requiring the agent to adjust vehicle attitude to keep the target in view while accounting for wind, varying target behaviors, altitude-based concealment constraints, and sun-related concealment. Previous work on fixed-camera target tracking has shown that RL-based algorithms can produce unstable behaviors such as control oscillations and large altitude changes. This work focuses on reward shaping to mitigate these issues and encourage stable, consistent tracking. In addition, the influence of including solar concealment information in the reward function is examined to assess its effect on vehicle behavior. The results demonstrate that the proposed reward structure effectively reduces unwanted behaviors such as diving and pitch and yaw ringing. The reward structure enables stable, long-duration tracking, despite the incorporation of constraints associated with sun concealment strategies. The resulting policy achieves reliable tracking across the evaluated conditions.
Payton’s research is supported by the Army Research Laboratory on the project Robust Threat Detection for Ground Combat Vehicles with Multi-Domain Surveillance in Hostile Environments
Approaches for teaching learning agents via human demonstrations have been widely studied and successfully applied to multiple domains. However, the majority of imitation learning work utilizes only behavioral information from the demonstrator, i.e. which actions were taken, and ignores other useful information. In particular, eye gaze information can give valuable insight towards where the demonstrator is allocating visual attention, and holds the potential to improve agent performance and generalization. In this work, we propose Gaze Regularized Imitation Learning (GRIL), a novel context-aware, imitation learning architecture that learns concurrently from both human demonstrations and eye gaze to solve tasks where visual attention provides important context. We apply GRIL to a visual navigation task, in which an unmanned quadrotor is trained to search for and navigate to a target vehicle in a photo-realistic simulated environment. We show that GRIL outperforms several state-of-the-art gaze-based imitation learning algorithms, simultaneously learns to predict human visual attention, and generalizes to scenarios not present in the training data.
) successfully defended his Ph.D. dissertation titled “Multiple-Timescale Adaptive Control for Uncertain Nonlinear Dynamical Systems”. Kameron’s dissertation investigated combining nonlinear multiple time-scale controllers that VSCL has been researching for the last 15 years, with adaptive controllers which VSCL has been researching for more than 20 years. Multiple-timescale control has been shown to have difficulty with uncertain systems and adaptive control has been shown to have difficulty with multiple-timescale systems. His dissertation describes a novel control methodology called [K]Control of Adaptive Multiple-timescale Systems (KAMS). KAMS seeks to address systems that simultaneously exhibit uncertain and multiple-timescale behaviors. Unlike traditional multiple-timescale control literature, KAMS uses adaptive control to stabilize the subsystems. The reference models and adapting parameters used in adaptive control significantly complicate the stability analysis. KAMS is a flexible theory and framework and the stability proofs apply to a wide array of adaptive algorithms and multiple-timescale fusion techniques. Additionally, formal and numerical validation of how KAMS can relax the minimum phase assumption for a multitude of common adaptive control methods. KAMS is demonstrated and evaluated on examples consisting of stabilization and attitude control of a quadrotor Unmanned Air System; fuel-efficient orbital transfer maneuvers; and preventing inlet unstart on hypersonic aircraft.
VSCL student Blake Krpec, who will graduate with his Master of Science degree in May 2022, has defended his thesis “Vision-Based Marker-Less Landing of a UAS On a Moving Ground Vehicle”. Blake’s defense had 34 people in attendance including many in attendance from the Army Research Lab. His committee is Drs. Reza Langari, Manoranjan Majji, Srikanth Saripalli, and Stephen Nogar (special committee member from Army Research Laboratory and the Technical Monitor). His research is supported as a Journeyman Fellow by the