Dr. John Valasek, Professor in the Department of Aerospace Engineering at Texas A&M University and Director of the Vehicle Systems & Control Laboratory, gave a virtual seminar titled “Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time” for the AI Seminar Series hosted by the NASA Jet Propulsion Laboratory (JPL).  The date of the seminar was 18 August 2021.

Cycle-of-Learning (CoL) (https://www.youtube.com/watch?v=AQwsk6kZfok) was presented as a framework using an actor-critic architecture with a loss function that combines behavior cloning and 1-step Q-learning losses with an off-policy pre-training step from human demonstrations.  This enables transition from behavior cloning to reinforcement learning without performance degradation and improves reinforcement learning in terms of overall performance and training time.  This approach is shown to outperform state-of-the-art techniques for combining behavior cloning and reinforcement learning, for both dense and sparse reward scenarios. Results are presented for haptic and eye tracking input modalities, and suggest that directly including the behavior cloning loss on demonstration data helps to ensure stable learning and ground future policy updates.

Kameron Eves, a Ph.D. student in the Vehicle Systems & Control Laboratory (VSCL), has been selected as a 2021-2022 Center for the Integration of Research, Teaching, and Learning (CIRTL) Teaching-as-Research (TAR) Fellow. CIRTL TAR Fellows design and perform an experimental research project about education. Research such as this can help educators adjust their teaching to best help students succeed. Eves’ project proposal seeks to find ways to merge the concepts of active learning and learning styles. These teaching ideas are commonly used in many classrooms, but are seldom used in a complementary way. More specifically, Eves will examine the way question structure can affect participation for students who are usually passive observers in class. The CIRTL TAR Fellowship is competitive. It includes up to $1,000 of funding, and those who publish their work in an academic journal can be awarded a CIRTL Scholar Certificate.

Eves is a second year Aerospace Engineering Ph.D. student with the VSCL. His primary research topic is adaptive control for hypersonic systems. He graduated in 2019 from Brigham Young University in Mechanical Engineering and joined the VSCL immediately after. Eves’ career goals are to join academia and teach at a university. There, he will be able to put into practice many teaching principles such as those to be discovered in this CIRTL TAR project.

VSCL Undergraduate Research Assistant Natalie Warnock graduated with her Bachelor of Science degree in Aerospace Engineering with Summa Cum Laude in May 2021. Warnock has been hired by Bell, Fort Worth, TX.

VSCL Undergraduate Research Assistant Garrett Dorsett graduated with his Bachelor of Science degree in Ocean Engineering in May 2021. Dorsett with be pursuing his Master of Science in Ocean Engineering in Fall 2021.

VSCL is proud to welcome and host Michaela Stratton for the 2021 Undergraduate Summer Research Grant (USRG) Program, where she will be researching Reinforcement Learning for aerospace applications.  Stratton is a rising junior at North Carolina State University currently pursuing a Bachelor of Science degree in Aerospace Engineering. She  maintains an interest in Computer Vision, Robotics, and Autonomous Machines. Her research interest began during her freshman year when she  developed a project that she presented at the National Conference of Undergraduate Research. The Research  involved evaluating the viability of using Computer Vision and Machine learning in order to translate American Sign Language(ASL) in real-time.

VSCL is proud to welcome four new Ph.D. graduate research assistants:

Ian Holmes is a Ph.D. student in the aerospace engineering department, and inaugural recipient of the Department of Aerospace Engineering National EXcellence in Aerospace Sciences (NEXAS) Fellowship.  He graduated in May 2021 from the California State University, Long Beach with a B.S. degree in Aerospace Engineering with an emphasis in Astronautics. During his undergraduate studies, he spent three years participating in the BUILD program as an NIH-funded research assistant focused on experimental and computational high-speed impact testing. He completed his engineering honors thesis on developing computational strategies using smoothed particle hydrodynamics to simulate bird strike impacts in commercial aviation.

 

 

 

 

MD-Nazmus Sunbeam is a Ph.D. student in the aerospace engineering department. He will graduate in May 2021 from the University of Texas at Austin with a B.S. degree in aerospace engineering. During his undergraduate studies, Sunbeam worked on making a chess AI by training neural nets through evolutionary algorithms using NEAT. Additionally, he assembled and configured quadcopters, using the drone footage to train object detection neural nets. He has experience implementing convolutional neural nets on different image classification and speech recognition problems. His research interests are AI/ML/robotics. In the fall at VSCL, Sunbeam will research Enhancing the Cycle-of-Learning for Autonomous Systems to Facilitate Human-Agent Teaming, which is sponsored by the Army Research Laboratory.

 

 

 

 

Ravi Kumar Thakur is a Ph.D. student in the aerospace engineering department.  His interest is in the field of aerospace robotics and autonomy. He graduated with an MS(Research) degree in Electronics and Communication from the Indian Institute of Information Technology Sri City, Chittoor in 2019. For his thesis, he worked on developing machine learning-based models for estimating scene flow from stereo images. He earned his BS in Engineering Physics from National Institute of Technology Calicut in 2014. In the past, he was a machine learning engineer at Ford Motor Company,  where he worked on driver assistant technology with a focus on visual odometry and object tracking. Before that, he worked at the Indian Institute of science working on the development of an endoscopy simulator. At VSCL, Thakur will be working on the project Enhancing the Cycle-of-Learning for Autonomous Systems to Facilitate Human-Agent Teaming which is sponsored by Army Research Laboratory.

 

 

 

David van Wijk is a Ph.D. student in the aerospace engineering department, funded by the College of Engineering Graduate Merit Fellowship. He will graduate in May of 2021 from Cornell University with a B.S. in Mechanical Engineering from the Sibley school of Mechanical and Aerospace Engineering. During his undergraduate studies, van Wijk was an active member in the Laboratory for Intelligent Systems and Controls under Professor Ferrari, concentrating on classical control for quadrotors, object detection, and mapping. His research contributed to a paper on Visibility-based Directional Sensor Path Planning submitted to IEEE Transactions on Robotics. He also interned at Northrop Grumman Remotec, focusing on mechanical design of hazardous duty robotic vehicles. He is interested in robotics, autonomous systems, and machine learning.