AI and Autonomy

The following are a list of papers regarding the autonomous control and Artificial Intelligence work that has been completed by VSCL and its members. Selected papers have a pdf linked at the end of the citation.

Books

Advances in Computational Intelligence and Autonomy for Aerospace Systems, 2018, Editor: John Valasek

• Chapter: Famularo, Douglas, Whitney, Sean G., Valasek, John, Muse, Jonathan A., Bolender, Michael A., “Handling Inlet Unstart in Hypersonic Vehicles Using Nonlinear Dynamic Inversion Adaptive Control with State Constraints”

Advances in Intelligent and Autonomous Aerospace Systems, 2012, Editor: John Valasek

• Chapter: Lampton, Amanda, and Valasek, John, “Multi-Resolution State-Space Discretization Method for Q-Learning for One or More Regions of Interest”

Reinforcement Learning and Approximate Dynamic Programming for Feedback Control, 2012, Editor: Lewis, Frank L., and Liu, Derong (Eds.)

• Chapter: Kirkpatrick, Kenton, and Valasek, John, “Reinforcement Learning Control With Time Dependent Agent Dynamics”

Journal Papers

• Krpec, Blake, Valasek, John, and Nogar, Stephen, “Vision-based Marker-less Landing of a UAS on Moving Ground Vehicle,” Journal of Aerospace Information Systems, Volume 29, Number 4, September 2024, pages 735-750.  Doi.org/10.2514/1.I011282
• Lehman, Hannah, and Valasek, John, “Design, Selection, Evaluation of Reinforcement Learning Single Agents for Ground Target Tracking” Journal of Aerospace Information Systems, Volume 2, Number 2, February 2024, pages 198-215.  Doi.org/10.2514/1.I011284
• Valasek, John, Kirkpatrick, Kenton, May, James, and Harris, Joshua, “Intelligent Motion Video Guidance for Unmanned Air System Ground Target Surveillance,” Journal of Aerospace Information Systems, Volume 13, Number 1, January 2016, pp. 10-26Paper: Intelligent Motion Video Guidance for Unmanned Air System Ground Target Surveillance
• Kirkpatrick, Kenton, Valasek, John, and Haag, Chris, “Characterization and Control of Hysteretic Dynamics Using Online Reinforcement Learning,” Journal of Aerospace Information Systems, Volume 10, Number 6, June 2013, pp. 297-305.Paper: Characterization and Control of Hysteretic Dynamics Using Online Reinforcement Learning
• Kirkpatrick, Kenton, and Valasek, John, “Active Length Control of Shape Memory Alloy Wires via Reinforcement Learning,” Journal of Intelligent Material Systems and Structures, Volume 22, Issue 14, September 2011, pp. 1595 – 1604.Paper: Active Length Control of Shape Memory Alloy Wires via Reinforcement Learning
• Lampton, Amanda, and Valasek, John, “Multi-Resolution State-Space Discretization for Q-Learning with Pseudo-Randomized Discretization,” Journal of Control Theory and Applications, Volume 9, Number 3, 2011, pp. 431-439.Paper: Multi-Resolution State-Space Discretization for Q-Learning with Pseudo-Randomized Discretization
• Lampton, Amanda, Niksch, Adam, and Valasek, John, “Reinforcement Learning of a Morphing Airfoil- Policy and Discrete Learning Analysis,” Journal of Aerospace Computing, Information, and Communication, Volume 7, Number 8, August 2010, pp. 241-260.Paper: Reinforcement Learning of a Morphing Airfoil- Policy and Discrete Learning Analysis
• Kirkpatrick, Kenton, and Valasek, John, “Reinforcement Learning for Characterization of Hysteresis Behavior in Shape Memory Alloys,” Journal of Aerospace Computing, Information, and Communication, Volume 6, Number 3, March 2009, pp. 227-238.Paper: Reinforcement Learning for Characterization of Hysteresis Behavior in Shape Memory Alloys
• Lampton, Amanda, Niksch, Adam, and Valasek, John, “Reinforcement Learning of Morphing Airfoils with Aerodynamic and Structural Effects,” Journal of Aerospace Computing, Information, and Communication, Volume 6, Number 1, January 2009, pp. 30-50. Paper: Reinforcement Learning of Morphing Airfoils with Aerodynamic and Structural Effects
•  Valasek, John, Tandale, Monish D., and Rong, Jie, “A Reinforcement Learning – Adaptive Control Architecture for Morphing,” Journal of Aerospace Computing, Information, and Communication, Volume 2, Number 4, April 2005, pp. 174-195.Paper: A Reinforcement Learning – Adaptive Control Architecture for Morphing

Conference Papers

• Thakur, Ravi Kumar, Sunbeam, MD, Goecks, Vinicius G., Novoseller, Ellen, Lawhern, Vernon, Gremillion, Greg, Valasek, John, and Waytowich, Nicholas R., “Imitation Learning with Human Eye Gaze via Multi-Objective Prediction,” Interactive Learning with Implicit Human Feedback Workshop at Fortieth International Conference on Machine Learning, Honolulu, HI, 23 July 2023.
• Goecks, Vinicius, G., Gremillion, Gregory M., Lawhern, Vernon J., Valasek, John, and Waytowich, Nicholas, “Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments,” International Conference on Autonomous Agents and Multi-Agent Systems 2020 (AAMAS20), Auckland, New Zealand, 9-13 May 2020.  doi.org/10.48550/arXiv.1910.04281.
• Bera, Ritwik, Goecks, Vinicius G., Gremillion, Gregory M., Valasek, John, and Waytowich, Nicholas R., “PODNet: A Neural Network for Discovery of Plannable Options,” Combining Machine Learning and Knowledge Engineering in Practice (AAAI-MAKE) 2020, Palo Alto, CA, 23-25 March 2020.  doi.org/10.48550/arXiv.1911.00171
• Goecks, V.G., Gremillion, G.M., Lawhern, V.J., Valasek, John, and Waytowich, N.R., “Efficiently Combining Human Demonstrations and Interventions for Safe Training of Autonomous Systems in Real-Time,” 6613, Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), Honolulu, HI, 27 January 2019.  Doi.org/10.48550/arXiv.1810.11545
• Santos, Raul, Binz, Sadie, McQuinn,Cassie-Kay, Valasek, John, Hamilton, Nathaniel, Hobbs, Kerianne L., and Dulap, Kyle, ”Deep Reinforcement Learning Waypoint Generation for Attitude Station-Keeping with Sun Avoidance,” AIAA 2026-0347, 2026 AIAA Science and Technology Forum and Exposition, Orlando, FL, 12 January 2026.
• Santos, R., Binz, S., McQuinn, C., Valasek, J., Hamilton, N., Hobbs, K., Dunlap, K., “Deep Reinforcement Learning Waypoint Generator,” MIT Lincoln Laboratory AI for Contested Space Workshop, Lexington, MA, September 2025.
• Clem, Payton, and Valasek, John, “Deep Reinforcement Learning for Autonomous Target Tracking of Hostile Ground Targets in Wind Disturbed Environments with Sun Concealment,” AIAA 2025-2280, 2025 AIAA Science and Technology Forum and Exposition, Orlando, FL, 9 January 2025 https://doi.org/10.2514/6.2025-2280
• Johnson, Seth, Escamilla, Laura, Lehman, Hannah, and Valasek, John, “Reinforcement Learning Environment to Realistic Simulations for Multi-Agent System Validation and Deployment,” AIAA 2025-1542, 2025 AIAA Science and Technology Forum and Exposition, Orlando, FL, 8 January 2025 https://doi.org/10.2514/6.2025-1542
• Lehman, Hannah, and Valasek, John, “Hierarchical Learned Auctions for Communication-Sparse Satellite Environments,” AIAA 2025-0560, 2025 AIAA Science and Technology Forum and Exposition, Orlando, FL, 6 January 2025 https://doi.org/10.2514/6.2025-0560
• Lehman, Hannah, and Valasek, John, “Machine Learning Across Different Levels of Auction Based Coordination Hierarchies,” AIAA 2024-1201, 2024 AIAA Science and Technology Forum and Exposition, Orlando, FL, 9 January 2024  doi.org/10.2514/6.2024-1201
• Van Wijk, David, Eves, Kameron, and Valasek, John, “Deep Reinforcement Learning Controller for Autonomous Tracking of Evasive Ground Target,” AIAA 2023-0128, 2023 AIAA Science and Technology Forum and Exposition, National Harbor, MD, 23 January 2023.doi.org/10.2514/6.2023-0128
• Lehman, Hannah, Hackett, Shelby, and Valasek, John, “Addressing Undesirable Emergent Behavior in Deep Reinforcement Learning UAS Ground Target Tracking,” AIAA-2022-2544, 2022 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA Science and Technology Forum and Exposition, San Diego, CA, 7 January 2022.  Doi.org/10.2514/6.2022-2544
• Goecks, Vinicius, G., Woods, Grayson, Das, Niladri, and Valasek, John, “Combining Visible and Infrared Spectrum Imagery Using Machine Learning for Small Unmanned Aerial System Detection,” Automatic Target Recognition XXX, SPIE Defense + Commercial Sensing 2020, Anaheim, CA, 28 April 2020.  Doi.org/10.48550/arXiv.2003.12638
• Goecks, Vinicius, and Valasek, John, “Deep Reinforcement Learning on Intelligent Motion Video Guidance for Unmanned Air System Ground Target Tracking,”AIAA-2019-0137, 2019 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA Science and Technology Forum and Exposition, San Diego, CA, 7 January 2019. doi.org/10.2514/6.2019-0137
• Lehman, Hannah, and Valasek, John, “Application of Computational Intelligence for Command & Control of Unmanned Air Systems,” AIAA-2019-0158, 2019 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA Science and Technology Forum and Exposition, San Diego, CA, 7 January 2019. doi.org/10.2514/6.2019-0158
• Valasek, John, Lehman, Hannah, and Goecks, Vinicius, “Online Intelligent Motion Video Guidance for Unmanned Air System Ground Target Surveillance,” AIAA-2019-0135, 2019 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA Science and Technology Forum and Exposition, San Diego, CA, 7 January 2019. Doi.org/10.2514/6.2019-0135
• Goecks, Vinicius, Leal, Pedro, Valasek, John, and Hartl, Darren, “Control of Morphing Wing Shapes with Deep Reinforcement Learning,” AIAA-2018-2139, 2018 AIAA Information Systems-AIAA Infotech @ Aerospace, AIAA Science and Technology Forum and Exposition, Kissimmee, FL, 12 January 2018. doi.org/10.2514/6.2018-2139
• Noren, Charles,  Andersen, Kendra, Gakhar, Kanika, Olinger, Angela, Palchuru, Preetam, Tran, Scott,  and Valasek, John, “Small Team Agile Systems Engineering For Rapid Prototyping of Robotic Systems,” AC2017-20374, 124th ASEE Annual Conference & Exposition, Columbus, OH, 25 June 2017.
• Woodbury, Timothy, Dunn, Caroline, and Valasek, John, “Autonomous Soaring Using Reinforcement Learning for Trajectory Generation,” AIAA-2014-0990, Proceedings of the AIAA Science and Technology Forum and Exposition 2014: 52nd Aerospace Sciences Meeting, National Harbor, MD, 15 January 2014. https://doi.org/10.2514/6.2014-0990
• Henrickson, James, Kirkpatrick, Kenton, and Valasek, John, “Rapid Characterization of Shape Memory Alloy Material Parameters Using Computational Intelligence Methods,” SMASIS2013-3016, Proceedings of the 15th ASME 2013 Conference on Smart Materials, Adaptive Structures and Intelligent Systems, Snowbird, UT, 16 September 2013.
• Kirkpatrick, Kenton, and Valasek, John, “Approximation of Agent Dynamics Using Reinforcement Learning,” AIAA 2013-0875, Proceedings of the 51st AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, Grapevine, TX, 7 January 2013.
• Henrickson, James, Kirkpatrick, Kenton, and Valasek, John, “Characterization of Shape Memory Alloys Using Artificial Neural Networks,” AIAA 2013-0129, Proceedings of the 51st AIAA Aerospace Sciences Meeting Including the New Horizons Forum and Aerospace Exposition, Grapevine, TX, 7 January 2013.
• Valasek, John, Kirkpatrick, Kenton, and May, James, “Intelligent Motion Video Guidance for Unmanned Air System Ground Target Surveillance ,” Proceedings of the 2012 AIAA Infotech@Aerospace Conference, Garden Grove, CA 21 June 2012.
• Lampton, Amanda, and Valasek, John, “Multiresolution State-Space Discretization Method for Q-Learning with Function Approximation and Policy Iteration,” 00944, Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics, San Antonio, TX, 11 October 2009.
• Kirkpatrick, Kenton, Valasek, John, and Lagoudas, Dimitris C., “Active Length Control of Shape Memory Alloy Wires via Reinforcement Learning,” SMASIS2009-1430, Proceedings of the ASME 2009 Conference on Smart Materials, Adaptive Structures and Intelligent Systems, Oxnard, CA, 21 September 2009.
• Lampton, Amanda, and Valasek, John, “Multiresolution State-Space Discretization Method for Q-Learning,”  Proceedings of the 2009 American Control Conference, St.  Louis, MO, 9-12 June 2009.
• Kirkpatrick, Kenton, and Valasek, John, “Reinforcement Learning for Determining Temperature/Strain Behavior of Shape Memory Alloys,” AIAA-2009-204, Proceedings of the 47th AIAA Aerospace Sciences Meeting including The New Horizons Forum and Aerospace Exposition, Orlando, Florida, 5 January 2009.
• Davis, Jeremy J., Doebbler, James, Junkins, John L., and Valasek, John, “Mobile Robotic System for Ground- Testing of Multi- Spacecraft Proximity Operations,” AIAA-2008-6548, Proceedings of the AIAA Modeling and Simulation Technologies Conference, Honolulu, HI, 18 August 2008.
• Tandale, Monish, Valasek, John, Doebbler, James, and Meade, Andrew J., “Improved Adaptive-Reinforcement Learning Control for Morphing Unmanned AirVehicles,”AIAA-2005-7159, Infotech@Aerospace, Arlington, VA, 26-29 September 2005.