Army Research Laboratory
Co-Principal Investigator
19 March 2019 – 18 March 2023
Total award $2,499,458
This project investigates Complex Battlefield operations through Vehicle Automation, Coordination, Multi-Domain Surveillance, and Intelligent Decision Support Systems. In complex, emergent scenarios of heterogeneous ground and aerial vehicles performing coordinated maneuvers to achieve common goals as a team, each of the vehicles will normally have inherently significant perception, autonomy and intelligence, and thus the scenarios that we are addressing are different from typical swarm robotics, where the individual elements of the swarm are normally simpler while trying to cooperatively achieve a larger purpose. This research will develop a decision support system that enables the commander of the operations to effectively coordinate their fleet of vehicles towards successful completion of various missions.
A critical requirement for effective coordination is rich, accurate, and shared situational awareness. This is achieved by using a variety of sensors on both air and ground vehicles and fusing the information. This is often denoted as Multi-Domain Surveillance (MDS). Vision-based cameras are popular sensors for MDS because of their cost, their ability to observe passively, and the richness of information they can provide. However, using an MDS with vision based cameras to support still poses significant fundamental challenges that we strive to address in this research. While significant advances have been made in vision processing through both image processing as well as through the use of machine learning algorithms, many of the results show promising results only under “nominal” environmental conditions. In degraded visual environments (DVEs), such as in low lighting, fog, rain, etc., the vision algorithms perform poorly, significantly reducing the effectiveness of an MDS.
The Key Challenges addressed in this project are:
Challenge 1: Robust Vision Algorithms in the presence of Degraded Visual Environments
Challenge 2: Effective MDS through optimal location of sensors (and aerial vehicles with sensors)
Challenge 3: Collation and Abstraction of information from diverse sources with varying levels of fidelity and timeliness to generate a coherent and succinct “Situational Awareness”
Challenge 4: Use of semantic situational awareness to identify suspicious activities and malicious threats