Human – AI Teaming to Fight Wildfires
This post launches a blog series that zeroes in on what AI Redefined is approaching in a different way, mainly through our orchestration framework Cogment. The series will cover everything from our use of simulations and interfaces to the versatility that Cogment’s modular structure supports. We’ll also discuss how those elements relate to our larger vision of human and AI collaboration, human augmentation, and bridging the gaps that characterize modern AI. Today’s post focuses on one of the most distinctive features of Cogment: its ability to offer continuous human training alongside machine agents in real and simulated contexts.
The idea of a human and a machine behaving as a cohesive, smart team to solve complex tasks is no small objective. When the context is time-sensitive and involves destructive and life threatening situations, with unforeseen dynamics in a constantly evolving environment, human-machine teaming becomes even more challenging. That’s where Cogment can help. In this post, we discuss why fighting wildfires is a perfect use case for Cogment’s continuous training of multiple autonomous agents alongside a human.
The context
Wildfires are notoriously difficult to fight. In California, for example, intense winds, difficult terrain, and droughts are among the critical elements that hinder the ability of first responders to get wildfires under control in a timely manner. We set out to think about how machine learning AI agents can support the already available arsenal against those wildfires. The use of coordinated autonomous drones trained to work alongside human first responders can help firefighting efforts both on land and in the air.
Unmanned aerial vehicles (UAVs) are already being used to support such efforts, helping detect, contain, and even extinguish fires. UAVs are able to fly low and for sustained periods of time, while helicopters or other manned aircrafts, thanks to their maneuverability and the need to protect the safety of their pilots, cannot. Furthermore, UAVs can provide real-time information about fires, detect people through smoke with precision, and measure wind speeds and directions. However, these UAVs are remotely piloted, and managing a 24/7 fleet of drones over any huge forested area poses many challenges.
Our approach
Most AI solutions work independently from human action. We believe that training AI on specific independent tasks is good, but insufficient. Cogment offers a better way. In a wildfire context, Cogment can help train AI alongside expert first responders so that it complements their actions and is able to support them in the field in real time. Cogment’s approach allows the formation of hybrid teams of AI agents (drones) and first responders (human) that can:
better coordinate efforts
give first responders more information
guide deployments and air support as the situation develops
prevent as much as possible the unwanted surprises of new fire seats or changing dynamics
minimize the risk to human first responders.
To show how this approach would work in practice, we built a rough Proof-of-Concept demo that shows simulated forest fires being tackled by a team made of several autonomous drones working alongside a human firefighting helicopter pilot. Not only could these tools and agents be used during actual fire responses, but they could also serve as a training tool for junior first responders. The Proof of Concept, which we call “Flames”, leaves the controls of a firefighting helicopter to a human pilot while a team of autonomous drones scout around to provide more information on the area. This technique can help mitigate the low visibility usually associated with smoke-generating fires as well as monitor found fire seats. The helicopter can drop water and fire retardant on the fire seats, while the drones actively survey the area and remain around fire seats they find to provide a real-time view of their evolution.
On this simple overview, where green cells show forests, red cells active fires, and brown cells doused fires, we can see one drone in the lower right quadrant circling a fire seat it found until the helicopter pilot arrives and drops the water and fire retardant.
The training environment
Our training environment, which is shared by the human piloting the helicopter and the support drones, simulates fire propagation as well as direct visibility. In the above image, these elements are indicated by the lighter circles around the moving object (showing areas that are directly visible), versus the darker view (which shows the last known state of a previously surveyed area) and the obfuscated ones (indicating areas that haven’t been surveyed yet).
The Machine Learning techniques
The Flames environment has two actors’ classes in them, the helicopter and the drones. While the helicopter, piloted by a human, is tasked with dousing fires, the drones, piloted by an AI agent, are mandated to discover seats of fire in the forest. The observation space of the AI agent consists of the locations of all the drones in the environment’s reference frame and a snapshot of the environment that is partially observable due to smoke from the forest fires. The action space of the drones is discrete, and allows them to move either east, west, north, south, north-east, south-east, north-west, or south-west, with respect to their current location in the environment’s reference frame. The agent is rewarded for discovering new fires in the forest and penalized for wasting time. The human operator, on the other hand, can interact with the helicopter and the drones via the web client and gets to see the same state of the forest and fires on the client as the drones.
We trained the AI agent in phases, the goal of which was to compare the success of the mission, as measured by the percentage of forest saved, with and without human-AI teaming.
Phase 1 consisted of only humans piloting both the drones as well as the helicopter to save the forest from fires. The results of this human-only phase acted as a baseline against which the results of AI-only and human-AI experiments were compared.
Phase 2 involved training the AI agent from scratch using reinforcement learning, without human inputs. While the helicopter was piloted by a scripted non-learning logic, the AI agent piloting the drones was trained with ‘Centralized training and decentralized execution approach’ using a Value-Decomposition Network (VDN).
Phase 3 aimed at teaming the human-piloted helicopter with the functional-AI-piloted drones from phase 2 to save the forest. The human could not only pilot the helicopter in the environment to douse fires, he/she could also order the drones in the desired direction if the human deemed that the drones were exploring for fires sub-optimally. The data generated from the humans was used to continually update the AI agent’s policy through policy aggregation.
After our initial 2D environment visualization, we also developed a 3D one, using the Unity engine. Switching from the initial web-based representation to this one was easy, as Cogment is designed to be tech-agnostic.
Going further
Although we focused on firefighting in this PoC, many other abilities could be bestowed on those autonomous agents; they could, for example, detect public drones that people are flying in operation areas despite the danger they pose to manned aircrafts, creating unsafe conditions for firefighters. Taking into account wind direction and speed, these agents could also provide recommendations to aircraft pilots to optimize how their firefighting payload is delivered, either through overlaid HUDs or by acting as “pilot fishes,” guiding aircrafts and signalling optimal release trajectories and timings directly in visual range of the pilots.
Going beyond our Proof-of-Concept demo into the building of a real project would require a number of back and forth chats with experts of the field to refine our understanding of current protocols, tools, issues, strategies and tactics used in fighting wildfires. Even though numerous challenges would have to be tackled to create a deployable system, we believe that real-world situations as critical as wildfires need the kind of contextual human-machine teaming that Cogment can provide.
