NSF-Backed Research Builds Safer AI Systems for Life-Saving Missions

Over the next five years, she will work on developing a set of standard algorithms and benchmarks to ensure that MARL-driven AI systems continue to function correctly, even if one of the systems, or "agents," fails or is compromised in a deliberate attack.

Potential Real-World Applications

In MARL, numerous AI agents collaborate to achieve a greater outcome. That could support optimal health for a diabetic patient with heart disease by running a pacemaker and an insulin pump, directing a fleet of self-driving cars to transport passengers to destinations without calamity, or coordinating a group of drones on a search-and-rescue mission.

However, Alqahtani said no standard exists for testing the safety and security of such MARL-driven AI systems, only a hodgepodge of de facto guidelines that primarily apply to gaming applications. And that's not something you want to rely on in "real-world, safety-critical domains," she said.

"I want to develop algorithms to define safety and security, and make them part of the optimization of the MARL system," she explained. "We want them to not only collaborate to achieve certain tasks, we want them to collaborate to achieve those tasks safely."

Exploring AI’s Vulnerabilities

In previous research, Alqahtani determined that MARL systems are vulnerable. One of the AI agents could be compromised, potentially leading to a hack that affects the entire system.

Her work focuses on reinforcement learning (RL). It is like teaching a child to ride a bike, Alqahtani said. You can explain how to do it, but the child needs to keep trying–and failing–before finally balancing. With MARL, researchers describe the task, its parameters, and the goal, and offer rewards for each correct decision until the goal is reached, regardless of whether one AI agent fails.

Alqahtani grew up in a small Saudi Arabian village and chose to study computer science because it was one of the most challenging courses offered in college. She began studying MARL as a post-doctoral associate at the University of Tulsa.

Commitment to Education and Mentorship

NSF CAREER grants are awarded to junior faculty who exemplify the role of teacher-scholars through research and education.

At Wake Forest, she has introduced classes covering RL at both the graduate and undergraduate levels, an uncommon offering for undergrads. With the NSF grant, she will develop a new, hands-on undergraduate curriculum on AI security and safety, as well as a new graduate course on the safety and security of RL.

She is committed to Wake Forest's teacher-scholar model and has made mentoring students a large part of her work, both in the lab and in the classroom.

"I tell them, 'You're going to be beyond just collecting results and generating graphs. You are going to come up with problems, and you are going to solve them and write papers, too,'" she said. "And they do. I believe they can do it. I think that's what the students need–you to believe in them."

Student Success and Future Plans

That belief has paid off: Four of her undergraduates have received the prestigious Goldwater scholarship and the NSF Graduate Research Fellowship Program award in her five years at Wake Forest. Both honors are awarded to students who plan to pursue research careers in science, technology, engineering, or mathematics. Alqahtani's NSF CAREER award will also bring a post-doctoral scholar in computer science to Wake Forest.

Field Testing in the Amazon

Wake Forest students will join Alqahtani in the Amazon forests of Peru to test her MARL safety algorithms. Her interest in the environment has led to a partnership with Wake Forest's Center for Amazonian Scientific Innovation (CINCIA), where researchers track the devastating effects of illegal artisanal gold mining on the health of both people and the surrounding environment, including the trees and animals. Alqahtani is using MARL to train a fleet of drones to identify mining sites, allowing the police to be alerted. MARL will help the fleet continue its work even if one drone is disabled.

She will conduct two workshops for CINCIA employees in summer 2027 and 2029, reinforcing a collaborative partnership between Wake Forest students and the Universidad Nacional Amazónica de Madre de Dios, the region where CINCIA is located.

"I want to be able to equip the drones with the ability to recognize unsafe or dangerous areas and avoid them," she explained. "I'm excited to take my work outside the lab."