Developing intelligent systems for missions characterized by uncertain and adversarial environments
Scientists and engineers in APL’s Intelligent Systems Center (ISC) work to enable confidence in intelligent systems for critical national security applications through research in uncertainty-aware risk sensitivity, adversarial vulnerabilities and defenses, fairness and privacy, and testing and evaluation.
TrojAI researcher Neil Fendley demonstrates a backdoor he embedded in the deep network weights of a common network used for object detection and classification. The network classifies dozens of objects correctly, but when a person puts the embedded trigger—in this case the black and white target sticker—on their clothes, the system immediately misidentifies them as a teddy bear. The backdoor is very specific: When placed on other objects—like the chair—the trigger has no impact, and the network makes correct classifications.
ISC researchers analyze vulnerabilities and defenses of critical AI applications relative to system-level performance and operational constraints across the entire development life cycle. Recent ISC projects studied the sensitivity of adversarial attacks on computer vision systems to physical constraints, general approaches for detecting adversarial inputs to deep learning models, methods for evaluating vulnerabilities to backdoor Trojan attacks at scale, and techniques for “sanitizing” deep networks infected by data poisoning.
