The US National Institute of Standards and Technology (NIST) has recently taken a significant leap towards developing strategies to defend against cyber threats that specifically target AI-powered chatbots and self-driving cars. As technological advancements continue to shape our world, ensuring the security and integrity of artificial intelligence (AI) systems is of paramount importance. To address this concern, NIST has released a comprehensive paper on January 4, 2024, which establishes a standardized approach to characterizing and defending against cyberattacks on AI.
NIST’s Paper: A Taxonomy and Terminology of Attacks and Mitigations
In an exemplary display of collaboration between academia and industry, NIST has teamed up with renowned experts to co-author a groundbreaking paper titled “Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations.” This paper serves as a foundational resource, providing a structured framework to understand and combat cyber threats directed towards AI systems.
Taxonomy: Categorizing Adversarial Machine Learning (AML) Attacks
NIST’s taxonomy categorizes AML attacks into two distinct categories: attacks targeting “predictive AI” systems and attacks targeting “generative AI” systems. Under the umbrella of “predictive AI,” NIST includes a sub-category called “generative AI,” which encompasses generative adversarial networks, generative pre-trained transformers, and diffusion models.
Attacks on Predictive AI Systems
Within the realm of predictive AI systems, the NIST report identifies three primary types of adversarial attacks: evasion attacks, poisoning attacks, and privacy attacks.
Evasion attacks aim to generate adversarial examples, which are intentionally designed to deceive an AI system and alter the classification of testing samples. These attacks exploit vulnerabilities in the AI system’s decision-making process, manipulating it to provide incorrect and potentially harmful outputs.
Unlike evasion attacks that target the testing phase, poisoning attacks occur during the training stage of an AI algorithm. Adversaries gain control over a relatively small number of training samples, injecting malicious data that can compromise the AI system’s performance and undermine its reliability.
Privacy attacks focus on extracting sensitive information about the AI model or the data on which it was trained. Adversaries aim to compromise the privacy and confidentiality of the AI system, potentially leading to significant consequences, such as data breaches or unauthorized access.
Attacks on Generative AI Systems
AML attacks targeting generative AI systems fall under the category of abuse attacks. These attacks involve the deliberate insertion of incorrect or malicious information into the AI system, leading it to generate inaccurate outputs. By strategically manipulating the learning process of generative AI models, adversaries can compromise the integrity of the system’s outputs, leading to potentially severe consequences in various domains such as content generation, voice recognition, or image manipulation.
NIST’s groundbreaking paper on adversarial machine learning attacks is a significant step towards creating a comprehensive defense against cyber threats targeting AI systems. By providing a taxonomy and terminology of attacks, NIST equips researchers, developers, and policymakers with a foundational understanding of the threats faced by AI-powered systems. This standardized approach empowers the cybersecurity community to develop robust and effective mitigation strategies, ensuring the continued advancement and adoption of AI technology while safeguarding against malicious attacks.
As the landscape of AI-powered technologies expands, NIST’s efforts will play a crucial role in establishing trust, reliability, and security within these systems. By staying vigilant and proactive in addressing emerging threats, we can pave the way for a future where AI-driven innovations thrive, benefiting our society in countless ways while mitigating the risks associated with cyber-attacks.