In a significant move to bolster the safety and security of generative artificial intelligence (AI) systems, Google recently announced the expansion of its Vulnerability Rewards Program (VRP). This program aims to compensate researchers for uncovering attack scenarios specifically tailored to AI systems. As the field of AI continues to advance rapidly, it becomes crucial to address the unique concerns and potential risks associated with generative AI. By incentivizing researchers to identify vulnerabilities in these systems, Google aims to promote AI safety and mitigate potential threats.
Concerns with Generative AI Systems
Generative AI systems, while remarkable in their ability to create new content and generate realistic outputs, bring about new and different concerns compared to traditional digital security. One significant concern is the potential for unfair bias in AI decision-making processes. These biases can emerge due to biased training data or biased algorithms used in the generative AI system. Additionally, model manipulation and misinterpretations of data, referred to as “hallucinations,” pose further risks to the integrity and reliability of AI models.
Categories in scope for the program
To effectively address the vulnerabilities specific to generative AI systems, Google’s expanded VRP includes several categories on which researchers can focus. These categories include:
Prompt Injections: Researchers will investigate potential vulnerabilities related to injecting biased or malicious prompts into generative AI systems, which can influence their output.
Leakage of Sensitive Data: This category involves identifying vulnerabilities that could result in the unauthorized exposure of sensitive data from the training datasets used by AI systems.
Model Manipulation: The goal here is to uncover vulnerabilities that allow malicious actors to manipulate or tamper with generative AI models, potentially leading to undesirable outcomes.
Adversarial Perturbation Attacks: Researchers will focus on finding attack scenarios that involve perturbing inputs to generative AI systems to trigger misclassification or generate misleading outputs.
Model Theft: This category centers around identifying vulnerabilities that could facilitate the theft or unauthorized use of generative AI models.
Google’s AI Red Team and Secure AI Framework
To further strengthen the security of AI systems, Google established an AI Red Team in July as part of its Secure AI Framework (SAIF). The Red Team will focus on identifying and addressing potential threats to AI systems. By adopting a proactive approach to AI security, Google aims to stay ahead of emerging risks and ensure the resilience and safety of its AI technologies.
Strengthening the AI supply chain
Efforts to fortify the AI supply chain are underway through various open-source security initiatives, including Supply Chain Levels for Software Artifacts (SLSA) and Sigstore. These initiatives offer tools and standards to enhance the security and integrity of the software used in AI systems.
Digital signatures, such as those provided by Sigstore, play a crucial role in this process. They enable users to verify that software has not been tampered with or replaced, ensuring the authenticity and trustworthiness of AI systems. Additionally, metadata such as SLSA provenance provides essential information about the software’s composition and construction, allowing consumers to identify vulnerabilities, ensure license compatibility, and detect advanced threats.
OpenAI’s Preparedness Team and AI Safety Fund
Recognizing the growing importance of AI safety, OpenAI, along with Google, Anthropic, and Microsoft, has unveiled a $10 million AI Safety Fund. This fund aims to support research focused on addressing potential risks and catastrophic scenarios associated with generative AI. OpenAI has also established an internal Preparedness team dedicated to tracking, evaluating, forecasting, and protecting against various risks, including cybersecurity and chemical, biological, radiological, and nuclear (CBRN) threats.
Google’s expansion of its VRP and its concerted efforts to secure the AI supply chain highlight the increasing significance of addressing security challenges in the field of artificial intelligence. By incentivizing researchers to uncover vulnerabilities specific to generative AI systems, Google takes a proactive stance towards improving AI safety and security. As AI continues to permeate various aspects of society, it is crucial to ensure that these advanced technologies are developed, deployed, and managed with a strong focus on safeguarding against potential risks and threats.