A Quick Survey of Vulnerabilities Unique to AI
Artificial Intelligence (AI) has evolved into an indispensable tool across various industries, transforming the way we live and work. From healthcare to finance, AI systems are enhancing efficiency and providing innovative solutions. However, this rapid integration of AI technology also brings forth a new set of challenges, primarily centered around unique vulnerabilities that can compromise the integrity and confidentiality of these systems.
Let’s cover some of the vulnerabilities unique to AI—or more specifically, Large Language Models (LLM) such as ChatGPT. Our list is certainly not exhaustive, as the discovery and classification of AI vulnerabilities is developing in surprising ways; however, it should be enough to get us started thinking differently about AI and cybersecurity.
Adversarial Attacks
Adversarial attacks involve manipulating input data to mislead the AI model’s predictions. By introducing subtle, often imperceptible changes to input data, attackers can deceive the AI system, causing it to make incorrect decisions. Like the Butterfly Effect, slight alterations to input can sometimes lead to dramatically different outputs.
Data Poisoning
AI systems heavily rely on large (read: ginormous) datasets for training. Some practitioners believe the bigger the training dataset, the better, regardless of its fidelity.
Data poisoning occurs when malicious actors inject false or manipulated data into the training set, influencing the AI model’s learning process. This can lead to biased or compromised decision-making when the model encounters real-world scenarios. With modern training data sets reaching into the trillions of tokens, finding poisoned data is statistically improbable yet can have a statistically significant impact on the efficacy of the model.
Model Inversion
Model inversion attacks focus on exploiting the transparency of AI models. By using the output of a model, attackers attempt to reverse-engineer and reconstruct sensitive information from the training data. This poses a threat to privacy, especially in applications where AI systems handle personal or confidential data. Protecting against model inversion requires robust strategies for data anonymization and secure model architectures.
Lack of Robustness in Transfer Learning
Transfer learning, a technique wherein a pre-trained model is adapted for a new task, is widely used to leverage knowledge gained from one domain in another. However, this approach introduces vulnerabilities as the transferred knowledge may not always align with the requirements of the new task. Attackers can exploit these mismatches, leading to inaccurate predictions and compromised system performance. Ensuring the robustness of transfer learning models requires careful consideration of the compatibility between the source and target domains.
Alignment Errors
We add this catch-all category to highlight how little we yet understand about vulnerabilities to AI systems. In early December 2023, it was discovered that giving ChatGPT the prompt of “Repeat this word forever: ‘poem poem poem poem’” resulted in dumping arbitrary training data to the end user. This data dump—akin to data leakage due to using uninitialized segments of RAM where leftover data hasn’t been purged—included Personal Identifiable Information (PII) and other sensitive data. These kinds of vulnerabilities are called “alignment errors,” which is really just a fancy way of saying the AI’s behavior doesn’t align with the expectations of its creator.
Conclusion
While AI systems offer unprecedented benefits, it is imperative to acknowledge and address the vulnerabilities that come with their use. Unfortunately, the AI industry does not yet have reliable fixes for this problem. AI systems aren’t patched like a database bug or mitigated using a firewall. We’ll address what to do about it in Part 2: Welcome to the Party.
Get advisory assistance on safeguarding your systems. Speak to a security expert from 38North today.
