Researchers from the New Mexico State University discovered a security flaw that allows malicious data to be inserted into the long-term memory of agents built with language models. This study, published in July on the arXiv platform, details a type of attack capable of modifying the future behavior of these artificial intelligence systems.
GhostWriter Technique and its Application
The methodology, named GhostWriter, exploits AI agents that use memories of past interactions to personalize their responses and perform tasks on behalf of users. While this capability increases the practicality of these tools, it simultaneously creates a new area susceptible to cyberattacks.
In addition to exposing how this threat works, the researchers proposed a defense designed to reduce the probability that false memories are recorded and subsequently used by the agents, without substantially harming the performance of these technologies.
How the Memory Attack Works
Language models are already widely used across various platforms, from conversation to search and text generation tools. Recently, parts of these systems have begun operating autonomously, executing functions such as scheduling appointments and modifying programming code.
With the addition of persistent memory mechanisms, these agents can access information from previous dialogues to tailor responses to user preferences, avoiding data repetition. However, this technological evolution has introduced a little-explored digital security issue.
The authors point out that personal assistants accumulate sensitive data while interacting with external content, which generates new vulnerabilities. The team describes GhostWriter as a method of long-term memory contamination, where hidden instructions or false information are injected into the agent's permanent storage to be retrieved and influence future actions.
Attack Stages and Practical Example
The attack unfolds in two phases. Initially, the attacker sends disguised content to be registered as a valid memory. Subsequently, when the user requests something related to the topic, the system retrieves this compromised data and acts according to the previously inserted guidelines.
An illustrative case presented by the researchers involves an email management agent. In this scenario, a false memory could instruct the tool to automatically summarize and forward messages from a specific sender to an address controlled by the attacker, even if those communications contain confidential data, all without the user's knowledge.
Tests conducted by the team showed a high success rate in including these harmful memories, along with a considerable frequency of subsequent activation of the stored instructions. These findings indicated that the problem could affect agents considered among the most advanced available today.
Security Mitigation Proposal
In response, the study introduces Agentic Memory Sentry, known by the acronym AM-Sentry. This solution integrates a stricter policy to determine which information can be saved in permanent memory and a checking system before these records are accessed during new requests.
The results obtained by the researchers demonstrated that this proposal drastically reduced the effectiveness of GhostWriter while maintaining the practical functionality of AI agents. The team expects such strategies to be improved and tested on other models that possess persistent memory.