Cybersecurity researchers have found a important safety flaw in a synthetic intelligence (AI)-as-a-service supplier Replicate that would have allowed risk actors to realize entry to proprietary AI fashions and delicate info.
“Exploitation of this vulnerability would have allowed unauthorized access to the AI prompts and results of all Replicate’s platform customers,” cloud safety agency Wiz stated in a report revealed this week.
The problem stems from the truth that AI fashions are sometimes packaged in codecs that enable arbitrary code execution, which an attacker might weaponize to carry out cross-tenant assaults by the use of a malicious mannequin.
Replicate makes use of an open-source device known as Cog to containerize and package deal machine studying fashions that would then be deployed both in a self-hosted setting or to Replicate.
Wiz stated that it created a rogue Cog container and uploaded it to Replicate, finally using it to attain distant code execution on the service’s infrastructure with elevated privileges.
“We suspect this code-execution technique is a pattern, where companies and organizations run AI models from untrusted sources, even though these models are code that could potentially be malicious,” safety researchers Shir Tamari and Sagi Tzadik stated.
The assault method devised by the corporate then leveraged an already-established TCP connection related to a Redis server occasion inside the Kubernetes cluster hosted on the Google Cloud Platform to inject arbitrary instructions.
What’s extra, with the centralized Redis server getting used as a queue to handle a number of buyer requests and their responses, it could possibly be abused to facilitate cross-tenant assaults by tampering with the method as a way to insert rogue duties that would impression the outcomes of different prospects’ fashions.
These rogue manipulations not solely threaten the integrity of the AI fashions, but additionally pose important dangers to the accuracy and reliability of AI-driven outputs.
“An attacker could have queried the private AI models of customers, potentially exposing proprietary knowledge or sensitive data involved in the model training process,” the researchers stated. “Moreover, intercepting prompts might have uncovered delicate knowledge, together with personally identifiable info (PII).
The shortcoming, which was responsibly disclosed in January 2024, has since been addressed by Replicate. There isn’t any proof that the vulnerability was exploited within the wild to compromise buyer knowledge.
The disclosure comes slightly over a month after Wiz detailed now-patched dangers in platforms like Hugging Face that would enable risk actors to escalate privileges, acquire cross-tenant entry to different prospects’ fashions, and even take over the continual integration and steady deployment (CI/CD) pipelines.
“Malicious models represent a major risk to AI systems, especially for AI-as-a-service providers because attackers may leverage these models to perform cross-tenant attacks,” the researchers concluded.
“The potential impact is devastating, as attackers may be able to access the millions of private AI models and apps stored within AI-as-a-service providers.”
