Want to know how bad guys attack AI systems? MITRE’S ATLAS can show you

Now that information has been gathered, bad actors begin to plan attacks with knowledge of the target systems. It could be training proxy models, poisoning the target model, or creating adversarial data to fit into the target model.

The four strategies identified include:

• Create a representative ML model
• ML backdoor model
• Confirm the attack
• Create conflicting data

Proxy ML models can be used to simulate attacks and take them offline while attackers hone their skills and desired outcomes. They can also use offline copies of target models to ensure the success of the attack without raising the suspicions of the victim organization.

After all the steps discussed, attackers get to what they really care about – immersion. This includes stealing ML artifacts or other information about the ML system. It could be intellectual property, financial information, PHI or other sensitive data depending on the model implementation and the ML systems involved.

Techniques associated with immersion include:

• Exfiltration with ML inference API
• Release by cyber means
• LLM meta release
• LLM data leak

All of this includes exfiltrating data, whether through an API, common online methods (e.g. ATT&CK exfiltration), or using information to allow LLM to leak sensitive data, such as private user data, organizational proprietary data, and training data, which may include between information. This has been one of the leading issues regarding the use of LLM by security professionals as organizations quickly adopt it.

Unlike immersion, the impact phase is where attackers cause damage or harm, which can cause disruption, destroy confidence, or destroy ML systems and data. At this stage, that may include acquiring a target (for ransom, for example) or harming integrity in an unfair way.

This strategy has six strategies, including:

• Avoiding ML models
• ML denial of service
• Spam ML programs with husk data
• It removes the integrity of the ML model
• Cost savings
• External injuries

Although we have discussed some strategies as part of other strategies, there are exceptions here related to influence. For example, ML denial of service refers to using resources or flooding systems with requests to degrade or disable services.

While most modern enterprise AI offerings are hosted in the cloud with elastic compute, they can still run into DDoS and resource endpoints, as well as cost implications if not properly mitigated, impacting both provider and consumer.

Additionally, attackers may look to destroy the integrity of the ML model instead of entering conflicting data that affects the reliability of the ML model and causes the model provider or organization to fix system and operational issues to address the integrity.

Finally, attackers may look to create external harm, such as abusing the access they get to affect the victim’s system, resources, and organization in ways that involve financial and reputational harm, impact users or wider social harm depending on the use and impacts. of the ML system.