AI startup Anthropic is launching a bug reporting system
Artificial intelligence startup Anthropic launched a vulnerability disclosure program (VDP), managed by HackerOne, in August with cash prizes of up to $15,000 for novel, general-purpose jailbreak attacks that could uncover vulnerabilities in sensitive, high-risk domains such as CBRN (chemical, biological, radiological , and nuclear) and cybersecurity.
Jailbreak attacks on AI involve a way to bypass the AI system’s built-in security measures and behavioral guidelines, allowing the user to elicit responses or behavior from the AI system that would normally be blocked.
“As we work to develop the next generation of our AI protection systems, we are expanding our bug bounty program to introduce a new program that focuses on finding errors in reducing our usage to prevent misuse of our models,” Anthropic said in a blog post. posted on the advanced program.
Source link