Redlining’s agent AI system is different from traditional systems. Agent AI and traditional AI systems are not deterministic, and scripts will need to be run multiple times. Each time the script is run the output will be different. You need to pay attention to these variables as you evaluate each situation. You should also remember that due to the way the agent works, the LLM itself, the diversity of information and behavior of the agent, will lead to more diversity. You’ll also find that doing the same task in the same situation will respond differently, and you’ll need to do more tests and test situations to cover any potential blind spots. Have your development teams map out all the rules and opportunities to flow through the process.
As with any tool, you cannot, and should not, always do everything automatically. Use a tool like PyRIT and manual testing. Manual testing will allow testers to check for specific problem areas and drill down deeper into any areas the automated test found.
Be sure to provide monitoring and logging for your test automation. This will help test the problem tracking process but also help as the team goes deeper with their manual testing. Review the process of using logged data to ensure clarity and readability at this stage, rather than when the problem presents itself in production.
Finallywork with other cybersecurity professionals to compare and contrast measures and processes. Continue to build your governance structure and constantly add and improve your processes.
The future of agent AI: It’s promising…and full of possibilities
The wide range of benefits, capabilities and efficiencies it can offer a business makes this the perfect time to explore this technology. However, the risks associated with security threats cannot be ignored. We must make sure that we expand the corporate culture so that safety is everyone’s responsibility. It is the responsibility of the teams to document all interactions, monitor the system and ensure that human control is in place. Tools must be integrated into end-to-end processes, to identify problems before they destroy user confidence and the business. Transparency, human oversight and AI security must always be top of mind.
Security teams need to outline controls and governance, security measures and rules. Development teams need to educate themselves, not only about these rules and requirements but also about the risks they will face and the mitigations they need to put in place.
Stephen Kaufman works as a master architect Microsoft Customer Success Unit A CTO office focused on AI and cloud computing. He brings more than 30 years of experience to some of the largest enterprise clients, helping them understand and implement AI from initial concepts to application development, design, development and delivery.
This article is made possible through our partnership with IASA Chief Architect Forum. CAF’s mission is to explore, challenge and support the art and science of Business Technology Architecture and its evolution over time and to increase the influence and leadership of great architects inside and outside the profession. CAF is the leading community of IASAthe leading non-profit professional organization for business technology designers.
Source link