Time to make decisions: The rich consequences of inviting rich questions
Fully developed LLMs can be used for forecasting and related analysis. Here, as before, the key is repetition. The exception to this section, however, should be the focus of the decision maker. Assessing key questions about cybersecurity work, change, and relevant external factors should be framed in terms that are understood by decision makers.
An important takeaway from the UCP study is that LLM results should be dissected and analyzed to understand points of convergence and divergence. Doing so allows editors to place their weight on variables that appear to be critical in determining the standing of some beliefs relative to others.
Then, so armed, editors can put these findings directly into decision makers’ documents as an alternative to reporting directly on the results of several AI models. In other words, a comparative analysis of how LLMs arrive at individual interesting conclusions is important, rather than the generated cases or propositions themselves.
Bottom line: Avoiding an AI CISO
When it comes to using LLMs effectively in cyber security planning, the bottom line is clear: Planners and managers should avoid the AI CISO. Simply put, the AI CISO concept describes situations where an organization uses AI without involving humans not only in decision-making, but also in discussions about ethics, methodology, and underlying technology.
The result will be the rise of AI systems as de facto decision makers. Not Skynet or HAL 9000, of course, but the support systems to which we delegate much of our decision-making.
This latest study and others like it lay out the first best practices for achieving this. They argue that effective use of LLMs in robust forecasting and analysis means having people in the loop at all stages of the deployment.
More importantly, they make the case that these interactions must reflect the full range of human expertise — from information expertise to investigative skills and marketing savvy — to get the most out of the machine.
Source link