The AI Security Institute conducted evaluations of Anthropic Claude Mythos Preview announced on 7th April to assess its cybersecurity capabilities. The testing has revealed concerning developments in how artificial intelligence systems can now carry out sophisticated cyber operations without human intervention, marking a significant shift in the cyber threat landscape facing British organisations and critical infrastructure.
Advanced AI Models Demonstrate Unprecedented Cyber Capabilities
Results show that Mythos Preview represents a step up over previous frontier models in a landscape where cyber performance was already rapidly improving. The evaluation painted a stark picture of how quickly AI technology has evolved in the cybersecurity domain. Two years ago the best available models could barely complete beginner-level cyber tasks, but now in controlled evaluations where Mythos Preview was explicitly directed and given network access to do so, it could execute multi-stage attacks on vulnerable networks and discover and exploit vulnerabilities autonomously, tasks that would take human professionals days of work.
AISI has tracked AI cyber capabilities since 2023, building progressively harder evaluations to keep pace with AI progress from chat-based probing to capture-the-flag challenges to multi-step cyber-attack simulations. This comprehensive approach to evaluation has allowed researchers to chart the dramatic acceleration in AI offensive capabilities over a relatively short period.
Simulated Corporate Network Attacks Show AI Progress
The testing involved complex scenarios designed to mirror real-world corporate environments. One particular evaluation called The Last Ones involved a 32-step simulated corporate network attack. The assessment measured how many steps different AI models could complete as they attempted to penetrate and move through a vulnerable network infrastructure. The results demonstrated that the newest models could progress significantly further through these attack chains than their predecessors, with some models continuing to make progress even with increased computational budgets.
Mythos Preview did also show some cyber capability limitations within the limits of the evaluation. However, the overall trajectory indicates that future iterations of these AI systems will likely possess even more advanced capabilities, raising urgent questions about how defenders can keep pace with AI-assisted threats.
Security Experts Call for Better Cyber Defences
Testing shows that Mythos Preview can exploit systems with weak security posture and it is likely that more models with these capabilities will be developed. This assessment has prompted calls for organisations across the United Kingdom to urgently strengthen their fundamental security practices. This highlights the importance of cybersecurity basics such as regular application of security updates, robust access controls, security configuration, and comprehensive logging.
Colleagues at the National Cyber Security Centre run the Cyber Essentials scheme to help organisations protect themselves against common online threats, whether those threats are AI assisted or not. The government-backed programme provides a baseline of cyber security measures that organisations should implement to protect against the majority of common cyber attacks.
Future Evaluations to Test Against Active Defences
In a regime where attackers can direct and provide network access to models to conduct autonomous attacks on poorly defended systems, cybersecurity evaluations must evolve, and as capabilities continue to improve, evaluation environments that lack defences will no longer be challenging enough to discriminate between the capabilities of the most cyber-capable models or assess trends.
Future work will involve evaluating capabilities using ranges simulating hardened and defended environments including ranges with active monitoring, endpoint detection and real-time incident response, and will also be tracking how AI-enabled vulnerability discovery and penetration testing campaigns perform on real-world systems. This evolution in testing methodology reflects the need to understand not just what AI can do against vulnerable systems, but how it performs against organisations with mature security programmes.
Investment in Cyber Defence Becomes Critical Priority
Future frontier models will be more capable still so investment now in cyber defence is vital. The findings have significant implications for organisations of all sizes across the United Kingdom. The ability of AI systems to autonomously discover and exploit vulnerabilities means that traditional security approaches focused primarily on human attackers may no longer be sufficient.
AI cyber capabilities are dual use and while they pose security challenges they can also help deliver game-changing improvements in defence, and AISI recently released a joint blog post with NCSC on how cyber defenders can both harness and prepare for frontier AI. This balanced perspective suggests that while AI presents new threats, it also offers opportunities for defenders to enhance their capabilities and better protect their networks and systems.
The revelations come at a time when UK organisations are already facing heightened cyber threats from multiple sources. The combination of increasingly sophisticated AI-powered attack tools and persistent threats from state-sponsored actors and criminal groups creates a complex threat environment. Security professionals across the country will need to adapt their strategies and invest in both traditional security fundamentals and next-generation defensive technologies to stay ahead of these evolving risks.