Working in IT security in the age of AI is both exciting and daunting, according to Professor Mathias Ekstedt. Photo: Jon Lindhe, KTH

Published Oct 16, 2026

A research project at KTH aims to improve autonomous AI agents’ ability to carry out so-called penetration testing. The agents are set to become more effective – but also more responsible. 

“Much of security testing today is carried out through penetration testing,” says Mathias Ekstedt, professor of Industrial Information and Control Systems at KTH Royal Institute of Technology. 

“You allow hackers to attack a system and see they can do.” 

Ekstedt and his colleagues are building models of IT systems in which various attacks can be simulated. The major advantage of the KTH models is that the attack simulations can be scaled up without tampering with the system itself and risking side effects in a production environment. 

Thousands of combinations of vulnerabilities and weaknesses are analysed to assess how they collectively form weak links. The tests can also be adapted to different types of hacker attacks. 

“How difficult was it to take down a system over there? How difficult was it to steal the credit card information from there? You can learn something about how resilient the system environment is,” Ekstedt says. 

The KTH researchers have been working this way for many years, and they have shared their findings on GitHub. But as AI agents powered by large language models (LLM) have emerged, IT security has undergone a fundamental shift. AI agents can both identify vulnerabilities and write code that exploits them – and are now being used by both hackers and defenders. 

“It’s already very clear that AI is extremely good at hacking and will only get better,” Ekstedt says. 

Agent vs agent 

To keep pace with malicious actors, the researchers are developing the AI agents further. In a new Vinnova-funded project, they are developing a support system for penetration testing that both provides the attacker with knowledge and can steer the tests towards specific behaviours. 

“In a given environment, one might say that certain tricks are perhaps unethical for some reason, so we should avoid those types of attacks. This can then also be incorporated into this decision-support system,” Ekstedt says. 

Steering the tests allows the hacking agents to focus on known security vulnerabilities and deliver faster results. Another advantage is that the penetration tests can be tailored to the expected behaviour of a specific attacker, says Ekstedt, who gives a few examples: 

“The North Korean state is known for seeking out cryptocurrencyin order to fund its regime. That’s how they operate. One criminal network is known for carrying out these kinds of activities, using these sorts of attacks and tricks. So, the question is, who am I afraid of? And I should, of course, tailor my defences to what I think they’re going to do.” 

However, this logic is being turned somewhat on its head by the rapid proliferation of AI agents. 

“All attackers are going to use AI, so it’s unclear whether they’ll leave any sort of digital footprint worth talking about,” Ekstedt says. 

An uncertain future 

Despite rapid change in the overall aim is still to help system owners build more effective defences. In the short term, Ekstedt believes attackers will gain the upper hand, as they are quicker to use AI. But in the long term, AI could tip the balance in favour of the defenders. 

“It has been too expensive for the defender to scrutinise their entire environment in depth, along with all its configurations,” Ekstedt says. “ I imagine AI will bring the cost down, so that hopefully they will now be able to make use of all the information they have about their own system.” 

In short, the information becomes a knowledge advantage over the attacker and allows the system owner to build their defences effectively. 

“The attacker still only needs to find one vulnerability, but where there used to be 1,000 vulnerabilities, there is now only one left – and it’s very difficult to find that one.” 

Jon Lindhe