Although almost perfect on its training data set, a neural network-based intrusion detector fails as soon as the composition of the traffic changes even slightly.
Tobias Hürten and his co-authors present a training environment called ACO gym (Autonomous Cyber Operations), designed to train and evaluate AI models for IT defense. It combines high-fidelity network emulation, simulation of the behavior of legitimate users and attacker profiles, and the generation of cybersecurity scenarios from open toolkits.
Labelled datasets, and a caveat
The tool extracts features at the packet and stream levels, either live or posteriori, allowing for synthetic datasets labeled to train intrusion detection systems, as well as testing autonomous defense agent bricks through emulated interactions.
The experiment conducted on a neural classifier is instructive: in its training domain, it detects almost all malicious flows. But slight changes to the network, such as a different distribution of traffic protocols, are enough to cause its performance to drop catastrophically. The authors see this as a warning signal about the transferability of trained detectors in a controlled environment.
Read the full article on the original source
Go to the source →