D6.6 Final version of the synthetic datasets for cyber intelligence using fair Machine Learning

Cyberattacks targeting critical and enterprise infrastructure have surged dramatically in recent decades, necessitating advanced and adaptive cybersecurity measures. This report focuses on enhancing Intrusion Detection Systems (IDS) through integrating large language models (LLMs) and synthetic data generation methodologies. The core objective of this research is to synthesize a robust and comprehensive dataset by combining newly generated synthetic data with existing historical data. This synthesized dataset aims to retrain and enhance IDS models, reducing false positives and improving overall detection capabilities.
A real-time pipeline for network anomaly detection was introduced, along with advanced tools such as SecGPT, an LLM-based agent designed for autonomous penetration testing. Generative Adversarial Networks (GANs) were utilized to generate synthetic data that accurately reflects dynamic network environments. Evaluation of this approach demonstrated that machine learning models trained on the synthesized dataset achieve performance metrics comparable to those trained exclusively on real data. Additionally, SecGPT's autonomous capabilities in conducting penetration tests and generating realistic attack profiles contribute to the continuous and dynamic improvement of IDS.
This dataset will enhance IDUNN project components and tools like THOR Analytics, HEIMDAL vEDR, ODIN, and FRIGG, improving the risk assessment, threat classification, and network security visualization. This research presents a novel approach for developing more resilient and adaptive IDS by utilizing the strengths of LLMs and GANs. The findings highlight the potential of these technologies in creating robust network security solutions capable of addressing the complexities of modern cyber threats. Additionally, the generated data has been publicly published in Fairdata IDA, which a service offered by CSC – IT Center for Science in Finland. The data can be accessed at: https://doi.org/10.23729/1cdd65b5-0487-45af-bd3c-43327b3f73d0.