We are excited to share another innovative research article from the IDUNN project, titled “HuntGPT: Integrating Machine Learning-Based Anomaly Detection and Explainable AI with Large Language Models (LLMs)”. This study, co-authored by Tarek Ali and Panos Kostakos from the University of Oulu, explores the integration of machine learning and explainable AI in the realm of network anomaly detection.
Abstract
Machine learning (ML) methods are proving to be effective in network anomaly detection, offering proactive threat-hunting strategies that significantly reduce detection and response times. Despite these advancements, challenges such as training and maintaining ML models and dealing with frequent false positives limit their adoption. Explainable AI (XAI) techniques have been introduced to enhance trust and confidence in AI-generated alerts, but they often fall short in meeting the decision-making needs of cybersecurity analysts and model maintainers.
Introducing HuntGPT
HuntGPT is a specialized intrusion detection dashboard that leverages a Random Forest classifier trained on the KDD99 dataset. This tool integrates XAI frameworks like SHAP and Lime, which improve the user-friendliness and intuitiveness of the model. Additionally, HuntGPT incorporates GPT-3.5 Turbo as a conversational agent, providing detected threats in an easily explainable format and offering a seamless interactive experience.
Methodology
The research focuses on the comprehensive architecture of HuntGPT and its diverse components. Key features include:
- Random Forest Classifier: Utilized for effective anomaly detection.
- XAI Frameworks: SHAP and Lime are incorporated to enhance model explainability.
- Conversational Agent: GPT-3.5 Turbo delivers explanations of detected threats, improving user understanding.
Evaluation
The technical accuracy of the HuntGPT prototype was assessed using the Certified Information Security Manager (CISM) Practice Exams. The quality of response readability was evaluated across six unique metrics, demonstrating the effectiveness of conversational agents underpinned by LLM technology in generating explainable and actionable AI solutions.
Conclusion
Our findings indicate that integrating LLMs with XAI can provide a robust mechanism for developing explainable and user-friendly intrusion detection systems. HuntGPT represents a significant advancement in cybersecurity, enhancing the effectiveness of anomaly detection and improving the overall user experience.
For a detailed exploration of our framework and findings, please refer to the full article published in the framework of the IDUNN project.