Denial of Service (DOS) Attack Detection on MQTT Protocol Using the Random Forest Method

Abstrak

The Message Queuing Telemetry Transport (MQTT) protocol serves as a critical lightweight communication infrastructure for Internet of Things (IoT) systems. Still, it remains highly vulnerable to Denial of Service (DoS) attacks that compromise network availability and security. Despite extensive IoT security research, existing MQTT-based intrusion detection systems predominantly employ binary classification approaches and lack comprehensive multi-class attack differentiation capabilities, limiting their practical deployment in real-world scenarios. This study addresses this critical gap by developing a multi-class DoS attack detection system utilizing the Random Forest algorithm to simultaneously classify normal traffic, MQTT flooding attacks, and SYN flood attacks. The methodology encompasses four systematic stages: collecting an MQTT network traffic dataset containing 1,634,286 records across three attack categories through controlled simulations; performing rigorous data preprocessing for cleaning and normalization; strategically extracting 60 MQTT-specific attributes to identify attack signatures; and implementing Random Forest with optimized hyperparameters for multi-class classification. Experimental results demonstrate optimal performance using an 80:20 train-test split with 5-fold cross-validation, achieving 95.27% precision, 95.09% recall, 95.08% F1-score, and 95.09% accuracy. A comprehensive evaluation using macro and micro-averaged metrics confirms the model's ability to autonomously classify MQTT network traffic types with high accuracy and balanced performance across all attack categories, offering a practical security solution for MQTT-enabled IoT infrastructure.