A new Big Data Cybersecurity Framework combines Apache Spark and deep learning to flag threats in power monitoring networks.
Researchers have developed a Big Data Cybersecurity Framework capable of identifying cyberattacks in intelligent power monitoring systems. This was investigated in a study published in the journal Energy Informatics.
Overview
Liang Li, Jun Zhang, Peng Zhang, Liangjing Cao and Chenxiao Zhao published their work on the 3rd June 2026. The system integrates Apache Flume, Kafka and Spark with an attention based Deep Gated Recurrent Unit (D-GRU). Computer simulation was used to evaluate the framework.
Why do power monitoring networks require protection?
Today's power networks are controlled digitally. Supervisory Control and Data Acquisition (SCADA), automated loops and smart meters all communicate across the network. If this communication is interrupted it can cause failures or blackouts. Traditional firewalls and standalone intrusion detection systems are inadequate. These use static signatures and cannot cope with unknown attacks. They are also slow to respond and produce plenty of false positives.
Framework
The framework has three layers: data, functional and interface. Data collection agents extract log files and sensor data from switches, routers and traffic collectors. The data goes through Flume for collecting, Kafka for buffering and routing. The micro-batch data goes through a Spark cluster on Hadoop.
Data cleaning occurs using the Gaussian kernel density estimation to remove redundant data. This is followed by the Linear Discriminant Analysis to reduce the number of features. Then the knowledge graph is used to identify relevant events. This can highlight multi-stage attacks.
The classifier categorises traffic into five different classes. These range from Security to Highly Dangerous. A situation value is assigned ranging from 0 to 1. Anything below 0.2 indicates that the network is healthy however anything above 0.75 indicates serious damage.
Results
The KDD Cup 1999 benchmark dataset was used containing 4.9 million connection records with 41 features. The attention based model performed better than traditional intrusion detection systems and Random Forest classifiers. However convolutional networks had slightly worse performance on sequential data. Transformers were too computationally heavy.
A case study was carried out on false data injection. The system was able to identify fraudulent sensor data while keeping false positives low. During peak load the admin server only used 60% of the processors and 65% memory.
Limitations
The benchmark lacks modern threats such as advanced persistent threats, modern IoT botnets, ransomware and zero day attacks. The framework will be tested using the CIC-IDS 2017 and TON_IoT benchmarks. Legacy SCADA hardware is also a challenge. This can be overcome by using lightweight protocol translators within the interface layer. Future work involves using game theory and verifying data transmission via blockchain.
Further Information
Telemetry data is formatted into 12 element tuples containing timestamp, security level, attack type, source and destination IP address, port, protocol and sensor.
Vulnerability score uses the Common Vulnerability Scoring System.
Operation and maintenance score uses disk, memory and processor utilisation.
The benchmark contains four types of intrusion: Denial of Service, Probe, User-to-Root and Remote-to-Local.
The test environment contained an admin server with 8GB RAM and 500 GB storage. The data collector on the remote side had 512Mb memory and 2.8GHz processor.
The paper can be found here. It is open access and can be found in Energy Informatics Volume 9 Article 67. DOI: 10.1186/s42162-025-00595-5
Business Fortune believes, the Big Data Cybersecurity Framework demonstrates how streaming infrastructure can be used with deep learning to provide quick and clean threat detection for grid operators. Results were obtained using simulation against a benchmark dataset. The authors will look to use modern datasets and integrate with legacy SCADA.
FAQs
1. What is a Big Data Cybersecurity Framework for power monitoring systems?
A Big Data Cybersecurity Framework is used to collect telemetry data across the network. This is processed at scale to classify threats. This framework uses Apache Flume, Kafka, Spark and a D-GRU neural network.
2. Who created the Big Data Cybersecurity Framework?
Liang Li, Jun Zhang, Peng Zhang, Liangjing Cao and Chenxiao Zhao published their study in Energy Informatics.
3. What dataset was used?
The KDD Cup 1999 intrusion detection benchmark dataset. This contained 4.9 million connection records.
4. Can it detect false data injection attacks?
In the authors case study they were able to identify manipulated sensor data while keeping false positives low.
5. Can this system be used in real utilities?
Not currently. The dataset is old and there are complications with the legacy SCADA.
Sources
Li et al., Energy Informatics 9, 67 (2026), Springer
.webp)
Comments