文章基本信息

标题：Apache Spark and MLlib-Based Intrusion Detection System or How the Big Data Technologies Can Secure the Data
本地全文：下载
作者：Otmane Azeroual ; Anastasija Nikiforova
期刊名称：Information
电子版ISSN：2078-2489
出版年度：2022
卷号：13
期号：2
页码：58
DOI：10.3390/info13020058
语种：English
出版社：MDPI Publishing
摘要：Since the turn of the millennium, the volume of data has increased significantly in both industries and scientific institutions. The processing of these volumes and variety of data we are dealing with are unlikely to be accomplished with conventional software solutions. Thus, new technologies belonging to the big data processing area, able to distribute and process data in a scalable way, are integrated into classical Business Intelligence (BI) systems or replace them. Furthermore, we can benefit from big data technologies to gain knowledge about security, which can be obtained from massive databases. The paper presents a security-relevant data analysis based on the big data analytics engine Apache Spark. A prototype intrusion detection system is developed aimed at detecting data anomalies through machine learning by using the k-means algorithm for clustering analysis implemented in Sparks MLlib. The extraction of features to detect anomalies is currently challenging because the problem of detecting anomalies is not actively and exhaustively monitored. The detection of abnormal data can be effectuated by using relevant data that are already in companies’ and scientific organizations’ possession. Their interpretation and further processing in a continuous manner can sufficiently contribute to anomaly and intrusion detection.