文章基本信息

标题：Log Data Real Time Analysis Using Big Data Analytic Framework with Storm and Hadoop
本地全文：下载
作者：Lv Jia-Ke ; Li Yang ; Wang Xuan 等
期刊名称：MATEC Web of Conferences
电子版ISSN：2261-236X
出版年度：2018
卷号：246
DOI：10.1051/matecconf/201824603009
语种：English
出版社：EDP Sciences
摘要：The log data real-time processing platform which is built using Storm On YARN integrated MapReduce and Storm that use MapReduce to complete large-scale off-line data global knowledge extraction, sudden knowledge extraction of small-scale data in Kafka buffers through Storm, and continuous real-time calculation of streaming data in combination with global knowledge. We tested our technique with the well-known KDD99 CUP data set. The experimentation results prove the system to be effective and efficient.