文章基本信息

标题：Beyond Batch Processing: Towards Real-Time and Streaming Big Data
本地全文：下载
作者：Saeed Shahrivari
期刊名称：Computers
电子版ISSN：2073-431X
出版年度：2014
卷号：3
期号：4
页码：117-129
DOI：10.3390/computers3040117
语种：English
出版社：MDPI Publishing
摘要：Today, big data are generated from many sources, and there is a huge demand for storing, managing, processing, and querying on big data. The MapReduce model and its counterpart open source implementation Hadoop, has proven itself as the de facto solution to big data processing, and is inherently designed for batch and high throughput processing jobs. Although Hadoop is very suitable for batch jobs, there is an increasing demand for non-batch requirements like: interactive jobs, real-time queries, and big data streams. Since Hadoop is not suitable for these non-batch workloads, new solutions are proposed to these new challenges. In this article, we discussed two categories of these solutions: real-time processing, and stream processing of big data. For each category, we discussed paradigms, strengths and differences to Hadoop. We also introduced some practical systems and frameworks for each category. Finally, some simple experiments were performed to approve effectiveness of new solutions compared to available Hadoop-based solutions.
关键词：big data; MapReduce; real-time processing; stream processing big data ; MapReduce ; real-time processing ; stream processing