文章基本信息

标题：PCSP: A Preemptive Capacity Scheduler Policy for Scheduling Hadoop Jobs
本地全文：下载
作者：Xue Shengjun ; Wang Delong ; Shi Suhong 等
期刊名称：International Journal of Grid and Distributed Computing
印刷版ISSN：2005-4262
出版年度：2015
卷号：8
期号：5
页码：33-46
DOI：10.14257/ijgdc.2015.8.5.03
出版社：SERSC
摘要：Map reduce is a parallel programming paradigm used for processing massive data sets. A popular open-source implementation of Map reduce is Hadoop. There are basic schedulers embedded in Hadoop, including First in First out (FIFO), Fair Scheduler, and Capacity Scheduler (CS). Currently, researches have been focused on Capacity Scheduler to improve the Capacity Scheduler. Native Capacity Scheduler does not support the preemption, which results in the starvation caused by the non-preemptive scheduling. To resolve this problem, a Preemptive Capacity Scheduler Policy (PCSP) is proposed. Finally, we implement the PCSP on Hadoop, the experimental results of which indicate that PCSP we proposed is efficient in running Hadoop jobs.
关键词：PCSP; Capacity Scheduler; preemption; Hadoop; Map reduce