文章基本信息

标题：Efficient Job Execution for Map Reduce Using Phase-Level Scheduling Algorithm
本地全文：下载
作者：Mary Vennila S ; Prabhakarana S
期刊名称：International Journal of Innovative Research in Computer and Communication Engineering
印刷版ISSN：2320-9798
电子版ISSN：2320-9801
出版年度：2016
卷号：4
期号：2
页码：2617
DOI：10.15680/IJIRCCE.2016.0402242
出版社：S&S Publications
摘要：Map Reduce has become a popular model for data - intensive computation in recent years. It can significantly reduce the running time of data - intensive jobs by breaking down each job into small map and reduce tasks and executing them in parallel across a lar ge number of machines. Designing resource - efficient Map Reduce schedulers is important for effectively reducing the job running time. Existing solutions are focused on scheduling a job at task - level. And the tasks can have highly varying resource requirem ents during their lifetime, which makes it difficult for task - level schedulers to effectively utilize available resources to reduce job execution time. To address this limitation, in existing system they introduced a scheduler called PRISM, a fine - grained resource - aware Map Reduce scheduler that divides tasks into phases, where each phase has a constant resource usage profile, and performs scheduling at the phase level. They first demonstrate the importance of phase - level scheduling by showing the resource usage variability within the lifetime of a task using a wide - range of Map Reduce jobs. And then present a phase - level scheduling algorithm that improves execution parallelism and resource utilization without introducing stragglers. There is a limitation in PRISM scheduling, such that when allocating each phase into the scheduler the phase will execute when only the resources available. If not, the phase will be paused until getting the resource available and the next phase is scheduled to execute. Due to th e pausing the phase will make delay in running a job. In my proposed system to address this limitation I will use a concept of virtual resource allocation, instead of pausing the phase it will be availed to use the resources virtually until getting the res ource available. It will reduce the delay in running a job significantly
关键词：Data intensive computation; MapReduce; ; ; scheduling; resource allocation