文章基本信息

标题：Experiences with Resource Provisioning for Scientific Workflows Using Corral
本地全文：下载
作者：Gideon Juve, Ewa Deelman, Karan Vahi ; Gaurang Mehta
期刊名称：Scientific Programming
印刷版ISSN：1058-9244
出版年度：2010
卷号：18
DOI：10.3233/SPR-2010-0300
出版社：Hindawi Publishing Corporation
摘要：The development of grid and workflow technologies has enabled complex, loosely coupled scientific applications to be executed on distributed resources. Many of these applications consist of large numbers of short-duration tasks whose runtimes are heavily influenced by delays in the execution environment. Such applications often perform poorly on the grid because of the large scheduling overheads commonly found in grids. In this paper we present a provisioning system based on multi-level scheduling that improves workflow runtime by reducing scheduling overheads. The system reserves resources for the exclusive use of the application, and gives applications control over scheduling policies. We describe our experiences with the system when running a suite of real workflow-based applications including in astronomy, earthquake science, and genomics. Provisioning resources with Corral ahead of the workflow execution, reduced the runtime of the astronomy application by up to 78% (45% on average) and of a genome mapping application by an order of magnitude when compared to traditional methods. We also show how provisioning can benefit applications both on a small local cluster as well as a large-scale campus resource.