Automate Scientific Workflow Execution between Local Cluster and Cloud

DOI

10.2991/ijndc.2016.4.1.5How to use a DOI?

Keywords

code offloading; scientific workflow; distributed computing; scheduling; cloud computing

Abstract

Scientific computational experiments often span multiple computational and analytical steps, and during execution, researchers need to store, access, transfer, and query information. Scientific workflow is a powerful tool to stream-line and organize scientific application. Numbers of tools have been developed to help build scientific workflows, they provide mechanisms for creating workflow but lack a native scheduling system for determining where code should be executed. This paper presents Emerald, a system that adds sophisticated computation offloading capabili-ties to scientific workflows. Emerald automatically offloads computation intensive steps of scientific workflow to the cloud in order to enhance workflow performance. Emerald minimizes the burden on developers to build work-flows with computation offloading ability by providing easy-to-use API. Evaluation showed that Emerald can ef-fectively reduce up to 55% of execution time for scientific applications.

Copyright

© 2017, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - JOUR
AU  - Hao Qian
AU  - Daniel Andresen
PY  - 2016
DA  - 2016/01/01
TI  - Automate Scientific Workflow Execution between Local Cluster and Cloud
JO  - International Journal of Networked and Distributed Computing
SP  - 45
EP  - 54
VL  - 4
IS  - 1
SN  - 2211-7946
UR  - https://doi.org/10.2991/ijndc.2016.4.1.5
DO  - 10.2991/ijndc.2016.4.1.5
ID  - Qian2016
ER  -