A Data Replica Selection Algorithm Based on Cloud Platform

DOI

10.2991/icmmita-15.2015.100How to use a DOI?

Keywords

Smart Grid Task; MapReduce; Data Replicas; Join Operation

Abstract

Aiming at the problem that how to choose the least cost data resources from multiple datasets which have the same contents but different locations for smart grid tasks, a least cost of selection algorithm based on locality is proposed. First, the problem of choosing data replicas is abstract as a shortest path issue that is from the source point to other vertices in directed graph. Second, in order to complete the join operation in Map stage, the related data is grouped in the same datanode, so as to avoid pulling data in Reduce stage. Experimental results show that the algorithm can effectively reduce the data transmission time, and then reduce the task completion time, improve the real-time performance of the smart grid processing tasks.

Copyright

© 2015, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Shaomin Zhang
AU  - Qi Wang
AU  - Baoyi Wang
PY  - 2015/11
DA  - 2015/11
TI  - A Data Replica Selection Algorithm Based on Cloud Platform
BT  - Proceedings of the 2015 3rd International Conference on Machinery, Materials and Information Technology Applications
PB  - Atlantis Press
SP  - 514
EP  - 519
SN  - 2352-538X
UR  - https://doi.org/10.2991/icmmita-15.2015.100
DO  - 10.2991/icmmita-15.2015.100
ID  - Zhang2015/11
ER  -