The Optimization of Task Assignments on Hadoop Platform for Large-Number Image Processing

DOI

10.2991/aiie-15.2015.15How to use a DOI?

Keywords

Hadoop cluster; Image processing; Hadoop streaming; genetic algorithm

Abstract

This paper proposes a large-number images processing model based on Hadoop platform. According to this model, the task allocation strategies are investigated for large-number image processing on the isomorphic Hadoop clusters and heterogeneous Hadoop clusters. Firstly, a series of different experiments is performed on the isomorphic Hadoop clusters. The obtained results show that the equal allocation of all images to each node is proved to be an efficient approach. For the heterogeneous clusters, the genetic algorithm is used as a task allocation strategy to optimize the processing task allocation for a large-number of images. Experimental results show that the GA-based optimization can significantly speed up the image processing so that the proposed approach is promising to apply to process big data of images.

Copyright

© 2015, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - G.R. Zhang
AU  - Q.X. Wu
AU  - L.P. Huang
AU  - B.S. Chen
PY  - 2015/07
DA  - 2015/07
TI  - The Optimization of Task Assignments on Hadoop Platform for Large-Number Image Processing
BT  - Proceedings of the 2015 International Conference on Artificial Intelligence and Industrial Engineering
PB  - Atlantis Press
SP  - 52
EP  - 55
SN  - 1951-6851
UR  - https://doi.org/10.2991/aiie-15.2015.15
DO  - 10.2991/aiie-15.2015.15
ID  - Zhang2015/07
ER  -