Implementation of Map-Reduce Based Distributed System

DOI

10.2991/iiicec-15.2015.226How to use a DOI?

Keywords

Distributed System; Map-reduce; System Performance

Abstract

A distributed system consists of a collection of computing components. It enables computers to coordinate their behavior and functions. Also, as a model that processing large sets of data with a parallel, distributed algorithm on a cluster, Map-Reduce has been widely used during the past decades. In this project, a distributed system as the implementation of Map-Reduce proposed to remove duplicate lines from a text. It proved to work with high efficiency and it especially works well for a relatively large set of data which is unable to processed by a single host. After designing and implementing the required system, the evaluation of the system is performed and it also provides a further analysis by comparing with real Map-Reduce.

Copyright

© 2015, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Yidan Wang
AU  - Yi Liu
AU  - Boqi Gao
PY  - 2015/03
DA  - 2015/03
TI  - Implementation of Map-Reduce Based Distributed System
BT  - Proceedings of the 2015 International Industrial Informatics and Computer Engineering Conference
PB  - Atlantis Press
SP  - 1014
EP  - 1017
SN  - 2352-538X
UR  - https://doi.org/10.2991/iiicec-15.2015.226
DO  - 10.2991/iiicec-15.2015.226
ID  - Wang2015/03
ER  -