Test Spark Installation on Linux Clusters

DOI

10.2991/eame-18.2018.37How to use a DOI?

Keywords

spark cluster; big data; cloud computing

Abstract

Apache Spark is an open source big data analytic framework on clusters. It runs on top of Hadoop distributed clusters but faster using in-memory processing. Very often it seems run normally even Hadoop runs abnormally. This paper describes that after install Spark 1.4.0 over Hadoop 2.7.0, how to test both Hadoop and Spark on CentOS 7.0 Linux clusters.

Copyright

© 2018, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Lee-Hur Shing
AU  - Lee-Pin Shing
AU  - Marn-Ling Shing
AU  - Chen-Chi Shing
PY  - 2018/06
DA  - 2018/06
TI  - Test Spark Installation on Linux Clusters
BT  - Proceedings of the 2018 3rd International Conference on Electrical, Automation and Mechanical Engineering (EAME 2018)
PB  - Atlantis Press
SP  - 177
EP  - 181
SN  - 2352-5401
UR  - https://doi.org/10.2991/eame-18.2018.37
DO  - 10.2991/eame-18.2018.37
ID  - Shing2018/06
ER  -