Spectral clustering algorithm based on Hadoop cloud platform research and implementation

DOI

10.2991/icamcs-16.2016.103How to use a DOI?

Keywords

cloud computing, MapReduce programming model, Hadoop cloud platform, spectral clustering.

Abstract

Spectral clustering algorithm based on the theory of spectrum, its meaning is the optimal clustering problem into graph partitioning problem is a point of clustering algorithms can be high-dimensional data set cluster after dimensionality reduction. Greatly reducing the time of clustering. Compared with the traditional clustering algorithm, spectral clustering which can have the advantage of clustering and converge to the global optimal solution in the sample space of arbitrary shape. However, the prevalence of large data sets are in the real world, when we want to clustering the spectral of large data sets, because the data is too large, the convergence rate will slow down, if not impossible to obtain results within the stipulated time we give us a lot of problems cluster. Thus, this paper based on Hadoop cloud platform to achieve large-scale clustering high-dimensional data sets. Experiments show that: spectral clustering algorithm after the parallel deployments running on Hadoop clusters, with good speedup and good scalability.

Copyright

© 2016, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - LiSheng Zhang
AU  - Ling Hou
AU  - DaJiang Lei
PY  - 2016/06
DA  - 2016/06
TI  - Spectral clustering algorithm based on Hadoop cloud platform research and implementation
BT  - Proceedings of the 2016 5th International Conference on Advanced Materials and Computer Science
PB  - Atlantis Press
SP  - 495
EP  - 498
SN  - 2352-5401
UR  - https://doi.org/10.2991/icamcs-16.2016.103
DO  - 10.2991/icamcs-16.2016.103
ID  - Zhang2016/06
ER  -