A Scheme of Implementing PCA Alogrithm on Storm Platform

DOI

10.2991/icmmita-15.2015.125How to use a DOI?

Keywords

Data Stream; Dimensional Reduction; Storm; PCA; Distribution and Parallelization

Abstract

In order to sovle the problem of dimension disaster when mining the high-dimensional data in the data stream and the problem of poor real-time response and insufficient system throughput of dimensional reduction algorithms, a scheme of implementing PCA algorithm on Storm platform is designed. This scheme programs each branch of PCA algorithm by using Storm’s own components, and each component forms the task entity through data flow communication. The scheme realizes the alogrithm distribution and parallelization by setting the threads number and the process number of task entity. Experimental results of running PCA algorithm on Storm and computer cluster according to the scheme show that the PCA algorithm on Storm platform can meet the requirement of real-time dimensional reduction of data stream.

Copyright

© 2015, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Yan Shan
AU  - Lingjuan Li
AU  - Yimu Ji
PY  - 2015/11
DA  - 2015/11
TI  - A Scheme of Implementing PCA Alogrithm on Storm Platform
BT  - Proceedings of the 2015 3rd International Conference on Machinery, Materials and Information Technology Applications
PB  - Atlantis Press
SP  - 644
EP  - 648
SN  - 2352-538X
UR  - https://doi.org/10.2991/icmmita-15.2015.125
DO  - 10.2991/icmmita-15.2015.125
ID  - Shan2015/11
ER  -