The Development and Application of “One-Stop” Cluster Analysis Application System Under the Background of Big Data

DOI

10.2991/978-94-6463-064-0_28How to use a DOI?

Keywords

Big data; Hadoop; Spark; Clustering application system

Abstract

In the era of big data, the application level of data analysis and processing algorithm determines the presentation of data value. As one of the widely used exploratory analyses, clustering analysis has many ways to realize it, but users still face many difficulties in the actual application process. For this reason, this paper expounds the design and development of functional modules of clustering analysis and processing algorithm based on Hadoop framework and Spark platform, focusing on K-Means clustering algorithm and BIRCH clustering algorithm, and combining Java language development environment to complete the construction of “one-stop” clustering analysis application system. The system is designed in the form of Web application, and the specific operation steps involved in cluster analysis, such as data collection, data cleaning, data storage, data analysis and mining, are highly encapsulated. The special API interface is opened to the outside world, which can widely support all kinds of users. Through simple operation, the cluster analysis of massive data content can be completed, and the data analysis results can be obtained intuitively through data visualization. It not only greatly improves the work efficiency of data analysis, processing and calculation, but also improves the high cost and non-general situation of previous big data clustering analysis systems, and further expands the use dimensions and application scenarios of big data.

Copyright

© 2023 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Zhengju Song
AU  - Jun Li
PY  - 2022
DA  - 2022/12/27
TI  - The Development and Application of “One-Stop” Cluster Analysis Application System Under the Background of Big Data
BT  - Proceedings of the 2022 3rd International Conference on Big Data and Social Sciences (ICBDSS 2022)
PB  - Atlantis Press
SP  - 235
EP  - 246
SN  - 2589-4900
UR  - https://doi.org/10.2991/978-94-6463-064-0_28
DO  - 10.2991/978-94-6463-064-0_28
ID  - Song2022
ER  -