Proceedings of the 2023 4th International Conference on Artificial Intelligence and Education (ICAIE 2023)

<Previous Article In Volume
Next Article In Volume>

Design and Implementation of Big Data Training Project Platform Based on Hyper-converged Architecture

Authors
Jun Zheng1, Yu Bai2, Wei Huang3, *
1Guizhou Police College, Guiyang, China
2College of Mathematics and Information Science, Guiyang University, Guiyang, China
3Department of Information Engineering, Guizhou Light Industry Polytechnic, Guiyang, China
*Corresponding author. Email: 39189550@qq.com
Corresponding Author
Wei Huang
Available Online 22 September 2023.
DOI
10.2991/978-94-6463-242-2_73How to use a DOI?
Keywords
Hyper-converged; Big data; Training courses; Teaching research; Education and teaching reform
Abstract

Building high quality big data training courses is the necessary foundation and urgent need for training big data talents, and it is also an urgent task at present. In this study, hyper-converged architecture technology is used to design and implement a big data training project platform for college students, providing different simulation scenarios for big data training. The system software architecture design adopts the principle of loose coupling and hierarchical design, from bottom to top, it is mainly divided into data acquisition layer, physical layer, device virtualization layer, data layer, business processing layer, service layer, User Interface layer. The platform deploys private cloud OpenStack on CentOS7. The big data analysis process and technology are the data acquisition module (Scrapy and NoSQL), data cleaning module (Kettle), data mining and analysis module (Python scripts), and data visualization module (Power BI). The big data training project platform based on Openstack provides reference for the construction of related platforms.

Copyright
© 2024 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

<Previous Article In Volume
Next Article In Volume>
Volume Title
Proceedings of the 2023 4th International Conference on Artificial Intelligence and Education (ICAIE 2023)
Series
Atlantis Highlights in Computer Sciences
Publication Date
22 September 2023
ISBN
978-94-6463-242-2
ISSN
2589-4900
DOI
10.2991/978-94-6463-242-2_73How to use a DOI?
Copyright
© 2024 The Author(s)
Open Access
Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Cite this article

risenwbib
TY  - CONF
AU  - Jun Zheng
AU  - Yu Bai
AU  - Wei Huang
PY  - 2023
DA  - 2023/09/22
TI  - Design and Implementation of Big Data Training Project Platform Based on Hyper-converged Architecture
BT  - Proceedings of the 2023 4th International Conference on Artificial Intelligence and Education (ICAIE 2023)
PB  - Atlantis Press
SP  - 598
EP  - 603
SN  - 2589-4900
UR  - https://doi.org/10.2991/978-94-6463-242-2_73
DO  - 10.2991/978-94-6463-242-2_73
ID  - Zheng2023
ER  -