International Journal of Computational Intelligence Systems

Volume 8, Issue 6, December 2015, Pages 1091 - 1102

Materialized View Selection Based on Adaptive Genetic Algorithm and Its Implementation with Apache Hive

Authors
Dongjin Yu, Wensheng Dou, Zhixiang Zhu, Jiaojiao Wang
Corresponding Author
Dongjin Yu
Received 28 December 2014, Accepted 14 September 2015, Available Online 1 December 2015.
DOI
10.1080/18756891.2015.1113744How to use a DOI?
Keywords
materialized view, multi-dimensional lattice, genetic algorithm, cost model, adaptive, Apache Hive
Abstract

Frequently accessed views in data warehouses are usually materialized in order to accelerate the speed of querying big data. However, the view materialization itself incurs huge costs. Moreover, some latest products of non-traditional data warehouse software, such as Apache Hive, still lack the support of ma- terialized views. In order to select the appropriate views to be materialized with the possible minimized cost, we propose a novel approach to the materialized view selection problem based on an adaptive ge- netic algorithm. We establish a cost model that integrates the query, maintenance and storage costs to evaluate the performance of approaches and measure the fitness of an individual in the genetic algorithm. In addition, we introduce the adjustable factors for crossover probability and mutation probability, allow- ing the genetic algorithm to run quickly and avoid premature convergence. We also conduct extensive experiments for its implementation with Apache Hive, which query and manage large datasets residing in distributed storage. Both the simulation results and experiments on Apache Hive show that the approx- imately optimal solution for selecting materialized views can be obtained effectively using the approach presented.

Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Journal
International Journal of Computational Intelligence Systems
Volume-Issue
8 - 6
Pages
1091 - 1102
Publication Date
2015/12/01
ISSN (Online)
1875-6883
ISSN (Print)
1875-6891
DOI
10.1080/18756891.2015.1113744How to use a DOI?
Copyright
© 2017, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - JOUR
AU  - Dongjin Yu
AU  - Wensheng Dou
AU  - Zhixiang Zhu
AU  - Jiaojiao Wang
PY  - 2015
DA  - 2015/12/01
TI  - Materialized View Selection Based on Adaptive Genetic Algorithm and Its Implementation with Apache Hive
JO  - International Journal of Computational Intelligence Systems
SP  - 1091
EP  - 1102
VL  - 8
IS  - 6
SN  - 1875-6883
UR  - https://doi.org/10.1080/18756891.2015.1113744
DO  - 10.1080/18756891.2015.1113744
ID  - Yu2015
ER  -