International Journal of Networked and Distributed Computing

Next Article In Issue>
Volume 6, Issue 4, September 2018, Pages 195 - 203

Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform

Authors
Jongyeop Kim1, *, Abhilash Kancharla2, Jongho Seol2, Indy N. Park3, Nohpill Park2
1Math and Computer Science, Southern Arkansas University, Magnolia, AR 71753, USA
2Computer Science, Oklahoma State University, Stillwater, OK USA
3Computer Science, Oklahoma City University, Oklahoma City, OK, USA
* Corresponding author. Email:Jkim@saumag.edu
Corresponding Author
Jongyeop Kim
Received 27 June 2018, Accepted 20 September 2018, Available Online 28 September 2018.
DOI
10.2991/ijndc.2018.6.4.1How to use a DOI?
Keywords
Big data; Hadoop; configuration; performance tuning
Abstract

This research proposes the methodology to extract common configuration parameter set by applying multiple benchmarking applications include TeraSort, TestDFSIO, and MrBench on the Hadoop distributed file system. The parameter search space conceptually conducted named Ω(x) to hold status of all parameter values and its evaluation results for every stage to eventually reduce benchmarking cost. In the process of determining parameter set for each stage, one parameter and its associated values selected which is reduced system performance in terms of overall execution time difference that are measured by multiple applications on a Hadoop cluster. The experimental results demonstrate the proposed extended greedy manner provide a feasible benchmark model for the multiple MapReduce tasks. This model classified several candidate parameter value sets that can be reduced the overall execution time by 27% of the values against Hadoop default settings. Moreover, we propose e-heuristic greedy with alternative parameter selection model to evaluate second candidate parameter value which will lead global optimum by returning back to the previous stage if local minimum is not found at the current stage compare to the previous ones.

Copyright
© 2018 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

Next Article In Issue>
Journal
International Journal of Networked and Distributed Computing
Volume-Issue
6 - 4
Pages
195 - 203
Publication Date
2018/09/28
ISSN (Online)
2211-7946
ISSN (Print)
2211-7938
DOI
10.2991/ijndc.2018.6.4.1How to use a DOI?
Copyright
© 2018 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article under the CC BY-NC license (http://creativecommons.org/licences/by-nc/4.0/).

Cite this article

risenwbib
TY  - JOUR
AU  - Jongyeop Kim
AU  - Abhilash Kancharla
AU  - Jongho Seol
AU  - Indy N. Park
AU  - Nohpill Park
PY  - 2018
DA  - 2018/09/28
TI  - Optimized Common Parameter Set Extraction Framework by Multiple Benchmarking Applications on a Big Data Platform
JO  - International Journal of Networked and Distributed Computing
SP  - 195
EP  - 203
VL  - 6
IS  - 4
SN  - 2211-7946
UR  - https://doi.org/10.2991/ijndc.2018.6.4.1
DO  - 10.2991/ijndc.2018.6.4.1
ID  - Kim2018
ER  -