Feature-Weighting and Clustering Random Forest

Authors

Zhenyu Liu^{1, 2, *,} , Tao Wen^{1, 2}, Wei Sun², Qilong Zhang¹

¹College of Computer Science and Engineering, Northeastern University, Shenyang, China

²Department of Computer Science and Technology, Dalian Neusoft University of Information, Dalian, China

^*Corresponding author. Email: liuzhenyu@neusoft.edu.cn

DOI

10.2991/ijcis.d.201202.001How to use a DOI?

Keywords

Random forest; Feature weighting; Node split method; Categorical feature; Decision tree ensemble

Abstract

Classical random forest (RF) is suitable for the classification and regression tasks of high-dimensional data. However, the performance of RF may be not satisfied in case of few features, because univariate split method cannot bring more diverse individuals. In this paper, a novel method of node split of the decision trees is proposed, which adopts feature-weighting and clustering. This method can combine multiple numerical features, multiple categorical features or multiple mixed features. Based on the framework of RF, we use this split method to construct decision trees. The ensemble of the decision trees is called Feature-Weighting and Clustering Random Forest (FWCRF). The experiments show that FWCRF can get the better ensemble accuracy compared with the classical RF based on univariate decision tree on low-dimensional data, because FWCRF has better individual accuracy and lower similarity between individuals. Meanwhile, the empirical performance of FWCRF is not inferior to the classical RF and AdaBoost on high-dimensional data. Furthermore, compared with other multivariate RFs, the advantage of FWCRF is that it can directly deal with the categorical features, instead of the conversion from the categorical features to the numerical features.

Copyright

© 2021 The Authors. Published by Atlantis Press B.V.

Open Access

This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

TY  - JOUR
AU  - Zhenyu Liu
AU  - Tao Wen
AU  - Wei Sun
AU  - Qilong Zhang
PY  - 2020
DA  - 2020/12/07
TI  - Feature-Weighting and Clustering Random Forest
JO  - International Journal of Computational Intelligence Systems
SP  - 257
EP  - 265
VL  - 14
IS  - 1
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.d.201202.001
DO  - 10.2991/ijcis.d.201202.001
ID  - Liu2020
ER  -