Interval Type-2 Fuzzy Systems as Deep Neural Network Activation Functions

DOI

10.2991/eusflat-19.2019.39How to use a DOI?

Keywords

Interval type-2 fuzzy system Footprint of Uncertainty Activation unit Deep learning

Abstract

In this paper, we propose a novel activation function, namely, Interval Type-2 Fuzzy (IT2) Rectifying Unit (FRU), to improve the performance of the Deep Neural Networks (DNNs). The IT2-FRU can generate linear or sophisticated activation functions by simply tuning the size of the footprint of uncertainty of the IT2 Fuzzy Sets. The novel IT2-FRU also alleviates vanishing gradient problem and has a fast convergence rate since it pushes the mean activation to zero by allowing the negative outputs. In order to test the performance of the IT2-FRU, comparative experimental studies are performed on the CIFAR-10 dataset. IT2-FRU is compared with widely used conventional activation functions. Experimental results show that IT2-FRU significantly speeds up the learning and has a superior performance compared to other activation functions.

Copyright

© 2019, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Aykut Beke
AU  - Tufan Kumbasar
PY  - 2019/08
DA  - 2019/08
TI  - Interval Type-2 Fuzzy Systems as Deep Neural Network Activation Functions
BT  - Proceedings of the 11th Conference of the European Society for Fuzzy Logic and Technology (EUSFLAT 2019)
PB  - Atlantis Press
SP  - 267
EP  - 273
SN  - 2589-6644
UR  - https://doi.org/10.2991/eusflat-19.2019.39
DO  - 10.2991/eusflat-19.2019.39
ID  - Beke2019/08
ER  -