Volume 12, Issue 2, 2019, Pages 1613 - 1621
Character-Level Quantum Mechanical Approach for a Neural Language Model
Authors
Zhihao Wang1, *, Min Ren2, Xiaoyan Tian3, Xia Liang1
1School of Management Science and Engineering, Shandong University of Finance and Economics, Jinan 250014, China
2School of Mathematics and Quantitative Economics, Shandong University of Finance and Economics, Jinan 250014, China
3Shandong Police College, Jinan 250014, China
*Corresponding author. Email: wzh861125@hotmail.com
Corresponding Author
Zhihao Wang
Received 22 April 2019, Accepted 9 November 2019, Available Online 25 November 2019.
- DOI
- 10.2991/ijcis.d.191114.001How to use a DOI?
- Keywords
- Character-level; Quantum theory; Network-in-network; Language model
- Abstract
This article proposes a character-level neural language model (NLM) that is based on quantum theory. The input of the model is the character-level coding represented by the quantum semantic space model. Our model integrates a convolutional neural network (CNN) that is based on network-in-network (NIN). We assessed the effectiveness of our model through extensive experiments based on the English-language Penn Treebank dataset. The experiments results confirm that the quantum semantic inputs work well for the language models. For example, the PPL of our model is 10%–30% less than the states of the arts, while it keeps the relatively smaller number of parameters (i.e., 6 m).
- Copyright
- © 2019 The Authors. Published by Atlantis Press SARL.
- Open Access
- This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).
Download article (PDF)
View full text (HTML)
Cite this article
TY - JOUR AU - Zhihao Wang AU - Min Ren AU - Xiaoyan Tian AU - Xia Liang PY - 2019 DA - 2019/11/25 TI - Character-Level Quantum Mechanical Approach for a Neural Language Model JO - International Journal of Computational Intelligence Systems SP - 1613 EP - 1621 VL - 12 IS - 2 SN - 1875-6883 UR - https://doi.org/10.2991/ijcis.d.191114.001 DO - 10.2991/ijcis.d.191114.001 ID - Wang2019 ER -