A Study of Sentence Similarity Based on the All-minilm-l6-v2 Model With “Same Semantics, Different Structure” After Fine Tuning

DOI

10.2991/978-94-6463-540-9_69How to use a DOI?

Keywords

all; MiniLM; L6; v2; meta learning; few; shot learning; sentence similarity

Abstract

Traditional natural language processing models often find it difficult to distinguish between sentences with “similar structure and different semantics” and sentences with “different structure and similar semantics”. Based on the all-MiniLM-L6-v2 and Bidirectional Encoder Representations from Transformers (BERT) model, this paper uses supervised learning and transfer learning methods to study the similarity of sentences with “similar structure, different semantics” and “different structure, similar semantics”. New datasets in medical aspects with the same format as the hard datasets are artificially constructed and used as subdivided small-volume datasets to verify the model performance, thus simulating the needs of specific fields. On the basis of meta-learning and small number of shots learning, different models are fine-tuned, and good verification results are obtained and compared. For the fine-tuned models, the performance has been improved, among which the most significant improvements are: BERT model: accuracy: 0.51 to 0.65, all-MiniLM-L6-v2 model: precision:0.74 to 0.91 and so on. In this paper, the supervised learning method is used to provide effective ideas and directions for sentence similarity division of “semantically similar, structurally different” and “semantically different, structurally similar”. This optimization can be proved to be effective and necessary.

Copyright

© 2024 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Chen Yin
AU  - Zixuan Zhang
PY  - 2024
DA  - 2024/10/16
TI  - A Study of Sentence Similarity Based on the All-minilm-l6-v2 Model With “Same Semantics, Different Structure” After Fine Tuning
BT  - Proceedings of the 2024 2nd International Conference on Image, Algorithms and Artificial Intelligence (ICIAAI 2024)
PB  - Atlantis Press
SP  - 677
EP  - 684
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-540-9_69
DO  - 10.2991/978-94-6463-540-9_69
ID  - Yin2024
ER  -