Prediction of Outfit Compatibility Based on Weighted Multimodal Fusion

DOI

10.2991/978-94-6463-108-1_2How to use a DOI?

Keywords

Multi-modal fusion; Outfit compatibility; Graph neural network; Visual feature; Textual features.

Abstract

Owing to the influence of too many complex factors between items, it is difficult to outfit compatibility. Although the current outfit compatibility technology has good results, however, previous works which focus on the compatibility of mod-eling based on the image mode or the text mode fail to make full use of the complex relations between text and image information interaction. To solve the clothing matching prediction of a single mode limit the variety of modes different information interaction, a method of clothing matching based on multimodal fusion with weights is proposed. Firstly, this method extracts clothing images as visual features, and at the same time extracts text information as textual features. Secondly, weighted fusion of the extracted visual features and text features. Finally, the fused features are used to input into the graph neural network model to learn the outfit compatibility. The fused features capture the most important clothing features into the clothing representation, thereby effectively improving the accuracy of outfit compatibility prediction.

Copyright

© 2022 The Author(s)

Open Access

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International License (http://creativecommons.org/licenses/by-nc/4.0/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

Download article (PDF)

TY  - CONF
AU  - Yan Li
AU  - Xiaobao Fu
AU  - You Du
PY  - 2022
DA  - 2022/12/30
TI  - Prediction of Outfit Compatibility Based on Weighted Multimodal Fusion
BT  - Proceedings of the 2022 International Conference on Computer Science, Information Engineering and Digital Economy (CSIEDE 2022)
PB  - Atlantis Press
SP  - 3
EP  - 14
SN  - 2352-538X
UR  - https://doi.org/10.2991/978-94-6463-108-1_2
DO  - 10.2991/978-94-6463-108-1_2
ID  - Li2022
ER  -