“Never fry carrots without chopping” Generating Cooking Recipes from Cooking Videos Using Deep Learning Considering Previous Process

DOI

10.2991/ijndc.k.190710.002How to use a DOI?

Keywords

Image captioning; deep learning; cooking; recipe; Doc2Vec

Abstract

Research on deep-training captioning models that modify the natural-language contents of images and moving images has produced considerable results and attracted attention in recent years. In this research, we aim to generate recipe sentences from cooking videos acquired from YouTube. We treat this as an image-captioning task and propose two methods suitable for the work. We propose a method that adds a vector of a sentence already generated in the same recipe to the input of a captioning model. Then, we compare generated and correct sentences to calculate scores. We also propose a data-processing method to improve accuracy. We use several widely used metrics to evaluate image-captioning problems. We then train the same data with the simplest encoder–decoder model, compare it with correct recipe sentences, and calculate the metrics. The results indicate that our proposed methods help increase accuracy.

Copyright

© 2019 The Authors. Published by Atlantis Press SARL.

Open Access

This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

TY  - JOUR
AU  - Tatsuki Fujii
AU  - Yuichi Sei
AU  - Yasuyuki Tahara
AU  - Ryohei Orihara
AU  - Akihiko Ohsuga
PY  - 2019
DA  - 2019/07/23
TI  - “Never fry carrots without chopping” Generating Cooking Recipes from Cooking Videos Using Deep Learning Considering Previous Process
JO  - International Journal of Networked and Distributed Computing
SP  - 107
EP  - 112
VL  - 7
IS  - 3
SN  - 2211-7946
UR  - https://doi.org/10.2991/ijndc.k.190710.002
DO  - 10.2991/ijndc.k.190710.002
ID  - Fujii2019
ER  -