Jun Harashima
ABSTRACT
In food-related services, image information is as important as text information for users. For example, in recipe search services, users find recipes based not only on text but also images. To promote studies on food images, many datasets have recently been published. However, they have the following three limitations: most of the datasets include only thousands of images, they only take account of images after cooking not during the cooking process, and the images are not linked to any recipes. In this study, we construct the Cookpad Image Dataset, a novel collection of food images taken from Cookpad, the largest recipe search service in the world. The dataset includes more than 1.64 million images after cooking, and it is the largest among existing datasets. Additionally, it includes more than 3.10 million images taken during the cooking process. To the best of our knowledge, there are no datasets that include such images. Furthermore, the dataset is designed to link to an existing recipe corpus and thus, a variety of recipe texts, such as the title, description, ingredients, and process, is available for each image. In this paper, we described our dataset's features in detail and compared it with existing datasets.
- Oscar Beijbom, Neel Joshi, Dan Morris, Scott Saponas, and Siddharth Khullar. 2015. Menu-Match: Restaurant-Specific Food Logging from Images Proceedings of the 2015 IEEE Winter Conference on Applications of Computer Vision (WACV 2015). 844--851.Google Scholar
- Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101 - Mining Discriminative Components with Random Forests Proceedings of the 13th European Conference on Computer Vision (ECCV 2014). 446--461.Google Scholar
- Jingjing Chen and Chong-Wah Ngo. 2016. Deep-based Ingredient Recognition for Cooking Recipe Retrieval Proceedings of the 2016 ACM on Multimedia Conference (ACMMM 2016). 32--41.Google Scholar
- Mei Chen, Kapil Dhingra, Wen Wu, Lei Yang, Rahul Sukthankar, and Jie Yang. 2009. PFID: Pittsburgh Fast-Food Image Dataset. In Proceedings of the 16th IEEE International Conference on Image Processing (ICIP 2009). 289--292. Google ScholarCross Ref
- Mei-Yun Chen, Yung-Hsiang Yang, Chia-Ju Ho, Shih-Han Wang, Shane-Ming Liu, Eugene Chang, Che-Hua Yeh, and Ming Ouhyoung. 2012. Automatic Chinese Food Identification and Quantity Estimation Proceedings of the 5th ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia 2012).Google Scholar
- Gianluigi Ciocca, Paolo Napoletano, and Raimondo Schettini. 2015. Food Recognition and Leftover Estimation for Daily Diet Monitoring New Trends in Image Analysis and Processing - ICIAP 2015 Workshops. 334--341.Google Scholar
- Gianluigi Ciocca, Paolo Napoletano, and Raimondo Schettini. 2017. Food Recognition: a New Dataset, Experiments and Results. IEEE Journal of Biomedical and Health Informatics (2017).Google Scholar
- Giovanni Maria Farinella, Dario Allegra, and Filippo Stanco. 2014. A Benchmark Dataset to Study the Representation of Food Images. Proceedings of the 13th ECCV Workshop on Assistive Computer Vision and Robotics (ACVR 2014). 584--599.Google Scholar
- Jun Harashima, Michiaki Ariga, Kenta Murata, and Masayuki Ioki. 2016. A Large-Scale Recipe and Meal Data Collection as Infrastructure for Food Research. Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016). 2455--2459.Google Scholar
- Hongsheng He, Fanyu Kong, and Jindong Tan 2016. DietCam: Multi-View Food Recognition Using a Multi-Kernel SVM. IEEE Journal of Biomedical and Health Informatics, Vol. 20, 3 (2016), 848--855. Google ScholarCross Ref
- Hokuto Kagaya and Kiyoharu Aizawa. 2015. Highly Accurate Food/Non-Food Image Classification Based on a Deep Convolutional Neural Network. New Trends in Image Analysis and Processing - ICIAP 2015 Workshops. 350--357.Google ScholarDigital Library
- Yoshiyuki Kawano and Keiji Yanai. 2014. Automatic Expansion of a Food Image Dataset Leveraging Existing Categories with Domain Adaptation. In Proceedings of the 13th ECCV Workshop on Transferring and Adapting Source Knowledge in Computer Vision (TASK-CV 2014). 3--17.Google Scholar
- Chang Liu, Yu Cao, Yan Luo, Guanling Chen, Vinod Vokkarane, and Yunsheng Ma. 2016. DeepFood: Deep Learning-Based Food Image Recognition for Computer-Aided Dietary Assessment. In Proceedings of the 14th International Conference on Inclusive Smart Cities and Digital Health (ICOST 2016). 37--48. Google ScholarDigital Library
- Jonathan Malmaud, Jonathan Huang, Vivek Rathod, Nick Johnston, Andrew Rabinovich, and Kevin Murphy. 2015. What's Cookin? Interpreting Cooking Videos using Text, Speech and Vision. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL 2015). 143--152.Google ScholarCross Ref
- Yuji Matsuda, Hajime Hoashi, and Keiji Yanai. 2012. Recognition of Multiple-Food Images by Detecting Candidate Regions. Proceedings of the 2012 IEEE International Conference on Multimedia and Expo (ICME 2012). 25--30. Google ScholarDigital Library
- Austin Myers, Nick Johnston, Vivek Rathod, Anoop Korattikara, and Alex Gorban. 2015. Im2Calories: towards an automated mobile vision food diary. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV 2015). 1233--1241. Google ScholarDigital Library
- Rakuten Institute of Technology. 2010. Rakuten Data Release. (2010). http://rit.rakuten.co.jp/opendata.html.Google Scholar
- Kyoko Sudo, Jun Shimamura, Kazuhiko Murasaki, and Yukinobu Taniguchi. 2014. Estimating nutritional value from food images based on semantic segmentation. Proceedings of the Workshop on Smart Technology for Cooking Eating Activities (CEA 2014). 571--576. Google ScholarDigital Library
- Keiji Yanai and Yoshiyuki Kawano. 2015. Food Image Recognition using Deep Convolutional Network with Pre-training and Fine-tuining. Proceedings of the 7th Workshop on Multimedia for Cooking and Eating Activities (CEA 2015). 1--6.Google Scholar
Index Terms
- Cookpad Image Dataset: An Image Collection as Infrastructure for Food Research
Recommendations
Estimating nutritional value from food images based on semantic segmentation
UbiComp '14 Adjunct: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct PublicationEstimating the nutritional value of food based on image recognition is important to health support services employing mobile devices. The estimation accuracy can be improved by recognizing regions of food objects and ingredients contained in those ...
MIAIS: A Multimedia Recipe Dataset with Ingredient Annotation at Each Instructional Step
CEA++ '22: Proceedings of the 1st International Workshop on Multimedia for Cooking, Eating, and related APPlicationsIn this paper, we introduce a multimedia recipe dataset with annotation of ingredients at every instructional step, named MIAIS (Multimedia recipe dataset with Ingredient Annotation at every Instructional Step). One unique feature of recipe data is that ...
ML-CookGAN: Multi-Label Generative Adversarial Network for Food Image Generation
Generating food images from recipe and ingredient information can be applied to many tasks such as food recommendation, recipe development, and health management. For the characteristics of food images, this paper proposes ML-CookGAN, a novel CGAN. This ...
Comments