Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images
Name
tpami19.pdf
Description
Accepted version
Size
16.37 MB
Format
Adobe PDF
Checksum (MD5)
c4f2ea26d52547c2162b20302d3b6e1c
Author(s)
Marin, Javier
Biswas, Aritro
Ofli, Ferda
Hynes, Nicholas
Salvador, Amaia
Aytar, Yusuf
Weber, Ingmar
Torralba, Antonio
Date Issued
January 2021
Journal
IEEE Transactions on Pattern Analysis and Machine Intelligence
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Citation
Marin, Javier et al. "Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images." IEEE Transactions on Pattern Analysis and Machine Intelligence (January 2021): 187 - 203 © 2021 IEEE
Version
Author's final manuscript
Abstract
In this paper, we introduce Recipe1M+, a new large-scale, structured corpus of over one million cooking recipes and 13 million food images. As the largest publicly available collection of recipe data, Recipe1M+ affords the ability to train high-capacity models on aligned, multimodal data. Using these data, we train a neural network to learn a joint embedding of recipes and images that yields impressive results on an image-recipe retrieval task. Moreover, we demonstrate that regularization via the addition of a high-level classification objective both improves retrieval performance to rival that of humans and enables semantic vector arithmetic. We postulate that these embeddings will provide a basis for further exploration of the Recipe1M+ dataset and food and cooking in general. Code, data and models are publicly available.11.http://im2recipe.csail.mit.edu.
MIT Department
Massachusetts Institute of Technology. Computer Science and Artificial Intelligence Laboratory
Massachusetts Institute of Technology. Department of Electrical Engineering and Computer Science
Terms of Use
Creative Commons Attribution-Noncommercial-Share Alike
Persistent DSpace Link
DOI of Published Version
https://doi.org/10.1109/tpami.2019.2927476
