Minerva : a smart video assistant for the kitchen

Abstract

Minerva, a video assistant for cooking, is controlled by the actions of the user. Specifically, the content and format of the video shown on Minerva's display are decided by the food that one places on the kitchen counter. Thus, it aims to develop new ways of interactivity and control for media that solve the problem of content choice in an innovative and useful manner. In parallel, it is a study of the applicability of a generalized image-matching algorithm [1] [21 and into reflecting a program's awareness of the context of its execution in its behavior. Minerva is, for the user, a personalized kitchen assistant, a culinary knowledge-base and a cooking tutor, all built into it by design rather than by accident. Minerva tackles problems faced by current computer systems, including trade-offs between automatic program execution and user control, access mechanisms for media from large databases, retrieval techniques for recipes and cooking information, and embedding computing systems in the home environment with useful features without compromising on non-obtrusiveness. The system has a camera that looks at the kitchen counter and a recognition engine that analyses the ingredient placed under the camera. Together, they form part of the input that decides the choice for the content of the media displayed. Contextual variables, including the user's identity and preferences form the other part of this input. A database look-up program that returns the location of the video and recipe image appropriate for these input forms the retrieval sub-system and a player for the cooking video retrieved displays the media chosen. Automatic control for the media is enabled wherever possible, allowing for prior information about the user to affect the choice and format of the cooking video displayed. This thesis describes the motivations, concept and implementation of Minerva, ending with an evaluation of its performance and impact, as well as directions for future work.