What is That? Object Recognition from Natural Features on a Mobile Phone

Henze, Niels and Schinke, Torben and Boll, Susanne and
Proceedings of the Workshop on Mobile Interaction with the Real World
Connecting the physical and the digital world is an upcoming trend that enables numberless use-cases. The mobile phone as the most pervasive digital device is often used to establish such a connection. The phone enables users to retrieve, use, and share digital information and services connected to physical objects. Recognizing physical objects is thus a fundamental precondition for such applications. Object recognition is usually enabled using visual marker (e.g. Qr Codes) or electronic marker (e.g. RFID). Marker based approaches are not feasible for a large range of objects such as sights, photos, and persons. Markerless approaches that use the image stream from the mobile phone’s camera are commonly server-based which dramatically limits the interactiveness. Recent work on image processing shows that interactive object recognition on mobile phones is at hand. In this paper we present a markerless object recognition that processes multiple camera images per second on recent mobile phones. The algorithm combines a stripped down SIFT with a scalable vocabulary tree and a simple feature matching. Based on this algorithm we implemented a simple application which recognizes poster segments and conducted an initial user study to get an understanding of the implications that accompany markerless interaction.
09 / 2009
Interactive Media with Personal Networked Devices