Skip to main content
SpringerLink
Log in

Hyperlinking reality via camera phones

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

A novel user interface concept for camera phones, called “Hyperlinking Reality via Camera Phones”, that we present in this article, provides a solution to one of the main challenges facing mobile user interfaces, that is, the problem of selection and visualization of actions that are relevant to the user in her current context. Instead of typing keywords on a small and inconvenient keypad of a mobile device, a user of our system just snaps a photo of her surroundings and objects in the image become hyperlinks to information. Our method commences by matching a query image to reference panoramas depicting the same scene that were collected and annotated with information beforehand. Once the query image is related to the reference panoramas, we transfer the relevant information from the reference panoramas to the query image. By visualizing the information on the query image and displaying it on the camera phone’s (multi-)touch screen, the query image augmented with hyperlinks allows the user intuitive access to information.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Kendall, P.: Worldwide cellular user forecasts, 2008–2013. Tech. rep., Strategy Analytics Inc. (2008)

  2. Lindholm C., Keinonen T., Kiljander H.: Mobile Usability: How Nokia Changed the Face of the Mobile Phone. McGraw-Hill, New York (2003)

    Google Scholar 

  3. Kähäri, M., Murphy, D.J.: MARA—Sensor based augmented reality system for mobile imaging. http://research.nokia.com/research/projects/mara/ (2006). Nokia Research Center

  4. Greene, K.: Hyperlinking reality via phones. MIT Technology Review (11–12) (2006)

  5. Chen, B.X.: If youre not seeing data, youre not seeing. http://www.wired.com/gadgetlab/2009/08/augmented-reality/

  6. Steinhoff, U., Omerčević, D., Perko, R., Schiele, B., Leonardis, A.: How computer vision can help in outdoor positioning. In: European Conference on Ambient Intelligence, vol. 4794, pp. 124–141. Springer LNCS, Berlin (2007)

  7. Höller, N., Geven, A., Tscheligi, M., Paletta, L., Amlacher, K., Luley, P., Omerčević, D.: Exploring the urban environment with a camera phone: lessons from a user study. In: Proceedings of the 11th International Conference on Human–Computer Interaction with Mobile Devices and Services (MobileHCI) (2009)

  8. Mawston, N.: Enabling technologies: CMOS beats CCD in half-billion global camera phone market. Tech. rep., Strategy Analytics Inc. (2007)

  9. Reynolds F.: Camera phones: a snapshot of research and applications. Pervasive Comput. IEEE 7(2), 16–19 (2008)

    Article  Google Scholar 

  10. Yeh, T., Tollmar, K., Darrell, T.: Searching the web with mobile images for location recognition. In: Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pp. 76–81 (2004)

  11. Wang, J., Zhai, S., Canny, J.: Camera phone based motion sensing: interaction techniques, applications and performance study. In: UIST ’06: Proceedings of the 19th annual ACM symposium on User interface software and technology, pp. 101–110 (2006)

  12. Cuellar, G., Eckles, D., Spasojevic, M.: Photos for information: a field study of cameraphone computer vision interactions in tourism. In: CHI’08: CHI’08 Extended Abstracts on Human Factors in Computing Systems, pp. 3243–3248 (2008)

  13. Takacs, G., Chandrasekhar, V., Gelfand, N., Xiong, Y., Chen, W.C., Bismpigiannis, T., Grzeszczuk, R., Pulli, K., Girod, B.: Outdoors augmented reality on mobile phone using loxel-based visual feature organization. In: Proceeding of the 1st ACM International Conference on Multimedia information retrieval—MIR’08, pp. 427–434 (2008)

  14. Henrysson, A.: Bringing augmented reality to mobile phones. Ph.D. thesis, Linköping University (2007)

  15. Schmid C., Mohr R.: Local grayvalue invariants for image retrieval. IEEE PAMI 19(5), 530–535 (1997)

    Google Scholar 

  16. Lowe D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91–110 (2004)

    Article  Google Scholar 

  17. Nistér, D., Stewénius, H.: Scalable recognition with a vocabulary tree. In: CVPR, vol. 2, pp. 2161–2168 (2006)

  18. Chum, O., Philbin, J., Sivic, J., Isard, M., Zisserman, A.: Total recall: automatic query expansion with a generative feature model for object retrieval. In: ICCV (2007)

  19. Baumberg, A.: Reliable feature matching across widely separated views. In: CVPR, vol. 1, pp. 774–781 (2000)

  20. Matas J., Chum O., Urban M., Pajdla T.: Robust wide-baseline stereo from maximally stable extremal regions. Image Vis. Comput. 22(10), 761–767 (2004)

    Article  Google Scholar 

  21. Tuytelaars T., Van Gool L.: Matching widely separated views based on affine invariant regions. IJCV 59(1), 61–85 (2004)

    Article  Google Scholar 

  22. Brown M., Lowe D.G.: Automatic panoramic image stitching using invariant features. IJCV 74(1), 59–73 (2007)

    Article  Google Scholar 

  23. Zhang, W., Košecká, J.: Image based localization in urban environments. In: International Symposium on 3D Data Processing, Visualization and Transmission, pp. 33–40 (2006)

  24. Sivic, J., Zisserman, A.: Video Google: a text retrieval approach to object matching in videos. In: ICCV, vol. 2, pp. 1470–1477 (2003)

  25. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge. ISBN: 0521540518 (2004)

  26. Tuytelaars, T.: A survey on local invariant features. Tutorial at ECCV2006 (2006)

  27. Omerčević, D., Drbohlav, O., Leonardis, A.: High-dimensional feature matching: employing the concept of meaningful nearest neighbors. In: ICCV (2007)

  28. Mikolajczyk K., Tuytelaars T., Schmid C., Zisserman A., Matas J., Schaffalitzky F., Kadir T., Van Gool L.: A comparison of affine region detectors. IJCV 65(1–2), 43–72 (2005)

    Article  Google Scholar 

  29. Moreels, P., Perona, P.: Evaluation of features detectors and descriptors based on 3D objects. In: ICCV, vol. 1, pp. 800–807 (2005)

  30. Mikolajczyk K., Schmid C.: A performance evaluation of local descriptors. IEEE PAMI 27(10), 1615–1630 (2005)

    Google Scholar 

  31. Fischler M.A., Bolles R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  32. Grauman, K., Darrell, T.: Approximate correspondences in high dimensions. In: NIPS 19, pp. 505–512 (2007)

  33. Böhm C., Berchtold S., Keim D.A.: Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput. Surv. (CSUR) 33(3), 322–373 (2001)

    Article  Google Scholar 

  34. Indyk P.: Nearest neighbors in high-dimensional spaces. In: Goodman, J.E., ORourke, J. (eds) Handbook of Discrete and Computational Geometry, 2nd edn., Chap. 39, CRC Press, Boca Raton (2004)

    Google Scholar 

  35. Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: Proceedings of the International Conference on Computer Vision and Applications (2009)

  36. Silpa-Anan, C., Hartley, R.: Optimised KD-trees for fast image descriptor matching. In: CVPR (2008)

  37. Nister D.: An efficient solution to the five-point relative pose problem. IEEE PAMI 26(6), 756–777 (2004)

    Google Scholar 

  38. Stewénius H., Engels C., Nistér D.: Recent developments on direct relative orientation. ISPRS J. Photogram. Remote Sens. 60, 284–294 (2006)

    Article  Google Scholar 

  39. Torr P.H.S., Zisserman A.: Mlesac: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. 78(1), 138–156 (2000)

    Article  Google Scholar 

  40. Tordoff B.J., Murray D.W.: Guided-mlesac: faster image transform estimation by using matching priors. IEEE PAMI 27(10), 1523–1535 (2005)

    Google Scholar 

  41. Strecha, C., Tuytelaars, T., Van Gool, L.: Dense matching of multiple wide-baseline views. In: ICCV (2003)

  42. Torr, P., Murray, D.: Outlier detection and motion segmentation. In: Proceedings of SPIE (1993)

  43. Ma Y., Soatto S., Kosecka J., Sastry S.S.: An Invitation to 3-D Vision: From Images to Geometric Models. Springer, Berlin (2003)

    Google Scholar 

  44. Su J., Chung R., Jin L.: Homography-based partitioning of curved surface for stereo correspondence establishment. Pattern Recognit. Lett. 28(12), 1459–1471 (2007)

    Article  Google Scholar 

  45. Vincent, E., Laganiere, R.: Detecting planar homographies in an image pair. In: Image and Signal Processing and Analysis, 2001. ISPA 2001. In: Proceedings of the 2nd International Symposium on, pp. 182–187 (2001)

  46. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment—a modern synthesis. In: ICCV ’99: Proceedings of the International Workshop on Vision Algorithms, pp. 298–372 (2000)

  47. Azuma R.T.: A survey of augmented reality. Presence Teleoper. Virtual Environ. 6(4), 355–385 (1997)

    Google Scholar 

  48. Davison, A.J.: Real-time simultaneous localisation and mapping with a single camera. In: ICCV (2003)

  49. Vincent, L.: Taking online maps down to street level. Computer 40(12) (2007)

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer and Information Science, University of Ljubljana, Tržaška cesta 25, Ljubljana, Slovenia

    Dušan Omerčević & Aleš Leonardis

Authors
  1. Dušan Omerčević
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aleš Leonardis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dušan Omerčević.

Additional information

This research has been supported in part by: Research program Computer Vision P2-0214 (RS), EU FP6-004250-IP project CoSy, EU MRTN-CT-2004-005439 project VISIONTRAIN, and EU FP6-511051project MOBVIS.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Omerčević, D., Leonardis, A. Hyperlinking reality via camera phones. Machine Vision and Applications 22, 521–534 (2011). https://doi.org/10.1007/s00138-010-0285-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-010-0285-9

Keywords

Search

Navigation