skip to main content
10.1145/1460096.1460165acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Outdoors augmented reality on mobile phone using loxel-based visual feature organization

Published: 30 October 2008 Publication History

Abstract

We have built an outdoors augmented reality system for mobile phones that matches camera-phone images against a large database of location-tagged images using a robust image retrieval algorithm. We avoid network latency by implementing the algorithm on the phone and deliver excellent performance by adapting a state-of-the-art image retrieval algorithm based on robust local descriptors. Matching is performed against a database of highly relevant features, which is continuously updated to reflect changes in the environment. We achieve fast updates and scalability by pruning of irrelevant features based on proximity to the user. By compressing and incrementally updating the features stored on the phone we make the system amenable to low-bandwidth wireless connections. We demonstrate system robustness on a dataset of location-tagged images and show a smart-phone implementation that achieves a high image matching rate while operating in near real-time.

References

[1]
R. Datta, J. Li, and J. Z. Wang, "Content-based image retrieval: approaches and trends of the new age," in MIR '05: Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. New York, NY, USA: ACM, 2005, pp. 253--262.
[2]
R. Fergus, P. Perona, and A. Zisserman, "Object class recognition by unsupervised scale-invariant learning," in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR), vol. 02. IEEE Computer Society, 2003, p. 264.
[3]
A. Ferencz, E. Learned-Miller, and J. Malik, "Learning Hyper-Features for Visual Identification," in Neural Information Processing Systems, vol. 18, 2004.
[4]
D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," phInternational Journal of Computer Vision, vol. 60, no. 2, pp. 91--110, 2004.
[5]
Y. Zhou, X. Fan, X. Xie, Y. Gong, and W.-Y. Ma, "Inquiring of the Sights from the Web via Camera Mobiles," ph2006 IEEE International Conference on Multimedia and Expo, pp. 661--664, July 2006.
[6]
K. Mikolajczyk and C. Schmid, "Performance Evaluation of Local Descriptors," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 10, pp. 1615--1630, 2005.
[7]
H. Bay, T. Tuytelaars, and L. V. Gool, "SURF: Speeded Up Robust Features," in ECCV (1), 2006, pp. 404--417.
[8]
J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object Retrieval with Large Vocabularies and Fast Spatial Matching," in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR), 2007.
[9]
K. Greene, "Hyperlinking Reality via Phones," MIT Technology Review, Nov/Dec 2006.
[10]
E. Bruns and O. Bimber, "Adaptive Training of Video Sets for Image Recognition on Mobile Phones," Journal of Personal and Ubiquitous Computing, 2008, to appear.
[11]
C. Seifert, L. Paletta, A. Jeitler, E. Hödl, J.-P. Andreu, P. M. Luley, and A. Almer, "Visual Object Detection for Mobile Road Sign Inventory," in Proc. of Mobile Human-Computer Interaction - Mobile HCI 2004, 6th International Symposium, 2004, pp. 491--495.
[12]
G. Fritz, C. Seifert, and L. Paletta, "A Mobile Vision System for Urban Detection with Informative Local Descriptors," in ICVS '06: Proceedings of the Fourth IEEE International Conference on Computer Vision Systems, 2006, p. 30.
[13]
H. Bay, B. Fasel, and L. V. Gool, "Interactive Museum Guide: Fast and Robust Recognition of Museum Objects," in Proceedings of the First International Workshop on Mobile Vision, May 2006.
[14]
I. Skrypnyk and D. G. Lowe, "Scene Modelling, Recognition and Tracking with Invariant Image Features," in ISMAR '04: Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'04), 2004, pp. 110--119.
[15]
G. Takacs, V. Chandrasekhar, B. Girod, and R. Grzeszczuk, "Feature Tracking for Mobile Augmented Reality Using Video Coder Motion Vectors," in ISMAR '07: Proceedings of the Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07), 2007.
[16]
T. Yeh, K. Tollmar, and T. Darrell, "Searching the Web with Mobile Images for Location Recognition," in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2. IEEE Computer Society, 2004, pp. 76--81.
[17]
S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Y. Wu, "An Optimal Algorithm for Approximate Nearest Neighbor Searching Fixed Dimensions," phJournal of the ACM, vol. 45, no. 6, pp. 891--923, 1998.
[18]
L. V. G. H. Shao, T. Svoboda, "Zubud-Zürich buildings database for image based recognition." ETH Zürich, Tech. Rep. 260, 2003.
[19]
Y. Ke and R. Sukthankar, "PCA-SIFT: A More Distinctive Representation for Local Image Descriptors," in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR), vol. IEEE Computer Society, 2004, pp. 506--513.
[20]
W.-C. Chen, Y. Xiong, J. Gao, N. Gelfand, and R. Grzeszczuk, "Efficient Extraction of Robust Image Features on Mobile Devices," in ISMAR '07: Proceedings of the Sixth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'07), 2007.
[21]
P. Viola and M. J. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. of Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, 2001, pp. I:511--518.

Cited By

View all
  • (2024)Digitalization, Communications, and MobilitiesDigital Geographies—Urbanisation, Economy, and Modelling10.1007/978-981-97-9278-8_7(835-944)Online publication date: 18-Dec-2024
  • (2023)Trine: Cloud-Edge-Device Cooperated Real-Time Video Analysis for Household ApplicationsIEEE Transactions on Mobile Computing10.1109/TMC.2022.315472122:8(4973-4985)Online publication date: 1-Aug-2023
  • (2022)Test and Validation of a Multi-Block Solution for Improved Tracking in Outdoor Scenarios: A Case Study in the Pinocchio ParkInformation10.3390/info1310044913:10(449)Online publication date: 25-Sep-2022
  • Show More Cited By

Index Terms

  1. Outdoors augmented reality on mobile phone using loxel-based visual feature organization

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MIR '08: Proceedings of the 1st ACM international conference on Multimedia information retrieval
    October 2008
    506 pages
    ISBN:9781605583129
    DOI:10.1145/1460096
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 30 October 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. augmented reality
    2. geo-referenced photos
    3. image search
    4. local search
    5. mobile computing
    6. photo collections

    Qualifiers

    • Research-article

    Conference

    MM08
    Sponsor:
    MM08: ACM Multimedia Conference 2008
    October 30 - 31, 2008
    British Columbia, Vancouver, Canada

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)16
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Digitalization, Communications, and MobilitiesDigital Geographies—Urbanisation, Economy, and Modelling10.1007/978-981-97-9278-8_7(835-944)Online publication date: 18-Dec-2024
    • (2023)Trine: Cloud-Edge-Device Cooperated Real-Time Video Analysis for Household ApplicationsIEEE Transactions on Mobile Computing10.1109/TMC.2022.315472122:8(4973-4985)Online publication date: 1-Aug-2023
    • (2022)Test and Validation of a Multi-Block Solution for Improved Tracking in Outdoor Scenarios: A Case Study in the Pinocchio ParkInformation10.3390/info1310044913:10(449)Online publication date: 25-Sep-2022
    • (2022)Software Engineering for Augmented Reality - A Research AgendaProceedings of the ACM on Human-Computer Interaction10.1145/35322056:EICS(1-34)Online publication date: 17-Jun-2022
    • (2022)A multi-block method to improve 2D tracking in outdoor Augmented Reality applications2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767530(97-99)Online publication date: 21-Mar-2022
    • (2021)CBIR for urban building using A-KAZE features2021 IEEE 27th International Symposium for Design and Technology in Electronic Packaging (SIITME)10.1109/SIITME53254.2021.9663587(218-221)Online publication date: 27-Oct-2021
    • (2021)Image stitching by feature positioning and seam eliminationMultimedia Tools and Applications10.1007/s11042-021-10694-6Online publication date: 11-Mar-2021
    • (2021)Satisfied or not: user experience of mobile augmented reality in using natural language processing techniques on review commentsVirtual Reality10.1007/s10055-021-00599-y26:3(839-848)Online publication date: 1-Nov-2021
    • (2020)A Survey of Marker-Less Tracking and Registration Techniques for Health & Environmental Applications to Augmented Reality and Ubiquitous Geospatial Information SystemsSensors10.3390/s2010299720:10(2997)Online publication date: 25-May-2020
    • (2020)Practical Urban Localization for Mobile ARProceedings of the 21st International Workshop on Mobile Computing Systems and Applications10.1145/3376897.3377855(27-32)Online publication date: 3-Mar-2020
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media