Skip to main content
SpringerLink
Log in

Localization of a door handle of unknown geometry using a single camera for door-opening with a mobile manipulator

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

Door-opening by a robot is a challenging task. Current approaches to localization of door handles are only applicable to certain types of handles or require the use of learning algorithms. In this paper, a method for localizing a door handle of unknown geometry is presented. The localization is performed by fitting a bounding box around the door handle. The proposed method extracts a three-dimensional (3D) point cloud using the optical flow calculated from images taken with a single CCD camera. The point cloud is then segmented to separate the door and the handle points. A bounding box is fitted to the door handle. The box is oriented along the principal components of the handle points after they are projected into the plane of the door, and the normal to the door. The box performs the function of both localizing the handle and estimating its pose. Experiments performed on a set of various door handles demonstrated the effectiveness of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Andreopoulos, A., & Tsotsos, J. (2008). Active vision for door localization and door opening using playbot: a computer controlled wheelchair for people with mobility impairments. In Proceedings of the 5th Canadian conference on computer and robot vision (pp. 3–10).

    Chapter  Google Scholar 

  • Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., & Szeliski, R. (2007). A database and evaluation methodology for optical flow. In Proceedings of the IEEE 11th international conference on computer vision (pp. 1–8).

    Chapter  Google Scholar 

  • Baker, S., Scharstein, D., Lewis, J., Roth, S., Black, M. J., & Szeliski, R. (2009). A database and evaluation methodology for optical flow (Tech. rep.). Middlebury College, Microsoft Research.

  • Cheng, Y. (1995). Mean shift, mode seeking, and clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(8), 790–799.

    Article  Google Scholar 

  • Chung, W., Kim, G., & Kim, M. (2007). Development of the multi-functional indoor service robot PSR systems. Autonomous Robots, 22(1), 1–17.

    Article  Google Scholar 

  • Chung, W., Rhee, C., Shim, Y., Lee, H., & Park, S. (2009). Door-opening control of a service robot using the multifingered robot hand. IEEE Transactions on Industrial Electronics, 56(10), 3975–3984.

    Article  Google Scholar 

  • Cyganek, B., & Siebert, J. (2009). An introduction to 3D computer vision techniques and algorithms. New York: Wiley.

    Book  MATH  Google Scholar 

  • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 6, 381–395.

    Article  MathSciNet  Google Scholar 

  • Fukunaga, K., & Hostetler, L. (1975). The estimation of the gradient of a density function, with applications in pattern recognition. IEEE Transactions on Information Theory, 21(1), 32–40.

    Article  MathSciNet  MATH  Google Scholar 

  • Fulgenzi, C., Ippoliti, G., & Longhi, S. (2009). Experimental validation of FastSLAM algorithm integrated with a linear features based map. Mechatronics, 19(5), 609–616.

    Article  Google Scholar 

  • Goddard, J. S. (1997). Pose and motion estimation from vision using dual quaternion-based extended Kalman filtering. PhD thesis. University of Tennessee, Knoxville.

  • Guerrero, J., & Sagues, C. (2001). Uncalibrated vision based on lines for robot navigation. Mechatronics, 11(6), 759–777.

    Article  Google Scholar 

  • Ihrke, I., Kutulakos, K., Lensch, H., Magnor, M., & Heidrich, W. (2010). Transparent and specular object reconstruction. Computer Graphics Forum, 29, 2400–2426.

    Article  Google Scholar 

  • Kim, D., Kang, J. H., Hwang, C. S., & Park, G. T. (2004). Mobile robot for door opening in a house. In M. Negoita, R. Howlett, & L. Jain (Eds.), Lecture notes in computer science: Vol. 3215. Knowledge-based intelligent information and engineering systems (pp. 596–602). Berlin: Springer.

    Chapter  Google Scholar 

  • Klingbeil, E., Saxena, A., & Ng, A. (2008). Learning to open new doors. In Proceedings of the robotics science and systems (RSS) workshop on robot manipulation.

    Google Scholar 

  • Klingbeil, E., Saxena, A., & Ng, A. (2010). Learning to open new doors. In Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 2751–2757).

    Google Scholar 

  • Kosov, S., Thormahlen, T., & Seidel, H. P. (2011). Using active illumination for accurate variational space-time stereo. In A. Heyden & F. Kahl (Eds.), Lecture notes in computer science: Vol. 6688. Image analysis (pp. 752–763). Berlin: Springer.

    Chapter  Google Scholar 

  • Kragic, D., Petersson, L., & Christensen, H. (2002). Visually guided manipulation tasks. Robotics and Autonomous Systems, 40(2–3), 193–203.

    Article  Google Scholar 

  • Lin, H., & Tsai, C. (2007). Improved global localization of an indoor mobile robot via fuzzy extended information filtering. Robotica, 26(2), 241–254.

    MathSciNet  Google Scholar 

  • Liu, G., He, X., Yuan, J., Abdul, S., & Goldenberg, A. (2008). Development of modular and reconfigurable robot with multiple working modes. In Proceedings of the 2008 IEEE international conference on robotics and automation, Pasadena, CA (pp. 3502–3507).

    Google Scholar 

  • Liu, G., Ahmad, S., & Ren, L. (2009). Hybrid control of door-opening by modular re-configurable robots. In Proceedings of the 2009 IEEE/RSJ international conference on intelligent robots and systems, St. Louis, MO (pp. 954–959).

    Chapter  Google Scholar 

  • Lowe, D. G. (1987). Three-dimensional object recognition from single two-dimensional images. Artificial Intelligence, 31(3), 355–395.

    Article  Google Scholar 

  • Lowe, D. G. (1991). Fitting parameterized three-dimensional models to images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5), 441–450.

    Article  MathSciNet  Google Scholar 

  • Munoz-Salinas, R., Aguirre, E., Garcia-Silvente, M., & Gomez, M. (2005). A multi-agent system architecture for mobile robot navigation based on fuzzy and visual behaviour. Robotica, 23(23), 689–699.

    Article  Google Scholar 

  • Petrovskaya, A., & Ng, A. (2007). Probabilistic mobile manipulation in dynamic environments, with application to opening doors. In Proceedings of the international joint conference on artificial intelligence (IJCAI), Hyderabad, India (pp. 2178–2184).

    Google Scholar 

  • Prats, M., Sanz, P., & del Pobil, A. (2010). Reliable non-prehensile door opening through the combination of vision, tactile and force feedback. Autonomous Robots, 29(2), 201–218.

    Article  Google Scholar 

  • Rhee, C., Chung, W., Kim, M., Shim, Y., & Lee, H. (2004). Door opening control using the multi-fingered robotic hand for the indoor service robot. In Proceedings of the IEEE international conference on robotics and automation (pp. 4011–4016).

    Google Scholar 

  • Ritter, X. G., & Wilson, N. J. (2001). Handbook of computer vision algorithms in image algebra. Boca Raton: CRC Press.

    MATH  Google Scholar 

  • Rosenhahn, B. (2003). Pose estimation revisited. PhD thesis. Universitat Kiel.

  • Rusu, R., Meeussen, W., Chitta, S., & Beetz, M. (2009). Laser-based perception for door and handle identification. In Proceedings of the International conference on advanced robotics, Munich (pp. 1–8).

    Google Scholar 

  • Scharstein, D., & Szeliski, R. (2003). High-accuracy stereo depth maps using structured light. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (Vol. 1, pp. I-195–I-202).

    Google Scholar 

  • Srinivasa, S., Ferguson, D., Helfrich, C., Berenson, D., Collet, A., Diankov, R., Gallagher, G., Hollinger, G., Kuffner, J., & Weghe, M. (2010). HERB: a home exploring robotic butler. Autonomous Robots, 28(1), 1–17.

    Article  Google Scholar 

  • Wang, L., Yang, R., & Davis, J. (2007). BRDF invariant stereo using light transport constancy. IEEE transactions on pattern analysis and machine intelligence, 29(9), 1616–1626.

    Article  Google Scholar 

  • Zach, C., Pock, T., & Bischof, H. (2007). A duality based approach for realtime TV-L1 optical flow. In Lecture notes in computer science: Vol. 4713. Pattern recognition (pp. 214–223). Berlin: Springer.

    Chapter  Google Scholar 

  • Zhang, L., Curless, B., & Seitz, S. M. (2003). Spacetime stereo: shape recovery for dynamic scenes. In Proceedings of the IEEE computer society conference on computer vision and pattern recognition (pp. 367–374).

    Google Scholar 

  • Zickler, T., Ho, J., Kriegman, D., Ponce, J., & Belhumeur, P. (2003). Binocular Helmholtz stereopsis. In Proceedings of the ninth IEEE international conference on computer vision (Vol. 2, pp. 1411–1417).

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported by Natural Sciences and Engineering Research Council (NSERC) of Canada.

Author information

Authors and Affiliations

  1. Ryerson University, 350 Victoria St., Toronto, Ontario, Canada

    Dmitri Ignakov, Galina Okouneva & Guangjun Liu

Authors
  1. Dmitri Ignakov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Galina Okouneva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangjun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dmitri Ignakov.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ignakov, D., Okouneva, G. & Liu, G. Localization of a door handle of unknown geometry using a single camera for door-opening with a mobile manipulator. Auton Robot 33, 415–426 (2012). https://doi.org/10.1007/s10514-012-9297-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-012-9297-9

Keywords

Search

Navigation