Skip to main content
SpringerLink
Log in

Visually Guided Cooperative Robot Actions Based on Information Quality

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

In field environments it is not usually possible to provide robots in advance with valid geometric models of its environment and task element locations. The robot or robot teams need to create and use these models to locate critical task elements by performing appropriate sensor based actions. This paper presents a multi-agent algorithm for a manipulator guidance task based on cooperative visual feedback in an unknown environment. First, an information-based iterative algorithm to intelligently plan the robot’s visual exploration strategy is used to enable it to efficiently build 3D models of its environment and task elements. The algorithm uses the measured scene information to find the next camera position based on expected new information content of that pose. This is achieved by utilizing a metric derived from Shannon’s information theory to determine optimal sensing poses for the agent(s) mapping a highly unstructured environment. Second, after an appropriate environment model has been built, the quality of the information content in the model is used to determine the constraint-based optimum view for task execution. The algorithm is applicable for both an individual agent as well as multiple cooperating agents. Simulation and experimental demonstrations on a cooperative robot platform performing a two component insertion/mating task in the field show the effectiveness of this algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Alur, R., Das, A., Esposito, J., Fierro, R., Grudic, G., Hur, Y., Kumar, V, Lee, I., Ostrowski, J., Pappas, G., Southall, B., Spletzer, J., and Taylor, C. 2000. A framework and architecture for multirobot coordination. In Proceedings of the International Symposium on Experimental Robotics, Waikiki, Hawaii.

  • Anousaki, G.C. and Kyriakopoulos, K.J. 1999. Simultaneous localization and map building for mobile robot navigation. IEEE Robotics & Automation Magazine, 6(3):42–53.

    Google Scholar 

  • Asada, M. 1990. Map building for a mobile robot from sensory data. IEEE Transactions on Systems, Man, and Cybernetics, 37(6).

  • Baumgartner, E.T., Schenker, P.S., Leger, C., and Huntsberger, T.L. 1998. Sensor-fused navigation and manipulation from a planetary rover. In Proceedings of SPIE Symposium on Sensor Fusion and Decentralized Control in Robotic Systems, vol. 3523, Boston, MA.

  • Burschka, D., Eberst, C., and Robl, C. 1997. Vision based model generation for indoor environments. In Proceedings of the 1997 IEEE International Conference on Robotics and Automation, vol. 3, Albuquerque, New Mexico, pp. 1940–1945.

  • Castellanos, J.A., Martinez, J.M., Neira, J., and Tardos, J.D. 1998. Simultaneous map building and localization for mobile robots: A multisensor fusion approach. In Proceedings of 1998 IEEE International Conference on Robotics and Automation vol. 2, Leuven, Belgium, pp. 1244–1249.

  • Choset, H. and Nagatani, K. 2001. Topological simultaneous localization and mapping (SLAM): Toward exact localization without explicit localization. IEEE Transactions on Robotics and Automation, 17(2):125–137.

    Article  Google Scholar 

  • Chu, G.-W., Yu, W.P., and Chung, M.J. 1997. A simple method of finding a visible region for the optimal camera position. In Proceedings of the IEEE International Conference on Advanced Robotics, Monterey, CA.

  • Connolly, C.I. 1985. The determination of the next best views. In Proceedings of the IEEE International Conference on Robotics and Automation, pp. 432–435.

  • Cowan, G.K. and Kovesi, P.D. 1988. Automatic sensor placement from vision task requirements. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(3): 407–416.

    Article  Google Scholar 

  • Donald, B., Jennings, J., and Rus, D. 1997. Information invariants for cooperating autonomous mobile robots. International Journal of Robotics Research, 16(5):673–702.

    Google Scholar 

  • Gelb, A. 1974. Applied Optimal Estimation. MIT Press: Cambridge, Massachusetts, U.S.A.

    Google Scholar 

  • Gerkey, B. and Mataric, M.J. 2000. Principled communication for dynamic multi-robot task allocation. In Proceedings of the International Symposium on Experimental Robotics 2000, Waikiki, HI.

  • Hager, G.D., Kriegman, D., Teh, E., and Rasmussen, C. 1997. Image-based prediction of landmark features for mobile robot navigation. In Proceedings of the 1997 IEEE International Conference on Robotics and Automation, vol. 2, Albuquerque, New Mexico, pp. 1040 –1046.

  • Howard, A., Mataric, M., and Sukhatme, G. 2003. Putting the ‘I’ in ‘Team’: An ego-centric approach to cooperative localization. In Proceedings of the IEEE International Conference on Robotics an Automation, Taipei, Taiwan, pp. 868–892.

  • Huntsberger, T.L., Rodriguez, G., and Schenker, P.S. 2000. Robotics: Challenges for robotic and human Mars exploration. In Proceedings of ROBOTICS2000, Albuquerque, New Mexico, pp. 299–305.

  • Huntsberger, T.L., Pirjanian, P., and Schenker, P.S. 2001. Robotic outposts as precursors to a manned Mars habitat. In Proceedings of the 2001 Space Technology and Applications International Forum (STAIF-2001), Albuquerque, New Mexico, pp. 46–51.

  • Huntsberger, T.L., Sujan, V.A., Dubowsky, S., and Schenker, P.S. 2003. Integrated system for sensing and traverse of cliff faces. In Proceedings of Aerosense’03: Unmanned Ground Vehicle Technology V. SPIE, vol. 5083, Orlando, Florida.

  • Hutchinson, S.A. and Kak, A. 1989. Planning sensing strategies in a robot work cell with multi-sensor capabilities. IEEE Trans. on Robotics and Automation, 5(6).

  • Jennings, C., Murray, D., and Little, J. 1999. Cooperative robot localization with vision-based mapping. In Proceedings of the 1999 IEEE International Conference on Robotics and Automation, Detroit, MI.

  • Kececi, F., Tonko, M., Nagel, H., and Gengenbach, V. 1998. Improving visually servoed disassembly operations by automatic camera placement. In Proceedings of 1998 IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 2947–2952.

  • Khatib, O., Yokoi, K., Chang, K., Ruspini, D., Holmberg, R., Casal, A., and Baader, A. 1995. Force strategies for cooperative tasks in multiple mobile manipulation systems. In Proceedings of the Seventh International Symposium on Robotics Research, Munich, Germany, pp. 333–342.

  • Kruse, E., Gutsche, R., and Wahl, F.M. 1996. Efficient, iterative, sensor based 3-D map building using rating functions in configuration space. In Proceedings of the 1996 IEEE International Conference on Robotics and Automation, vol. 2, Minneapolis, Minnesota, pp. 1067–1072.

  • Kuipers, B. and Byun, Y. 1991. A robot exploration and mapping strategy based on semantic hierarchy of spatial representations. Journal of Robotics and Autonomous Systems, 8:47–63.

    Article  Google Scholar 

  • Lara, B., Althoefer, K., and Seneviratne, L.D. 1998. Automated robot-based screw insertion system. In Proceedings Conference of the IEEE of the 24th Annual Industrial Electronics Society, 1998, IECON ‘98, vol. 4, pp. 2440–2445.

  • Laugier, C. and Triggs, B. 1995. Automatic camera placement for robot vision tasks. In IEEE International Conference on Robotics and Automation, Nagoya, Japan.

  • Leonard, J.J. and Durrant-Whyte, H.F. 1991. Simultaneous map building and localization for an autonomous mobile robot. In IEEE 1991 International Workshop on Intelligent Robots and Systems, Vol. 3, Osaka, Japan, pp. 1442–1447.

  • Lumelsky, V., Mukhopadhyay, S., and Sun, K. 1989. Sensor-based terrain acquisition: The “sightseer” strategy. In Proceedings of 28th Conference on Decision and Control, vol. 2, Tampa, FL, pp. 1157–1161.

  • Luo, R.C. and Kay, M.G. 1989. Multisensor integration and fusion in intelligent systems. IEEE Transactions on Systems, Man, and Cybernetics, 19(5).

  • Marapane, S.B., Trivedi, M.M., Lassiter, N., and Holder, M.B. 1996. Motion control of cooperative robotic teams through visual observation and fuzzy logic control. In Proceedings of IEEE International Conference on Robotics and Automation, vol. 2, Minnesota, MN, pp. 1738–1743.

  • Mataric, M.J. 1998. Coordination and learning in multi-robot systems. IEEE Intelligent Systems, 6–8.

  • O’Rourke, J. 1998. Computational Geometry in C, 2nd edn. Cambridge University Press.

  • Osborn, J.F., Whittaker, W.L., and Coppersmith, S. 1987. Prospects for robotics in hazardous waste management. In Proceedings of the Second International Conference on New Frontiers for Hazardous Waste Management, Pittsburgh, Pennsylvania.

  • Park, J., Jiang, B., and Neumann, U. 1999. Vision-based pose computation: Robust and accurate augmented reality tracking. In Proceedings of the 2nd IEEE/ACM International Workshop on Augmented Reality, San Francisco, California, pp. 3–12.

  • Parker, L. 1995. The effect of action recognition and robot awareness in cooperative robotic teams. In Proceedings of IEEE International Conference on Robotics and Automation, vol. 1, Nagoya, Japan, pp. 212–219.

  • Pirjanian, P., Huntsberger, T.L., and Schenker, P.S. 2001. Development of CAMPOUT and its further applications to planetary rover operations: A multirobot control architecture. In Proceedings of SPIE Sensor Fusion and Decentralized Control in Robotic Systems IV, vol. 4571, Newton, MA, pp. 108–119.

  • Rekleitis, I., Dudek, G., and Milios, E. 2000. Multi-robot collaboration for robust exploration. In Proceedings of the 2000 IEEE International Conference on Robotics and Automation, vol. 4, San Francisco, California, pp. 3164–3169.

  • Schenker, P.S., Baumgartner, E.T., Lindemann, R.A., Aghazarian, H., Ganino, A.J., Hickey, G.S., Zhu, D.Q., Matthies, L.H., Hoffman, B.H., and Huntsberger, T.L. 1998. New planetary rovers for long-range Mars science and sample return. In Proceedings of SPIE Symposium on Intelligent Robots and Computer Vision XVII: Algorithms, Techniques, and Active Vision, vol. 3522, Boston, Massachusetts.

  • Schenker, P.S., Pirjanian, P., Huntsberger, T.L., Trebi-Ollennu, A., Aghazarian, H., Leger, C., Dubowsky, S., and MMcKee, G.T. 2001. Robotic intelligence for space: Planetary surface exploration, task-driven robotic adaptation, and multirobot cooperation. In Proceedings of SPIE Symposium on Intelligent Robots and Computer Vision XX: Algorithms, Techniques, and Active Vision, vol. 4572, Newton, MA.

  • Shaffer, G. and Stentz, A. 1992. A robotic system for underground coal mining. In Proceedings of 1992 IEEE International Conference on Robotics and Automation, vol. 1, Nice, France, pp. 633–638.

  • Shannon, C.E. 1948. A mathematical theory of communication. The Bell System Technical Journal, 27:379–423, 623–656.

    Google Scholar 

  • Simhon, S. and Dudek, G. 1998. Selecting targets for local reference frames. In Proceedings of the IEEE International Conference on Robotics and Automation, vol. 4, Leuven, Belgium, pp. 2840–2845.

  • Smith, R.C. and Cheeseman, P. 1986. On the representation and estimation of spatial uncertainty. International Journal of Robotics Research, 5:(4) 56–68.

    CAS  Google Scholar 

  • Stroupe, A.W., Martin, M., and Balch, T. 2000. Merging gaussian distributions for object localization in multi-robot systems. In Proceedings of the Seventh International Symposium on Experimental Robotics, ISER’00, Hawaii.

  • Sujan, V.A. 2002a. Compensating for model uncertainty in the control of cooperative field robots. Ph.D. thesis June 2002, Dept. of Mech. Eng., MIT, Cambridge, MA, U.S.A.

  • Sujan, V.A. and Dubowsky, S. 2002b. Visually built task models for robot teams in unstructured environments. In Proceedings of the 2002 IEEE International Conference on Robotics and Automation, vol. 2, Washington, DC pp. 2840–2845.

  • Sujan, V.A., Dubowsky, S., Huntsberger, T., Aghazarian, H., Cheng, Y., and Schenker, P. 2003. Multi agent distributed sensing architecture with application to cliff surface mapping. In Proceedings of the 11th International Symposium of Robotics Research (ISRR), Siena, Italy.

  • Sujan, V.A. and Dubowsky, S. 2003b. Efficient information-based visual robotic mapping in unstructured environments. International Journal of Robotics Research (submitted).

  • Tarabanis, K.A., Allen, P.K., and Tsai, R.Y. 1995. A survey of sensor planning in computer vision. IEEE Transactions on Robotics and Automation, 11(1): 86–104.

    Article  Google Scholar 

  • Thrun, S, Fox, D., and Burgard, W. 1998. A probabilistic approach to concurrent mapping and localization for mobile robots. Machine Learning, 31:29–53.

    Article  Google Scholar 

  • Thrun, S., Burgard, W., and Fox, D. 2000. A real-time algorithm for mobile robot mapping with applications to multi-robot and 3D mapping. In Proceedings of the 2000 IEEE International Conference on Robotics and Automation, vol. 1, San Francisco, California, pp. 321–328.

  • Tomatis, N., Nourbakhsh, I., and Siegwar, R. 2001. Simultaneous localization and map building: A global topological model with local metric maps. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, Maui, Hawaii, pp. 421–426.

  • Trebi-Ollennu, A., Das, H., Aghazarian, H., Ganino, A., Pirjanian, P., Huntsberger, T., and Schenker, P. 2002. Mars Rover Pair cooperatively transporting a long payload. In Proceedings of 2002 IEEE International Conference on Robotics and Automation, vol. 4, Washington, DC, pp. 3136–3141.

  • Veloso, M. and Stone, P. 1999. Task decomposition and dynamic role assignment for real-time strategic teamwork. In Intelligent Agents V-Proc. of Int. Wkshp on Agent Theories, Architectures and Languages, Springer-Verlag, Heidelberg.

  • Victorino, A.C., Rives, P., and Borrelly, J.-J. 2000. Localization and map building using a sensor-based control strategy. In Proceedings of 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2, Takamatsu, Japan, pp. 937–942.

  • Yamauchi, B., Schultz, A., and Adams, W. 1998a. Mobile robot exploration and map-building with continuous localization. In Proceedings of 1998 IEEE International Conference on Robotics and Automation, vol. 4, Leuven, Belgium, pp. 3715–3720.

  • Yamauchi, B. 1998b. Frontier-based exploration using multiple robots. In Proceedings of the Second International Conference on Autonomous Agents (Agents ‘98), Minneapolis, MN, pp. 47–53.

  • Yeh, E. and Kriegman, D.J. 1995. Toward selecting and recognizing natural landmarks. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 1, Pittsburgh, PA, pp. 47–53.

Download references

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

    Vivek A. Sujan & Steven Dubowsky

Authors
  1. Vivek A. Sujan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steven Dubowsky
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vivek A. Sujan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sujan, V.A., Dubowsky, S. Visually Guided Cooperative Robot Actions Based on Information Quality. Auton Robot 19, 89–110 (2005). https://doi.org/10.1007/s10514-005-6013-2

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-005-6013-2

Keywords

Search

Navigation