Elsevier

Advances in Computers

Volume 34, 1992, Pages 59-111
Advances in Computers

Multisensory Computer Vision

https://doi.org/10.1016/S0065-2458(08)60324-1Get rights and content

Publisher Summary

The integration of multiple sensors or multiple sensing modalities is an effective method of minimizing the ambiguities inherent in interpreting perceived scenes. The multisensory approach is useful for a variety of tasks, including pose determination, surface reconstruction, object recognition, and motion computation among others. Several problems that were previously difficult or even impossible to solve because of the ill-posed nature of the formulations are converted to well-posed problems with the adoption of a multisensory approach. The chapter discusses modeling and the advantages of multisensory approaches to computer vision. The integration of multiple sensors or multiple sensing modalities is an effective method of minimizing the ambiguities inherent in interpreting perceived scenes. The chapter classified existing multisensory systems into three broadly defined groups: those that combine the output of multiple processing techniques applied to a single image of the scene, those that combine information extracted from multiple views of the same scene by using the same imaging modality, and those that combine different modalities of imaging, different processing techniques, or multiple views of the scene. The choice of a computational framework for a multisensory vision system depends on the application task. Several computational paradigms have been employed in different recent multisensory vision systems. The paradigms can be categorized as statistical, variational, artificial intelligence, and phenomenological approaches. The chapter also highlights issues pertaining to the hierarchical processing of multisensory imagery and levels of sensory information fusion.

References (88)

  • J.K. Aggarwal et al.

    Multisensor Fusion for Automatic Scene Interpretation—Research Issues and Directions

  • J. Aloimonos A. Basu (1988). Combining Information in Low-Level Vision, “Proceedings of DARPA Image Understanding...
  • R.C. Arkin E. Riseman A. Hanson (1988). AuRA: An Architecture for Vision-Based Robot Navigation, “Proceedings of the...
  • H. Asar N. Nandhakumar J.K. Aggarwal (1990). Pyramid-Based Image Segmentation Using Multisensory Data, Pattern...
  • N. Ayache C. Hansen (1988). Rectification of Images for Binocular and Trinocular Stereovision, “Proceedings of the...
  • N. Ayache et al.

    Trinocular Stereovision for Robotics

    IEEE Trans. Pattern Analysis and Machine Intelligence

    (1991)
  • D.C. Baker J.K. Aggarwal S.S. Hwang (1988). Geometry-Guided Segmentation of Outdoor Scenes, “Proceedings of the SPIE...
  • D.C. Baker et al.

    Detection and Segmentation of Man-Made Objects in Outdoor Scenes: Concrete Bridges

    Journal of the Optical Society of America A

    (1989)
  • D.H. Ballard et al.

    “Computer Vision”

    (1982)
  • R. Belknap E. Riseman A. Hanson (1986). The Information Fusion Problem and Rule-Based Hypotheses Applied to Complex...
  • B. Bhanu P. Symosek (1987). Interpretation of Terrain Using Hierarchical Symbolic Grouping from Multi-Spectral Images,...
  • A. Blake

    Comparison of the Efficiency of Deterministic and Stochastic Algorithms for Visual Reconstruction

    IEEE Trans. PAMI

    (1989)
  • R. Bolle A. Califano R. Kjeldsen R.W. Taylor (1988). Visual Recognition Using Concurrent and Layered Parameter...
  • C.C. Chu N. Nandhakumar J.K. Aggarwal (1988). Image Segmentation and Information Integration of Laser Radar Data,...
  • CIE

    “Recommendation on Uniform Color Spaces- Color Difference Equations, Psychometric Color Terms”

    (1978)
  • R. Courant et al.

    “Methods of Mathematical Physics”

    (1953)
  • U.R. Dhond et al.

    Structure from Stereo—A Review

    IEEE Tram. Systems, Man and Cybernetics

    (1989)
  • U.R. Dhond J.K. Aggarwal (1989b). A Closer Look at the Contribution of a Third Camera Towards Accuracy in Stereo...
  • S. Di Zenzo et al.

    Gaussian Maximum Likelihood and Contextual Classification for Multicrop Classification

    IEEE Trans, on Geo-science and Remote Sensing

    (1987)
  • R.O. Duda et al.

    “Pattern Classification and Scene Analysis”

    (1973)
  • J.S. Duncan L.H. Staib (1987). Shape Determination from Incomplete and Noisy Multisensory Imagery, “Proceedings of AAAI...
  • J.S. Duncan G.R. Gindi K.S. Narendra (1987). Multisensor Scene Segmentation Using Learning Automata, “Proceedings of...
  • R.T. Dunlay (1988). Obstacle Avoidance Perception Processing for the Autonomous Land Vehicle, “Proceedings of the IEEE...
  • H.F. Durrant-Whyte (1987). Sensor Models and Multi-Sensor Integration, “Proceedings of the AAAI Workshop on Spatial...
  • H.F. Durrant-Whyte

    “Integration, Coordination and Control of Multi-Sensor Robot Systems”

    (1988)
  • R.T. Frankot et al.

    A Method for Enforcing Integrability in Shape from Shading Algorithms

    IEEE Trans. Pattern Analysis and Machine Intelligence

    (1988)
  • K. Fukunaga

    “Introduction to Statistical Pattern Recognition”

    (1990)
  • J.J. Gelfand J.C. Pearson C.D. Spence (1988). Multisensor Integration in Biological Systems, “Proceedings of the Third...
  • S. Geman et al.

    Stochastic Relaxation, Gibbs Distribution and the Bayesian Restoration of Images

    IEEE Trans. Pattern Analysis and Machine Intelligence

    (1984)
  • R.C. Gonzalez et al.

    “Digital Image Processing”

    (1987)
  • G. Hager M. Mintz (1987). Searching for Information, “Proceedings of the AAAI Workshop on Spatial Reasoning and...
  • S.Y. Harmon (1988). Tools for Multisensor Data Fusion in Autonomous Robots, “Proceedings of the NATO Advanced Research...
  • S.Y. Harmon et al.

    Information Processing System Architecture for an Autonomous Robot System

    (1983)
  • G. Healey (1991). Using Color to Segment Images of 3-D Scenes, “Proceedings of the SPIE Conference on Applications of...
  • Cited by (13)

    • Unified 3D models for multisensor image synthesis

      1995, Graphical Models and Image Processing
    • Real-time lane markings recognition based on seed-fill algorithm

      2019, ACM International Conference Proceeding Series
    • Lane Departure Identification for Advanced Driver Assistance

      2015, IEEE Transactions on Intelligent Transportation Systems
    • Simulation of FLIR and LADAR data using graphics animation software

      2000, Proceedings - Pacific Conference on Computer Graphics and Applications
    • Detecting moving objects in airborne forward looking infra-red sequences

      1999, Proceedings - IEEE Workshop on Computer Vision Beyond the Visible Spectrum: Methods and Applications, CVBVS 1999
    View all citing articles on Scopus
    *

    Supported in part by the Commonwealth of Virginia's Center for Innovative Technology under contract VCIT INF-91–007, and in part by the National Science Foundation under grant IRI-91109584.

    Supported by Army Research Office under contract no. DAAL-03–91-G-0050.

    View full text