Skip to main content
Springer Nature Link
Log in

A vision of ‘vision and language’ comprises action: An example from road traffic

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

This contribution is based on two previously published approaches one of which automatically extracts vehicle trajectories from image sequences of traffic scenes and associates these trajectories with motion verbs. The second approach exploits machine vision in order to maneuver autonomous road vehicles. The combination of these two approaches provides a link from the evaluation of video signals via an abstract representation at the level of natural language concepts to actuator devices in automatic closed loop control of road vehicles. Building on implemented representations for elementary motion verbs and for elementary road vehicle maneuvers, a grammar to represent a nontrivial subset of more complex driving activities on a highway is formulated. Driving on a highway can thereby be investigated not only at the level of control algorithms, but simultaneously at the level of natural language descriptions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Enkelmann, W. (1991). Obstacle Detection by Evaluation of Optical Flow Fields from Image Sequences.Image and Vision Computing 9: 160–168.

    Google Scholar 

  • Heinze, N., Krüger, W. & Nagel, H.-H. (1991). Berechnung von Bewegungsverben zur Beschreibung von aus Bildfolgen gewonnenen Fahrzeugtrajektorien in Straßenverkehrsszenen.Informatik — Forschung und Entwicklung 6: 51–61 (in German).

    Google Scholar 

  • Herzog, G., Sung, C.-K., André, E., Enkelmann, W., Nagel, H.-H., Rist, T., Wahlster, W. & Zimmermann, G. (1989). Incremental Natural Language Description of Dynamic Imagery. In Brauer, W. & Freksa, C. (eds) InProceedings of Dritter Internationaler GI-Kongreß Wissensbasierte Systeme, 153–162. Springer-Verlag: Berlin Heidelberg New York/NY, Informatik-Fachberichte 227.

    Google Scholar 

  • Koller, D., Heinze, N. & Nagel, H.-H. (1991). Algorithmic Characterization of Vehicle Trajectories from Image Sequences by Motion Verbs. In Proceedings ofThe IEEE Conference on Computer Vision and Pattern Recognition CVPR '91, 90–95. Lahaina, Maui/Hawaii, 3–6 June 1991.

  • Koller, D. (1992).Detektion, Verfolgung und Klassifikation bewegter Objekte in monokularen Bildfolgen am Beispiel von Straßenverkehrsszenen. Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH); appeared in the series ‘Dissertationen zur Künstlichen Intelligenz’ DISKI vol. 13. infix: Sankt Augustin/Germany (in German).

  • Koller, D., Daniilidis, K. & Nagel, H.-H. (1993). Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes.International Journal of Computer Vision 10: 257–281.

    Google Scholar 

  • Kollnig, H. & Nagel, H.-H. (1993). Ermittlung von begrifflichen Beschreibungen von Geschehen in Straßenverkehrsszenen mit Hilfe unscharfer Mengen.Informatik — Forschung und Entwicklung 8: 186–196 (in German).

    Google Scholar 

  • Kollnig, H., Nagel, H.-H. & Otte, M. (1994). Association of Motion Verbs with Vehicle Movements Extracted from Dense Optical Flow Fields. In Eklundh, J.-O. (ed.) In Proceedings ofThe Third European Conference on Computer Vision ECCV '94, Stockholm/Sweden, vol. II, 338–347. Springer-Verlag: Berlin Heidelberg New York/NY, Lecture Notes in Computer Science vol. 801.

    Google Scholar 

  • Krüger, W. (1991).Begriffsgraphen zur Situationsmodellierung in der Bildfolgenauswertung. Dissertation, Fakultät für Informatik der Universität Karlsruhe (TH), Karlsruhe/Germany. A revised version appeared asSituationsmodellierung in der Bildfolgenauswertung. Springer-Verlag: Berlin Heidelberg (1992), Informatik-Fachberichte vol. 311 (in German).

  • Nagel, H.-H. (1988). From Image Sequences towards Conceptual Descriptions.Image and Vision Computing 6: 59–74.

    Google Scholar 

  • Nagel, H.-H. (1989). Zur Erkennung von Situationen durch Answertung von Bildfolgen.FhG-Berichte 1(89): 25–33.

    Google Scholar 

  • Nagel, H.-H. (1991). The Representation of Situations and Their Recognition from Image Sequences. In Proceedings ofAFCET 8 e Congrès Reconnaissance des Formes et Intelligence Artificielle, 1221–1229. Lyon-Villeurbanne/France, 25–29 November.

  • Nagel, H.-H. (1993).Überlegungen zur Formalisierung der sichtsystemgestützten Straßenfahrzeugführung. Internal report, Fraunhofer-Institut für Informations- und Datenverarbeitung (IITB), Karlsruhe/Germany (in German).

  • Nagel, H.-H. & Enkelmann, W. (1991). Generic Road Traffic Situations and Driver Support Systems. In Proceedings of theFifth PROMETHEUS Workshop, 76–85. Munich/Germany, 15–16 October 1991.

  • Nagel, H.-H., Enkelmann, W. & Struck, G. (1994).FhG-Co-Driver: From Map-Guided Automatic Driving by Machine Vision to a Cooperative Driving Support. Computers and Mathematics with Applications, Special Issue on “Network, Control, Communication and Computing Technologies for Intelligent Vehicle Highway Systems” (in press).

  • Retz-Schmidt, G. (1991). Recognizing Intentions, Interactions, and Causes of Plan Failures.User-Modelling and User-Adapted Interaction 1: 173–202.

    Google Scholar 

  • Retz-Schmidt, G. (1992).Die Interpretation des Verhaltens mehrerer Akteure in Szenenfolgen. Springer-Verlag: Berlin Heidelberg, Informatik-Fachberichte vol. 308 (in German).

  • Siegle, G., Geisler, J., Laubenstein, F., Nagel, H.-H. & Struck, G. (1992). Autonomous Driving on a Road Network. In Proceedings ofIntelligent Vehicles '92 Symposium, 403–408. Detroit/MI: 29 June–1 July 1992.

  • Struck, G., Geisler, J., Laubenstein, F., Nagel, H.-H. & Siegle, G. (1993). Interaction between Digital Road Map Systems and Trinocular Autonomous Driving. In Proceedings ofIntelligent Vehicles '93 Symposium, 461–466. Tokyo/Japan: 14–16 July 1993.

  • Winograd, T. (1972).Understanding Natural Language. Academic Press, Inc.: New York, NY. For a synopsis, see: Winograd, T. (1973) A Procedural Model of Language Understanding. In Schank, R. C. & Colby, K. M. (eds.)Computer Models of Thought and Language. W.H. Freeman and Company: San Francisco/CA.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fraunhofer-Institut für Informations- und Datenverarbeitung (IITB), Fraunhoferstr. 1, D-76131, Karlsruhe, Germany

    Hans-Hellmut Nagel

  2. Fakultät für Informatik der Universität Karlsruhe (TH), Karlsruhe

    Hans-Hellmut Nagel

Authors
  1. Hans-Hellmut Nagel
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Nagel, HH. A vision of ‘vision and language’ comprises action: An example from road traffic. Artif Intell Rev 8, 189–214 (1994). https://doi.org/10.1007/BF00849074

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1007/BF00849074

Key words