research-article

CRF based method for curb detection using semantic cues and stereo depth

Authors:

Sarthak Upadhyay,

K Madhava Krishna,

Shanti SwarupAuthors Info & Claims

ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing

Article No.: 41, Pages 1 - 7

https://doi.org/10.1145/3009977.3010058

Published: 18 December 2016 Publication History

Abstract

Curb detection is a critical component of driver assistance and autonomous driving systems. In this paper, we present a discriminative approach to the problem of curb detection under diverse road conditions. We define curbs as the intersection of drivable and non-drivable area which are classified using dense Conditional random fields(CRF). In our method, we fuse output of a neural network used for pixel-wise semantic segmentation with depth and color information from stereo cameras. CRF fuses the output of a deep model and height information available in stereo data and provides improved segmentation. Further we introduce temporal smoothness using a weighted average of Segnet output and output from a probabilistic voxel grid as our unary potential. Finally, we show improvements over the current state of the art neural networks. Our proposed method shows accurate results over large range of variations in curb curvature and appearance, without the need of retraining the model for the specific dataset.

References

[1]

V. Badrinarayanan, A. Kendall, and R. Cipolla. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. arXiv preprint arXiv:1511.00561, 2015.

[2]

C. Fernández, R. Izquierdo, D. Llorca, and M. Sotelo. Road curb and lanes detection for autonomous driving on urban scenarios. In 17th International IEEE Conference on Intelligent Transportation Systems (ITSC), pages 1964--1969. IEEE, 2014.

[3]

C. Fernández, D. Llorca, C. Stiller, and M. Sotelo. Curvature-based curb detection method in urban environments using stereo and laser. In 2015 IEEE Intelligent Vehicles Symposium (IV), pages 579--584. IEEE, 2015.

[4]

A. Geiger, J. Ziegler, and C. Stiller. Stereoscan: Dense 3d reconstruction in real-time. In IEEE Intelligent Vehicles Symposium, Baden-Baden, Germany, June 2011.

[5]

H. Hirschmuller. Stereo processing by semiglobal matching and mutual information. IEEE Transactions on pattern analysis and machine intelligence, 30(2):328--341, 2008.

Digital Library

[6]

V. Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. Adv. Neural Inf. Process. Syst, 2011.

Digital Library

[7]

F. Oniga and S. Nedevschi. Polynomial curb detection based on dense stereovision for driving assistance. In Intelligent Transportation Systems (ITSC), 2010 13th International IEEE Conference on, pages 1110--1115. IEEE, 2010.

[8]

F. Oniga and S. Nedevschi. Processing dense stereo data using elevation maps: Road surface, traffic isle, and obstacle detection. IEEE Transactions on Vehicular Technology, 59(3):1172--1182, 2010.

[9]

F. Oniga, S. Nedevschi, and M. M. Meinecke. Curb detection based on elevation maps from dense stereo. In 2007 IEEE International Conference on Intelligent Computer Communication and Processing, pages 119--125. IEEE, 2007.

[10]

F. Oniga, S. Nedevschi, and M. M. Meinecke. Curb detection based on a multi-frame persistence map for urban driving scenarios. In 2008 11th International IEEE Conference on Intelligent Transportation Systems, pages 67--72. IEEE, 2008.

[11]

C. D. Pantilie and S. Nedevschi. Real-time obstacle detection in complex scenarios using dense stereo vision and optical flow. In Intelligent Transportation Systems (ITSC), 2010 13th International IEEE Conference on, pages 439--444. IEEE, 2010.

[12]

M. Pauly, M. Gross, and L. P. Kobbelt. Efficient simplification of point-sampled surfaces. In Visualization, 2002. VIS 2002. IEEE, pages 163--170, 2002.

Digital Library

[13]

S. Se and M. Brady. Vision-based detection of kerbs and steps. In BMVC, 1997.

[14]

J. Siegemund, U. Franke, and W. Förstner. A temporal filter approach for detection and reconstruction of curbs and road surfaces based on conditional random fields. In Intelligent Vehicles Symposium (IV), 2011 IEEE, pages 637--642. IEEE, 2011.

[15]

K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.

Cited By

Zou MKageyama Y(2025)Detection of Curb Boundary in Snow-covered Road Image via Feature MatchingIEEJ Journal of Industry Applications10.1541/ieejjia.2400525514:1(84-93)Online publication date: 1-Jan-2025
https://doi.org/10.1541/ieejjia.24005255
Kao CWang SHuang C(2024)Application of Edge Detection Technology Based on YOLOv8 in Smart Mobility Aids2024 IEEE 6th Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS)10.1109/ECBIOS61468.2024.10885417(258-262)Online publication date: 14-Jun-2024
https://doi.org/10.1109/ECBIOS61468.2024.10885417
Guo DYang GQi BWang C(2022)Curb Detection and Compensation Method for Autonomous Driving via a 3-D-LiDAR SensorIEEE Sensors Journal10.1109/JSEN.2022.319886122:20(19500-19512)Online publication date: 15-Oct-2022
https://doi.org/10.1109/JSEN.2022.3198861
Show More Cited By

Index Terms

CRF based method for curb detection using semantic cues and stereo depth
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Obstacle Detection Using a Single Camera Stereo Sensor
SITIS '07: Proceedings of the 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System

In this paper we present a catadioptric stereovision system based on a single camera, two plane mirrors and a prism mounted on a rotating axis. On the one hand, the system is able to project a real scene on two half images. Thus, contrary to classical ...
Data‐driven recovery of hand depth using CRRF on stereo images

Hand pose is emerging as an important interface for human–computer interaction. This study presents a data‐driven method to estimate a high‐quality depth map of a hand from a stereoscopic camera input by introducing a novel superpixel‐based regression ...
Vision-based active safety system for automatic stopping

Intelligent systems designed to reduce highway fatalities have been widely applied in the automotive sector in the last decade. Of all users of transport systems, pedestrians are the most vulnerable in crashes as they are unprotected. This paper deals ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICVGIP '16: Proceedings of the Tenth Indian Conference on Computer Vision, Graphics and Image Processing

December 2016

743 pages

ISBN:9781450347532

DOI:10.1145/3009977

Program Chairs:
Dhruv Batra
Georgia Tech
,
Michael Brown
York University, Canada
,
Vijay Natarajan
Indian Institute of Science, Bangalore

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Google Inc.
QI: Qualcomm Inc.
Tata Consultancy Services
NVIDIA
MathWorks: The MathWorks, Inc.
Microsoft Research: Microsoft Research

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 December 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICVGIP '16

Sponsor:

QI
MathWorks
Microsoft Research

ICVGIP '16: Indian Conference on Computer Vision, Graphics and Image Processing

December 18 - 22, 2016

Assam, Guwahati, India

Acceptance Rates

ICVGIP '16 Paper Acceptance Rate 95 of 286 submissions, 33%;

Overall Acceptance Rate 95 of 286 submissions, 33%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
187
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zou MKageyama Y(2025)Detection of Curb Boundary in Snow-covered Road Image via Feature MatchingIEEJ Journal of Industry Applications10.1541/ieejjia.2400525514:1(84-93)Online publication date: 1-Jan-2025
https://doi.org/10.1541/ieejjia.24005255
Kao CWang SHuang C(2024)Application of Edge Detection Technology Based on YOLOv8 in Smart Mobility Aids2024 IEEE 6th Eurasia Conference on Biomedical Engineering, Healthcare and Sustainability (ECBIOS)10.1109/ECBIOS61468.2024.10885417(258-262)Online publication date: 14-Jun-2024
https://doi.org/10.1109/ECBIOS61468.2024.10885417
Guo DYang GQi BWang C(2022)Curb Detection and Compensation Method for Autonomous Driving via a 3-D-LiDAR SensorIEEE Sensors Journal10.1109/JSEN.2022.319886122:20(19500-19512)Online publication date: 15-Oct-2022
https://doi.org/10.1109/JSEN.2022.3198861
Zou MKageyama YAkashi T(2022) Curb Detection Using a Novel Deep Learning Framework Based on YOLO ‐v2 IEEJ Transactions on Electrical and Electronic Engineering10.1002/tee.2364717:9(1321-1329)Online publication date: 14-Jun-2022
https://doi.org/10.1002/tee.23647
Romero LGuerrero JRomero G(2021)Road Curb Detection: A Historical SurveySensors10.3390/s2121695221:21(6952)Online publication date: 20-Oct-2021
https://doi.org/10.3390/s21216952
Yang FWang HJin Z(2020)A fusion network for road detection via spatial propagation and spatial transformationPattern Recognition10.1016/j.patcog.2019.107141100:COnline publication date: 1-Apr-2020
https://dl.acm.org/doi/10.1016/j.patcog.2019.107141
Pengpeng SXiangmo Zzhigang XRunmin WHaigen M(2019)A 3D LIDAR DATA BASED DEDICATED ROAD BOUNDARY DETECTION ALGORITHM FOR AUTONOMOUS VEHICLESIEEE Access10.1109/ACCESS.2019.2902170(1-1)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2902170
Yu BFan Z(2019)A comprehensive review of conditional random fields: variants, hybrids and applicationsArtificial Intelligence Review10.1007/s10462-019-09793-6Online publication date: 13-Dec-2019
https://doi.org/10.1007/s10462-019-09793-6
Cheng MZhang YSu YAlvarez JKong H(2018)Curb Detection for Road and Sidewalk DetectionIEEE Transactions on Vehicular Technology10.1109/TVT.2018.286583667:11(10330-10342)Online publication date: Nov-2018
https://doi.org/10.1109/TVT.2018.2865836

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten