Authors:
Hazem Rashed
1
;
Senthil Yogamani
2
;
Ahmad El-Sallab
1
;
Pavel Křížek
3
and
Mohamed El-Helw
4
Affiliations:
1
CDV AI Research, Cairo and Egypt
;
2
Valeo Vision Systems and Ireland
;
3
Valeo R&D DVS, Prague and Czech Republic
;
4
Nile University, Cairo and Egypt
Keyword(s):
Semantic Segmentation, Visual Perception, Dense Optical Flow, Automated Driving.
Related
Ontology
Subjects/Areas/Topics:
Computer Vision, Visualization and Computer Graphics
;
Image and Video Analysis
;
Segmentation and Grouping
Abstract:
Motion is a dominant cue in automated driving systems. Optical flow is typically computed to detect moving objects and to estimate depth using triangulation. In this paper, our motivation is to leverage the existing dense optical flow to improve the performance of semantic segmentation. To provide a systematic study, we construct four different architectures which use RGB only, flow only, RGBF concatenated and two-stream RGB + flow. We evaluate these networks on two automotive datasets namely Virtual KITTI and Cityscapes using the state-of-the-art flow estimator FlowNet v2. We also make use of the ground truth optical flow in Virtual KITTI to serve as an ideal estimator and a standard Farneback optical flow algorithm to study the effect of noise. Using the flow ground truth in Virtual KITTI, two-stream architecture achieves the best results with an improvement of 4% IoU. As expected, there is a large improvement for moving objects like trucks, vans and cars with 38%, 28% and 6% incre
ase in IoU. FlowNet produces an improvement of 2.4% in average IoU with larger improvement in the moving objects corresponding to 26%, 11% and 5% in trucks, vans and cars. In Cityscapes, flow augmentation provided an improvement for moving objects like motorcycle and train with an increase of 17% and 7% in IoU.
(More)