Abstract
In rainy scenes, visibility can be degraded by raindrops which have adhered to the windscreen or camera lens. In order to resolve this degradation, we propose a method that automatically detects and removes adherent raindrops. The idea is to use long range trajectories to discover the motion and appearance features of raindrops locally along the trajectories. These motion and appearance features are obtained through our analysis of the trajectory behavior when encountering raindrops. These features are then transformed into a labeling problem, which the cost function can be optimized efficiently. Having detected raindrops, the removal is achieved by utilizing patches indicated, enabling the motion consistency to be preserved. Our trajectory based video completion method not only removes the raindrops but also complete the motion field, which benefits motion estimation algorithms to possibly work in rainy scenes. Experimental results on real videos show the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
The 24fps framerate only for reference on how we can deal with raindrop dynamics since our method assumes static raindrops during the detection process, while in fact in the real world raindrops can move. Hence, assuming the widely adopted framerate, it means we assume raindrops at least do not move in 1-s period of time. Obviously, a higher framerate does not pose any problem (except for the computation time), however a much lower framerate will create a large displacement problem, which can affect the optical flow accuracy.
References
Tan, R.: Visibility in bad weather from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2008)
Fattal, R.: Single image dehazing. In: SIGGRAPH (2008)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2009)
Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: ICCV (2013)
Barnum, P., Narasimhan, S., Kanade, T.: Analysis of rain and snow in frequency space. Int. J. Comput. Vis. (IJCV) 86, 256–274 (2010)
Garg, K., Nayar, S.: Vision and rain. Int. J. Comput. Vis. (IJCV) 75, 3–27 (2007)
Chen, Y.L., Hsu, C.T.: A generalized low-rank appearance model for spatio-temporally correlated rain streaks. In: ICCV (2013)
Garg, K., Nayar, S.: Photometric model of a raindrop. CMU Technical report (2003)
Garg, K., Nayar, S.: Detection and removal of rain from video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2004)
Kang, L., Lin, C., Fu, Y.: Automatic single-image-based rain streaks removal via image decomposition. IEEE Trans. Image Process. (TIP) 21, 1742–1755 (2012)
Eigen, D., Krishnan, D., Fergus, R.: Restoring an image taken through a window covered with dirt or rain. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 633–640 (2013)
Kurihata, H., Takahashi, T., Ide, I., Mekada, Y., Murase, H., Tamatsu, Y., Miyahara, T.: Rainy weather recognition from in-vehicle camera images for driver assistance. In: IEEE Intelligent Vehicles Symposium (2005)
Yamashita, A., Tanaka, Y., Kaneko, T.: Removal of adherent water-drops from images acquired with stereo camera. In: IROS (2005)
Yamashita, A., Fukuchi, I., Kaneko, T.: Noises removal from image sequences acquired with moving camera by estimating camera motion from spatio-temporal information. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2009)
Roser, M., Geiger, A.: Video-based raindrop detection for improved image registration. In: Workshops of IEEE International Conference on Computer Vision (2009)
Roser, M., Kurz, J., Geiger, A.: Realistic modeling of water droplets for monocular adherent raindrop recognition using bezier curves. In: Asian Conference on Computer Vision (ACCV) (2010)
You, S., Tan, R.T., Kawakami, R., Ikeuchi, K.: Adherent raindrop detection and removal in video. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) (2013)
Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2004)
Willson, R.G., Maimone, M., Johnson, A., Scherr, L.: An Optical Model for Image Artifacts Produced by Dust Particles on Lenses. Jet Propulsion Laboratory, National Aeronautics and Space Administration, Pasadena, CA (2005)
Zhou, C., Lin, S.: Removal of image artifacts due to sensor dust. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007, pp. 1–8. IEEE (2007)
Gu, J., Ramamoorthi, R., Belhumeur, P., Nayar, S.: Removing image artifacts due to dirty camera lenses and thin occluders. ACM Trans. Graph. (TOG) 28, 144 (2009)
Sand, P., Teller, S.: Particle video: Long-range motion estimation using point trajectories. Int. J. Comput. Vis. (IJCV) 80, 72–91 (2008)
Sundaram, N., Brox, T., Keutzer, K.: Dense point trajectories by GPU-accelerated large displacement optical flow. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 438–451. Springer, Heidelberg (2010)
Brox, T., Malik, J.: Large displacement optical flow: descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 33, 500–513 (2011)
Volz, S., Bruhn, A., Valgaerts, L., Zimmer, H.: Modeling temporal coherence for optical flow. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 1116–1123. IEEE (2011)
Rubinstein, M., Liu, C., Freeman, W.T.: Towards longer long-range motion trajectories. In: British Machine Vision Conference (BMVC), pp. 1–11 (2012)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)
Fulkerson, B., Vedaldi, A., Soatto, S.: Class segmentation and object localization with superpixel neighborhoods. In: IEEE International Conference on Computer Vision (ICCV) (2009)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI)s 26, 1124–1137 (2004)
Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 20, 1222–1239 (2001)
Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) 26, 147–159 (2004)
Geiger, A., Lenz, P., Urtasun, R.: Are we ready for autonomous driving? the kitti vision benchmark suite. In: Conference on Computer Vision and Pattern Recognition (CVPR) (2012)
Acknowledgement
This work is supported by Next-generation Energies for Tohoku Recovery (NET), MEXT, Japan.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material (mp4 17,261 KB)
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
You, S., Tan, R.T., Kawakami, R., Mukaigawa, Y., Ikeuchi, K. (2015). Raindrop Detection and Removal from Long Range Trajectories. In: Cremers, D., Reid, I., Saito, H., Yang, MH. (eds) Computer Vision -- ACCV 2014. ACCV 2014. Lecture Notes in Computer Science(), vol 9004. Springer, Cham. https://doi.org/10.1007/978-3-319-16808-1_38
Download citation
DOI: https://doi.org/10.1007/978-3-319-16808-1_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16807-4
Online ISBN: 978-3-319-16808-1
eBook Packages: Computer ScienceComputer Science (R0)