research-article

User-assisted reflection detection and feature point tracking

Authors:

Mohamed A. Elgharib,

François Pitite,

Venkatesh SaligramaAuthors Info & Claims

CVMP '13: Proceedings of the 10th European Conference on Visual Media Production

Article No.: 13, Pages 1 - 10

https://doi.org/10.1145/2534008.2534011

Published: 06 November 2013 Publication History

Abstract

Reflections in image sequences violate the single layer model used by most current image processing techniques. As a result reflections cause many techniques to fail e.g. detection, tracking, motion estimation, etc. Recent work was proposed by Ahmed et al. [5] to detect reflections. Their technique is robust to pathological motion and motion blur. This paper has three main contributions. The first simplifies and fully automates the technique of Ahmed et al. User feedback is common in post-production video manipulation tools. Hence in the second contribution we propose an effective way of integrating few user-assisted masks to improve detection rates. The third contribution of this paper is an application for reflection detection. Here we explore better feature point tracking for the regions detected as reflection. Tracks usually die quickly in such regions due to temporal color inconsistencies. In this paper we show that the lifespan of such tracks can be extended through layer separation. Results show reduction in missed detections and in computational load over Ahmed et al. Results also show the generation of more reliable tracks despite strong layer mixing.

References

[1]

Adobe Inc., After Effects. www.adobe.com.

[2]

The Foundry, Nuke, Furnace Suite. www.thefoundry.co.uk.

[3]

T. Aach, C. Mota, I. Stuke, M. Muhlich, and E. Barth. Analysis of Superimposed Oriented Patterns. TIPS, 15(12): 3690--3700, 2006.

Digital Library

[4]

M. Ahmed, F. Pitie, and A. Kokaram. Motion Estimation for Regions of Reflections through Layer Separation. In IEEE Conference on Visual Media Production (CVMP), pages 49--58, 2011.

Digital Library

[5]

M. Ahmed, F. Pitie, and A. Kokaram. Reflection Detection in Image Sequences. In Computer Vision and Pattern Recognition (CVPR), pages 705--712, 2011.

Digital Library

[6]

A. M. Bronstein, M. M. Bronstein, M. Zibulevsky, and Y. Y. Zeevi. Sparse ICA for blind separation of transmitted and reflected images. International Journal of Imaging Systems and Technology, 15(1): 84--91, 2005.

[7]

M. A. Elgharib. Handling Transparency in Digital Video. PhD Thesis, Trinity College Dublin.

[8]

H. Farid and E. Adelson. Separating reflections from images by use of Independent Components Analysis. Journal of the Optical Society of America, 16(9): 2136--2145, 1999.

[9]

R. Ferzli and L. J. Karam. A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB). Transactions on Image Processing (TIPS), 18(4): 717--728, 2009.

Digital Library

[10]

Y. Freund and R. E. Schapire. A decision-theoretic generalization of on-line learning and an application to boosting. In European Conference on Computational Learning Theory, pages 23--37, 1995.

Digital Library

[11]

A. Levin, A. Zomet, and Y. Weiss. Separating reflections from a single image using local features. In Conference on Computer Vision and Pattern Recognition (CVPR), pages 306--313, 2004.

[12]

D. G. Lowe. Distinctive Image Features from Scale-Invariant Keypoints. International Journal of Computer Vision (IJCV), 60(2): 91--110, 2004.

Digital Library

[13]

D. Ring and F. Pitie. Feature-Assisted Sparse to Dense Motion Estimation using Geodesic Distances. In International Machine Vision and Image Processing Conference, pages 7--12, 2009.

Digital Library

[14]

B. Sarel. Separation Transparent Layers in Images and Video. PhD Thesis, Weizmann Institute of Science.

Digital Library

[15]

I. Stuke, T. Aach, E. Barth, and C. Mota. Multiple-Motion-Estimation by Block-matching using MRF. International Journal of Computer and Information Science, 5(1), 2004.

[16]

R. Szeliski, S. Avidan, and P. Anandan. Layer extraction from multiple images containing reflections and transparency. In Computer Vision and Pattern Recognition (CVPR), pages 246--253, 2000.

[17]

C. T. Takeo and T. Kanade. Detection and tracking of point features. Carnegie Mellon University Technical Report CMU-CS-91-132, 1991.

[18]

Y. Weiss. Deriving intrinsic images from image sequences. In International Conference on Computer Vision (ICCV), pages 68--75, 2001.

Cited By

Zhang TTang C(2018)Multiple-target tracking on mixed images with reflections and occlusionsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2018.02.00152(45-57)Online publication date: Apr-2018
https://doi.org/10.1016/j.jvcir.2018.02.001
Nandoriya AElgharib MKim CHefeeda MMatusik W(2017)Video Reflection Removal Through Spatio-Temporal Optimization2017 IEEE International Conference on Computer Vision (ICCV)10.1109/ICCV.2017.264(2430-2438)Online publication date: Oct-2017
https://doi.org/10.1109/ICCV.2017.264
Chen HTang C(2015)Robust tracking using visual cue integration for mobile mixed imagesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2015.04.00630:C(208-218)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1016/j.jvcir.2015.04.006
Show More Cited By

Index Terms

User-assisted reflection detection and feature point tracking
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Tracking
      2. Computer vision tasks
        Scene understanding
        Video summarization
2. Information systems
  1. Information systems applications
    1. Multimedia information systems

Recommendations

Reflection detection in image sequences
CVPR '11: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition

Reflections in image sequences consist of several layers superimposed over each other. This phenomenon causes many image processing techniques to fail as they assume the presence of only one layer at each examined site e.g. motion estimation and object ...
Robust tracking using visual cue integration for mobile mixed images

Maximizing observation likelihood to optimize particle weights under reflections.Combining co-inference and maximum likelihood for visual cue integration.Co-inference is combined with SIR to avoid the particle degeneracy problem.Maximum likelihood ...
Light Flickering Guided Reflection Removal
Abstract
When photographing through a piece of glass, reflections usually degrade the quality of captured images or videos. In this paper, by exploiting periodically varying light flickering, we investigate the problem of removing strong reflections from ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CVMP '13: Proceedings of the 10th European Conference on Visual Media Production

November 2013

166 pages

ISBN:9781450325899

DOI:10.1145/2534008

Conference Chair:
Anil Kokaram
Youtube/Google Inc
,
Program Chair:
Peter Hall
University of Bath, UK

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Google Inc.
ACM: Association for Computing Machinery
NVIDIA
VES: Visual Effects Society
framestore: framestore
fxphd: fxphd Pty. Ltd.
Escape Studios: Escape Studios
Intel: Intel

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 November 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

CVMP '13

Sponsor:

ACM
VES
framestore
fxphd
Escape Studios
Intel

CVMP '13: Conference on Visual Media Production 2013

November 6 - 7, 2013

London, United Kingdom

Acceptance Rates

CVMP '13 Paper Acceptance Rate 18 of 28 submissions, 64%;

Overall Acceptance Rate 40 of 67 submissions, 60%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
98
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang TTang C(2018)Multiple-target tracking on mixed images with reflections and occlusionsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2018.02.00152(45-57)Online publication date: Apr-2018
https://doi.org/10.1016/j.jvcir.2018.02.001
Nandoriya AElgharib MKim CHefeeda MMatusik W(2017)Video Reflection Removal Through Spatio-Temporal Optimization2017 IEEE International Conference on Computer Vision (ICCV)10.1109/ICCV.2017.264(2430-2438)Online publication date: Oct-2017
https://doi.org/10.1109/ICCV.2017.264
Chen HTang C(2015)Robust tracking using visual cue integration for mobile mixed imagesJournal of Visual Communication and Image Representation10.1016/j.jvcir.2015.04.00630:C(208-218)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1016/j.jvcir.2015.04.006
Zhang TChen HTang C(2015)Multi-target Tracking Using Sample-Based Data Association for Mixed ImagesAdvances in Visual Computing10.1007/978-3-319-27857-5_12(127-137)Online publication date: 18-Dec-2015
https://doi.org/10.1007/978-3-319-27857-5_12
Chen HTang C(2014)Visual tracking using Blind Source Separation for mixed images2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2014.6854866(6548-6552)Online publication date: May-2014
https://doi.org/10.1109/ICASSP.2014.6854866
Toyokawa KOno SYamagishi MYamada I(2014)Detecting edges of reflections from a single image via convex optimization2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2014.6854231(3400-3404)Online publication date: May-2014
https://doi.org/10.1109/ICASSP.2014.6854231

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten