Abstract
We present a practical system prototype to convincingly restore eye contact between two video chat participants, with a minimal amount of constraints. The proposed six-fold camera setup is easily integrated into the monitor frame, and is used to interpolate an image as if its virtual camera captured the image through a transparent screen. The peer user has a large freedom of movement, resulting in system specifications that enable genuine practical usage. Our software framework thereby harnesses the powerful computational resources inside graphics hardware, and maximizes arithmetic intensity to achieve over real-time performance up to 42 frames per second for 800 ×600 resolution images. Furthermore, an optimal set of fine tuned parameters are presented, that optimizes the end-to-end performance of the application to achieve high subjective visual quality, and still allows for further algorithmic advancement without loosing its real-time capabilities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Gemmell, J., Toyama, K., Zitnick, C.L., Kang, T., Seitz, S.: Gaze awareness for video-conferencing: A software approach. IEEE MultiMedia 7(4), 26–35 (2000)
Criminisi, A., Shotton, J., Blake, A., Torr, P.H.S.: Gaze manipulation for one-to-one teleconferencing. In: Proceedings of International Conference on Computer Vision, Washington, DC, USA (2003)
Schreer, O., Brandenburg, N., Askar, S., Trucco, M.: A virtual 3d video-conference system providing semi-immersive telepresence: A real-time solution in hardware and software. In: Proceedings of the eBusiness-eWork Conference, Venice, Italy, pp. 184–190 (2001)
Baker, H.H., Bhatti, N.T., Tanguay, D., Sobel, I., Gelb, D., Goss, M.E., Culbertson, W.B., Malzbender, T.: The coliseum immersive teleconferencing system. In: Proceedings of the International Workshop on Immersive Telepresence, Juan-les-Pins, France (2002)
Chien, S.Y., Yu, S.H., Ding, L.F., Huang, Y.N., Chen, L.G.: Efficient stereo video coding system for immersive teleconference with two-stage hybrid disparity estimation algorithm. In: Proceedings of the International Conference on Image Processing, Barcelona, Spain, pp. 749–752 (2003)
Guo, X., Gao, W., Zhao, D.: Motion vector prediction in multiview video coding. In: Proceedings of the International Conference on Image Processing (2005)
Yang, R., Pollefeys, M.: Multi-resolution real-time stereo on commodity graphics hardware. In: Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, June 2003, pp. 211–220 (2003)
Geys, I., Van Gool, L.: Extended view interpolation by parallel use of the GPU and the CPU. In: Proceedings of the Society of Photo-Optical Instrumentation Engineers Conference, December 2004, pp. 96–107 (2004)
Nozick, V., Michelin, S., Arquès, D.: Real-time plane-sweep with local strategy. Journal of WSCG 14 (January 2006)
Rogmans, S., Lu, J., Bekaert, P., Lafruit, G.: Real-time stereo-based view synthesis algorithms: A unified framework and evaluation on commodity GPUs. Special Issue on Advances in Three Dimensional Television and Video of Journal on Signal Processing: Image Communication (January 2009)
Middlebury: Stereo vision page (2001), http://www.middlebury.edu/stereo
Ramanath, R., Snyder, W.E., Bilbro, G.L., Sander III, W.A.: Demosaicking methods for bayer color arrays. Journal of Electronic Imaging 11(3), 306–315 (2002)
Brown, D.C.: Decentering distortion of lenses. Photometric Engineering 32(3), 444–462 (1966)
Magnor, M., Pollefeys, M., Cheung, G., Matusik, W., Theobalt, C.: Video-based rendering. In: ACM SIGGRAPH 2005 Courses, Los Angeles, California (2005)
Yang, R., Welch, G.: Fast image segmentation and smoothing using commodity graphics hardware. Journal of Graphics Tools 7(4), 91–100 (2002)
Yang, R., Welch, G., Bishop, G.: Real-time consensus-based scene reconstruction using commodity graphics hardware. In: Proceedings of the Conference on Computer Graphics and Applications, Washington, DC, USA, pp. 225 (2002)
Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 47(1-3), 7–42 (2002)
Lei, B.J., Hendriks, E.A.: Real-time multi-step view reconstruction for a virtual teleconference system. EURASIP Journal on Applied Signal Processing 2002(10), 1067–1087 (2002)
Geys, I., Koninckx, T.P., Van Gool, L.: Fast interpolated cameras by combining a gpu based plane sweep with a max-flow regularisation algorithm. In: 3DPVT 2004: Proceedings of the 3D Data Processing, Visualization, and Transmission, 2nd International Symposium, Washington, DC, USA, pp. 534–541 (2004)
Rogmans, S., Dumont, M., Cuypers, T., Lafruit, G., Bekaert, P.: Complexity reduction of real-time depth scanning on graphics hardware. In: Proceedings of International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009)
Svoboda, T., Martinec, D., Pajdla, T.: A convenient multi-camera self-calibration for virtual environments. PRESENCE: Teleoperators and Virtual Environments 14(4), 407–422 (2005)
Dumont, M., Maesen, S., Rogmans, S., Bekaert, P.: A prototype for practical eye-gaze corrected video chat on graphics hardware. In: International Conference on Signal Processing and Multimedia Applications, Porto, Portugal, July 2008, pp. 236–243 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Dumont, M., Rogmans, S., Maesen, S., Bekaert, P. (2009). Optimized Two-Party Video Chat with Restored Eye Contact Using Graphics Hardware. In: Filipe, J., Obaidat, M.S. (eds) e-Business and Telecommunications. ICETE 2008. Communications in Computer and Information Science, vol 48. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-05197-5_26
Download citation
DOI: https://doi.org/10.1007/978-3-642-05197-5_26
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-05196-8
Online ISBN: 978-3-642-05197-5
eBook Packages: Computer ScienceComputer Science (R0)