Skip to main content
Springer Nature Link
Log in

Color invariant chroma keying and color spill neutralization for dynamic scenes and cameras

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this article we show how temporal backdrops that alternately change their color rapidly at recording rate can aid chroma keying by transforming color spill into a neutral background illumination. Since the chosen colors sum up to white, the chromatic (color) spill component is neutralized when integrating over both backdrop states. The ability to separate both states additionally allows to compute high-quality alpha mattes. Besides the neutralization of color spill, our method is invariant to foreground colors and supports applications with real-time demands. In this article, we explain different realizations of temporal backdrops and describe how keying and color spill neutralization are carried out, how artifacts resulting from rapid motion can be reduced, and how our approach can be implemented to be compatible with common real-time post-production pipelines.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Ben Ezra, M.: Segmentation with invisible keying signal. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 1, pp. 32–37. IEEE Computer Society, Los Alamitos (2000)

    Google Scholar 

  2. Chaplin, D.J.: Chroma keyer with fringe control offset. uS Patent number 5,313,304 (1994)

  3. Choudhury, B., Singla, D., Chandran, S.: Fast color-space decomposition based environment matting. In: I3D ’08: Proceedings of the 2008 Symposium on Interactive 3D Graphics and Games, p. 1. ACM, New York (2008)

    Google Scholar 

  4. Chuang, Y.Y., Zongker, DE, Hindorff, J., Curless, B., Salesin, D.H., Szeliski, R.: Environment matting extensions: towards higher accuracy and real-time capture. In: SIGGRAPH ’00: ACM SIGGRAPH 2000 Papers, pp. 121–130. ACM, New York (2000)

    Google Scholar 

  5. Chuang, Y.Y., Curless, B., Salesin, D.H., Szeliski, R.: A Bayesian approach to digital matting. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR, vol. 2, pp. 264–271. IEEE Computer Society, Los Alamitos (2001)

    Google Scholar 

  6. Debevec, P., Wenger, A., Tchou, C., Gardner, A., Waese, J., Hawkins, T.: A lighting reproduction approach to live-action compositing. ACM Trans. Graph. 21(3), 547–556 (2002)

    Article  Google Scholar 

  7. Dupont, J., Deschenes, F.: Toward a realistic interpretation of blue-spill for blue-screen matting. In: CRV ’06: Proceedings of the 3rd Canadian Conference on Computer and Robot Vision, p. 33. IEEE Computer Society, Washington (2006)

    Chapter  Google Scholar 

  8. Graham, T., Richard, R.: Flash-based keying. European Patent Application EP1499117 (2005)

  9. Grau, O., Pullen, T., Thomas, G.: A combined studio production system for 3-d capturing of live action and immersive actor feedback. IEEE Trans. Circuits Syst. Video Technol. 14(3), 370–380 (2004)

    Article  Google Scholar 

  10. Grundhöfer, A., Bimber, O.: Virtualstudio2go: digital video composition for real environments. In: SIGGRAPH Asia ’08: ACM SIGGRAPH Asia 2008 Papers, pp. 1–8. ACM, New York (2008)

    Google Scholar 

  11. Grundhöfer, A., Seeger, M., Häntsch, F., Bimber, O.: Dynamic adaptation of projected imperceptible codes. In: ISMAR ’07: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–10. IEEE Computer Society, Washington (2007)

    Chapter  Google Scholar 

  12. Gvili, R., Kaplan, A., Ofek, E., Yahav, G.: Depth keying. In: Woods, A.J., Bolas, M.T., Merritt, J.O., Benton, S.A. (eds.) Stereoscopic Displays and Virtual Reality Systems X. SPIE, vol. 5006, pp. 564–574. SPIE, Bellingham (2003)

    Google Scholar 

  13. Kuechler, M., Kunz, A.: Imperceptible projection blanking for reliable segmentation within mixed reality applications. In: Kjems, E., Blach, R. (eds.) 9th International Workshop on Immersive Projection Technology, 11th Eurographics Workshop on Virtual Environments, pp. 23–30. Eurographics Association, Denmark (2005)

    Google Scholar 

  14. Matusik, W., Pfister, H., Ngan, A., Beardsley, P., Ziegler, R., McMillan, L.: Image-based 3d photography using opacity hulls. ACM Trans. Graph. 21(3), 427–437 (2002)

    Article  Google Scholar 

  15. McGuire, M., Matusik, W.: Real-time triangulation matting using passive polarization. In: SIGGRAPH ’06: ACM SIGGRAPH 2006 Sketches, p. 88. ACM, New York (2006)

    Chapter  Google Scholar 

  16. McGuire, M., Matusik, W., Pfister, H., Hughes, J.F., Durand, F.: Defocus video matting. In: SIGGRAPH ’05: ACM SIGGRAPH 2005 Papers, pp. 567–576. ACM, New York (2005)

    Chapter  Google Scholar 

  17. McGuire, M., Matusik, W., Yerazunis, W.: Practical, Real-Time Studio Matting Using Dual Imagers, pp. 235–244 (2006)

  18. Moon, J.H., Kim, D.O., Park, R.H.: Video matting based on background estimation. In: World Academy of Science, Engineering and Technology, vol. 2 (2005)

  19. RA, Moses, Hart, W.M.: The temporal responsiveness of vision. Adler’s Physiology of the Eye (1987)

  20. Peers, P., Dutré, P.: Wavelet environment matting. In: EGRW ’03: Proceedings of the 14th Eurographics Workshop on Rendering, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, pp. 157–166 (2003)

  21. Porter, T., Duff, T.: Compositing digital images. In: SIGGRAPH ’84: ACM SIGGRAPH 1984 Papers, pp. 253–259. ACM, New York (1984)

    Google Scholar 

  22. Sharma, G.: Digital Color Imaging Handbook. CRC Press, Boca Raton (2002)

    Google Scholar 

  23. Smith, A.R., Blinn, J.F.: Blue screen matting. In: SIGGRAPH ’96: ACM SIGGRAPH 1996 Papers, pp. 259–268. ACM, New York (1996)

    Google Scholar 

  24. Sun, J., Jia, J., Keung Tang, C., Yeung Shum, H.: Poisson matting. In: SIGGRAPH ’04: ACM SIGGRAPH 2004 Papers, vol. 23, pp. 315–321. ACM, New York (2004)

    Chapter  Google Scholar 

  25. Sun, J., Sun, J., Kang, S.B., Xu, Z.B., Tang, X., Shum, H.Y.: Flash cut: foreground extraction with flash and no-flash image pairs. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2007)

  26. Vlahos, P.: Electronic composite photography with color control. uS Patent number 4,007,487 (1977)

  27. Vlahos, P.: Backing color and luminance nonuniformity compensation for linear image compositing. uS Patent number 5,032,901 (1991)

  28. Wang, O., Finger, J., Yang, Q., Davis, J., Yang, R.: Automatic natural video matting with depth. In: PG ’07: Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pp. 469–472. IEEE Computer Society, Washington (2007)

    Google Scholar 

  29. Wenger, A., Gardner, A., Tchou, C., Unger, J., Hawkins, T., Debevec, P.: Performance relighting and reflectance transformation with time-multiplexed illumination. ACM Trans. Graph. 24(3), 756–764 (2005)

    Article  Google Scholar 

  30. Whitesides, T., Walls, M., Paolini, R., Sohn, S., Gates, H., McCreary, M., Jacobson, J.: 10.2: Towards video-rate microencapsulated dual-particle electrophoretic displays. SID Symp. Dig. Tech. Pap. 35(1), 133–135 (2004)

    Article  Google Scholar 

  31. Zhu, J., Yang, Y.H.: Frequency-based environment matting. In: PG ’04: Proceedings of the Computer Graphics and Applications, 12th Pacific Conference, pp. 402–410. IEEE Computer Society, Washington (2004)

    Google Scholar 

  32. Zongker, D.E., Werner, D.M., Curless, B., Salesin, D.H.: Environment matting and compositing. In: SIGGRAPH ’99: ACM SIGGRAPH 1999 Papers, pp. 205–214. ACM, New York (1999)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Bauhaus University Weimar, Bauhausstr. 11, 99423, Weimar, Germany

    Anselm Grundhöfer, Daniel Kurz & Sebastian Thiele

  2. JKU Institute of Computer Graphics, Altenberger Straße 69, 4040, Linz, Austria

    Oliver Bimber

Authors
  1. Anselm Grundhöfer
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Daniel Kurz
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Sebastian Thiele
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Oliver Bimber
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Anselm Grundhöfer.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Grundhöfer, A., Kurz, D., Thiele, S. et al. Color invariant chroma keying and color spill neutralization for dynamic scenes and cameras. Vis Comput 26, 1167–1176 (2010). https://doi.org/10.1007/s00371-010-0464-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0464-8

Keywords