Skip to main content
SpringerLink
Log in

Real-time foveation filtering using nonlinear Mipmap interpolation

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In recent years, several techniques have been proposed to simulate the gaze effect of the Human Visual System (HVS). It is believed that this effect is due to the foveation filtering. Current techniques to simulate the foveation filtering in computer graphics are either slow or suffer from artifacts and limitations. In this paper, we present a new approach of foveation filtering based on the Mipmap Pyramid of the current view by considering the relationship between the Gaussian kernel and Mipmap level. Due to the nonlinear Mipmap interpolation under the Bilateral Filtering scheme, we are able to simulate the foveation filtering more naturally and efficiently than in previous work. Moreover, a detail enhancement method based on the Cornsweet illusion is proposed to augment the gazing effect. We demonstrate our new approach with a variety of examples and provide comparisons with recent approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Badamchizadeh, M.A., Aghagolzadeh, A.: Comparative study of unsharp masking methods for image enhancement. In: International Conference on Image and Graphics, pp. 27–30 (2004)

  2. Banks, M.S., Sekuler, A.B., Anderson, S.J.: Peripheral spatial vision: Limits imposed by optics, photoreceptors, and receptor pooling. J. Opt. Soc. Am. 8, 1775–1787 (1991)

    Article  Google Scholar 

  3. Bjorke, K.: High-quality filtering, GPU Gems: programming techniques. In: Tips & Tricks for Real-Time Graphics, pp. 391–415 (2004)

  4. Burt, P.J., Adelson, E.H.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 532–540 (1983)

    Article  Google Scholar 

  5. Cornsweet, T.: Visual Perception. Academic Press, San Diego (1970)

    Google Scholar 

  6. Dante Garcia, D., Barsky, B.A.: CWhatUC: Software Tools for Predicting, Visualizing and Simulating Corneal Visual Acuity. Sinauer Associates (2000)

  7. Dhavale, N., Itti, L.: Saliency-based multifoveated MPEG compression. In: Proc. 7th International Symposium on Signal Processing and Its Applications, vol. 1, pp. 229–232 (2003)

  8. Dikici, C., Civanlar, R., Isil Bozma, H.: Fovea based coding for video streaming. Image Anal. Recognit. 3211, 285–294 (2004)

    Google Scholar 

  9. Duchowski, A.T.: Acuity-matching resolution degradation through wavelet coefficient scaling. IEEE Trans. Image Process. 9, 1437–1440 (2000)

    Article  Google Scholar 

  10. Duchowski, A.T., Coltekin, A.: Foveated gaze-contingent displays for peripheral LOD management, 3D visualization, and stereo imaging. ACM Trans. Multimed. Comput. Commun. Appl. (TOMCCAP) 3(4), 1–18 (2007)

    Article  Google Scholar 

  11. Etienne-Cummings, R., van der Spiegel, J., Mueller, P., Zhang, M.-Z.: A foveated silicon retina for two-dimensional tracking. IEEE Trans. Circuits Syst. 47, 504–517 (2000)

    Article  Google Scholar 

  12. Farbman, Z., Fattal, R., Lischinski, D., Szeliski, R.: Edge-preserving decompositions for multi-scale tone and detail manipulation. ACM Trans. Graph. 27 (2008)

  13. Fattal, R., Agrawala, M., Rusinkiewicz, S.: Multiscale shape and detail enhancement from multi-light image collections. ACM Trans. Graph. 26 (2007)

  14. Geisler, W.S., Banks, M.S.: Visual Performance. Handbook of Optics. McGraw-Hill, New York (1995)

    Google Scholar 

  15. Geisler, W.S., Perry, J.S.: A real-time foveated multiresolution system for low-bandwidth video communication. Proc. SPIE 3299, 294–305 (1998)

    Article  Google Scholar 

  16. Itti, L.: Automatic foveation for video compression using a neurobiological model of visual attention. IEEE Trans. Image Process. 13, 1304–1318 (2004)

    Article  Google Scholar 

  17. Kinser, J.M.: Foveation from pulse images. In: Proceedings of the 1999 International Conference on Information Intelligence and Systems, pp. 86–89 (1999)

  18. Klarquist, W.N., Bovik, A.C.: FOVEA: A foveated vergent active stereo vision system for dynamic three-dimensional scene recovery. IEEE Trans. Robot. Autom. 14, 755–770 (1998)

    Article  Google Scholar 

  19. Koz, A., Alatan, A.: Foveated image watermarking. In: International Conference on Image Processing, vol. 3, pp. 661–664 (2002)

  20. Krawczyk, G., Myszkowski, K., Seidel, H.-P.: Contrast restoration by adaptive countershading. Comput. Graph. Forum 26 (2007)

  21. Lee, S., Bovik, A.C.: Fast algorithms for foveated video processing. IEEE Trans. Circuits Syst. Video Technol. 13, 149–162 (2003)

    Article  Google Scholar 

  22. Lee, S., Pattichis, M.S., Bovik, A.C.: Foveated video compression with optimal rate control. IEEE Trans. Image Process. 10, 911–992 (2001)

    MathSciNet  Google Scholar 

  23. Lee, S., Pattichis, M.S., Bovik, A.C.: Foveated video quality assessment. IEEE Trans. Multimed. 4, 129–132 (2002)

    Article  Google Scholar 

  24. Levoy, M., Whitaker, R.: Gaze-directed volume rendering. Comput. Graph. 24, 217–223 (1990)

    Article  Google Scholar 

  25. Murphy, H.A., Duchowski, A.T., Tyrrell, R.A.: Hybrid image/model-based gaze-contingent rendering. ACM Trans. Appl. Percept. (TAP) 5(4), 1–21 (2009)

    Article  Google Scholar 

  26. Perry, J.S., Geisler, W.S.: Gaze-contingent real-time simulation of arbitrary visual fields. Proc. SPIE 4662, 57–69 (2002)

    Article  Google Scholar 

  27. Popkin, T., Cavallaro, A., Hands, D.: An accurate and efficient method for smoothly space-variant Gaussian blurring. IEEE Trans. Image Process. 18 (2009)

  28. Ritschel, T., Smith, K., Ihrke, M., Grosch, T., Myszkowski, K., Seidel, H.-P.: 3D unsharp masking for scene coherent enhancement. ACM Trans. Graph. 27, 1–8 (2008)

    Google Scholar 

  29. Robson, J.G., Graham, N.: Probability summation and regional variation in contrast sensitivity across the visual field. Vis. Res. 21, 409–418 (1981)

    Article  Google Scholar 

  30. Smith, K., Landes, P.-E., Thollot, J., Myszkowski, K.: Apparent greyscale: a simple and fast conversion to perceptually accurate images and video. Comput. Graph. Forum 27, 193–200 (2008)

    Article  Google Scholar 

  31. Stiles, W.H., Crawford, B.H.: The luminous efficiency of rays entering the eye pupil at different points. Proc. R. Soc. Lond. B 112, 428–450 (1933)

    Article  Google Scholar 

  32. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision, pp. 839–846 (1998)

  33. Tsumura, N., Endo, C., Haneishi, H., Miyake, Y.: Image compression and decompression based on gazing area. Proc. SPIE 2657, 361–367 (1996)

    Article  Google Scholar 

  34. Wallace, R.S., Ong, P.-W., Bederson, B.B., Schwartz, E.L.: Spacevariant image processing. Int. J. Comput. Vis. 13, 71–90 (1994)

    Article  Google Scholar 

  35. Wandell, B.A.: Foundations of Vision. Sinauer Associates (1995)

  36. Wang, Z., Bovik, A.C.: Foveated image and video coding. In: Digital Video, Image Quality and Perceptual Coding, pp. 431–457. CRC Press, Boca Raton (2006)

    Google Scholar 

  37. Ward, G., Rushmeier, H., Piatko, C.: A visibility matching tone reproduction operator for high dynamic range scenes. IEEE Trans. Vis. Comput. Graph. 4, 291–306 (1997)

    Google Scholar 

  38. Yitzhaky, Y., Peli, E.: Vision-model-based image foveation and motion estimation. Opt. Eng. 44, 107004-1–107004-11 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. State Key Lab of CAD&CG, Zhejiang University, Hangzhou, 310027, P.R. China

    Xin Zhang, Zhangye Wang, Ran Wang, Zhonglei Yang, Wei Chen & Qunsheng Peng

Authors
  1. Xin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhangye Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ran Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhonglei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Qunsheng Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qunsheng Peng.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Zhang, X., Wang, Z., Wang, R. et al. Real-time foveation filtering using nonlinear Mipmap interpolation. Vis Comput 26, 923–932 (2010). https://doi.org/10.1007/s00371-010-0432-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-010-0432-3

Keywords

Search

Navigation