Skip to main content
SpringerLink
Log in

The eISP low-power and tiny silicon footprint programmable video architecture

  • Special Issue
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

CMOS sensors are now more and more frequently integrated into popular consumer products. Images from these sensors thus need to be digitally processed for display purposes. To do so, CMOS sensors are associated with dedicated components that keep power consumption low. However, use of dedicated components limits hardware flexibility and prevents updating of image processing algorithms. This paper describes the eISP, a programmable processing architecture that combines enough computational efficiency for 1080p HD video with silicon area and power characteristics suitable for the next generation of mobile phones (lower than 1 mm2 and 500 mW in TSMC 65 nm).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. Several processes can be inserted, and the order of processes may differ from the one presented here.

References

  1. Alleysson, D., Süsstrunk, S., Hérault, J.: Linear demosaicing inspired by the human visual system. IEEE Trans. Image Process. 14(4), 439–449 (2005). doi:10.1109/TIP.2004.841200

    Article  Google Scholar 

  2. Bigas, M., Cabruja, E., Forest, J., Salvi, J.: Review of CMOS image sensors. Microelectron. J. 37(5), 433–451 (2006)

    Article  Google Scholar 

  3. Cat, H.H., Gentile, A., Eble, J.C., Lee, M., Vendier, O., Joo, Y.J., Wills, D.S., Brooke, M., Jokerst, N.M., Brown, A.S.: SIMPil: An OE integrated SIMD architecture for focal plane processing applications. In: Proceedings of Third International Conference on Massively Parallel Processing using Optical Interconnections, Maui, pp. 44–52 (1996)

  4. Chen, J., Chien, S.Y.: Crisp: coarse-grained reconfigurable image stream processor for digital still cameras and camcorders. IEEE Trans. Circuits Syst. Video Technol. 18(9), 1223–1236 (2008). doi:10.1109/TCSVT.2008.928529

    Article  Google Scholar 

  5. Chen, T., Catrysse, P., Gamal, A.E., W.B.: How small should pixel size be. In: Proceedings of SPIE, April 2000, pp. 451–459 (2000)

  6. Cheung, N., Henkel, J., Parameswaran, S.: Rapid configuration and instruction selection for an ASIP: a case study. In: Design, Automation and Test in Europe Conference and Exhibition, 2003, pp. 802–807 (2003)

  7. Chou, C., Liu, K., Lee, W.: Adaptive color filter array demosaicking based on constant hue and local properties of luminance. In: Advances in Image and Video Technology, pp. 357–370 (2007)

  8. Ding, L., Goshtasby, A.: On the canny edge detector. Pattern Recognit. 34(3), 721–725 (2001)

    Article  MATH  Google Scholar 

  9. Gentile, A., Vitabile, S., Verdoscia, L., Sorbello, F.: Image processing chain for digital still cameras based on the SIMPil architecture. In: 2005 ICPP 2005 Workshops International Conference Workshops on Parallel Processing, pp. 215–222 (2005). doi:10.1109/ICPPW.2005.41

  10. Goodwin, D., Rowen, C., Martin, G.: Configurable multi-processor platforms for next generation embedded systems. In: 2007 ASP-DAC ’07 Asia and South Pacific Design Automation Conference, pp. 744–746 (2007). doi:10.1109/ASPDAC.2007.358076

  11. Graham, S.L., Kessler, P.B., Mckusick, M.K.: Gprof: a call graph execution profiler. SIGPLAN Not 17(6), 120–126 (1982). doi:10.1145/872726.806987

    Google Scholar 

  12. Horiuchi, T., Watanabe, K., Tominaga, S.: Adaptive filtering for color image sharpening and denoising. In: 2007 ICIAPW 2007 14th International Conference on Image Analysis and Processing Workshops, pp. 196–201 (2007). doi:10.1109/ICIAPW.2007.11

  13. Kehtarnavaz, N., Oh, H.J., Yoo, Y.: Development and real-time implementation of auto white balancing scoring algorithm. Real Time Imaging 8(5), 379–386 (2002). doi:10.1006/rtim.2001.0287

  14. Khailany, B., Dally, W.J., Kapasi, U.J., Mattson, P., Namkoong, J., Owens, J.D., Towles, B., Chang, A., Rixner, S.: Imagine: Media processing with streams. IEEE Micro 21(2), 35–46 (2001). doi:10.1109/40.918001

  15. Khailany, B., Williams, T., Lin, J., Long, E., Rygh, M., Tovey, D., Dally, W.: A programmable 512 GOPS stream processor for signal, image, and video processing. IEEE J. Solid State Circuits 43(1), 202–213 (2008). doi:10.1109/JSSC.2007.909331

    Google Scholar 

  16. Khawam, S., Nousias, I., Milward, M., Yi, Y., Muir, M., Arslan, T.: The reconfigurable instruction cell array. IEEE Trans. Very Large Scale Integr. Syst. 16(1), 75–85 (2008). doi:10.1109/TVLSI.2007.912133

    Article  Google Scholar 

  17. Kyo, S., Okazaki, S., Arai, T.: An integrated memory array processor architecture for embedded image recognition systems. In: 2005 ISCA ’05 Proceedings 32nd International Symposium on Computer Architecture, pp. 134–145 (2005). doi:10.1109/ISCA.2005.11

  18. Lee, S.Y., Kumar, Y., Cho, J.M., Kim, S.W.L.S.W.: Enhanced autofocus algorithm using robust focus measure and fuzzy reasoning. IEEE Trans. Circuits Syst. Video Technol. 18, 1237–1246 (2008). doi:10.1109/TCSVT.2008.924105

    Article  Google Scholar 

  19. Meylan, L., Süsstrunk, S.: High dynamic range image rendering using a Retinex-based Adaptive filter. IEEE Trans. Image Process 15(9), 2820–2830 (2006). doi:http://rr.epfl.ch/12/

    Google Scholar 

  20. Mucci, C., Campi, F., Deledda, A., Fazzi, A., Ferri, M., Bocchi, M.: A cycle-accurate ISS for a dynamically reconfigurable processor architecture. Parallel and Distributed Processing Symposium, 2005 Proceedings 19th IEEE International 4(8), 8 (2005). doi:10.1109/IPDPS.2005.14

  21. Nethercote, N.: Dynamic binary analysis and instrumentation. Ph.D. thesis, Trinity College (2004)

  22. Pinto, C.A., Beric, A., Singh, S.P., Farfade, S.: Hiveflex-video vsp1: Video signal processing architecture for video coding and post-processing. In: Multimedia, 2006 ISM’06 Eighth IEEE International Symposium, pp. 493–500 (2006). doi:10.1109/ISM.2006.83

  23. Ramanath, R., Snyder, W.E., Bilbro, G.L., Iii, W.A.S.: Demosaicking methods for bayer color arrays. J. Electron. Imaging 11, 306–315 (2002)

    Article  Google Scholar 

  24. Russ, J.C.: The Image Processing Handbook, 5th edn. CRC Press, New York (1997)

  25. Sakaue, S., Nakayama, M., Tamura, A., Maruno, S.: Adaptive gamma processing of the video cameras for the expansion of the dynamic range. IEEE Trans. Consum. Electron. 41(3), 555–562 (1995). doi:10.1109/30.468094

    Article  Google Scholar 

  26. Shimizu, S., Kondo, T., Kohashi, T., Tsurata, M., Komuro, T.: A new algorithm for exposure control based on fuzzy logic for video cameras. IEEE Trans. Consum. Electron. 38(3), 617–623 (1992). doi:10.1109/30.156745

    Article  Google Scholar 

  27. Tamburrino, D., Alleysson, D., Meylan, L., Süsstrunk, S.: Digital camera workflow for high dynamic range images using a model of retinal processing. In: IS& T/SPIE Electronic Imaging: Digital Photography IV, vol. 6817 (2008)

  28. Thevenin, M., Paindavoine, M., Letellier, L., Heyrman, B.: Processor extensions for CMOS sensor imaging in camera phone. In: Photonics Europe 2008—Photonics in Multimedia, vol. 7001 (2008)

  29. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the 1998 IEEE International Conference on Computer Vision, IEEE, Bombay, India (1998)

  30. van de Weijer, J., Gevers, T.: Color constancy based on the grey-edge hypothesis. In: ICIP (2), pp. 722–725 (2005)

  31. Yoo, Y., Lee, S., Choe, W., Kim, C.Y.: CMOS image sensor noise reduction method for image signal processor in digital cameras and camera phones. In: Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, SPIE Conference, vol. 6502 (2007). doi:10.1117/12.702758

  32. Zhiming, W., Jianhua, T.: A fast implementation of adaptive histogram equalization. In: 2006 8th International Conference on Signal Processing (2006). doi:10.1109/ICOSP.2006.345602

Download references

Author information

Authors and Affiliations

  1. Embedded Computing Lab, CEA, LIST, PC 94, 91191, Gif Sur Yvette, France

    Mathieu Thevennin, Laurent Letellier & Renaud Schmit

  2. LEAD UMR 5022 CNRS, Université de Bourgogne, AAFE Esplanade Erasme, BP 26513, 21065, Dijon Cedex, France

    Michel Paindavoine

  3. LE2i UMR 5158 CNRS, Université de Bourgogne, Rue Alain Savary, 21000, Dijon, France

    Barthelemy Heyrman

Authors
  1. Mathieu Thevennin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michel Paindavoine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Laurent Letellier
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Renaud Schmit
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Barthelemy Heyrman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mathieu Thevennin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Thevennin, M., Paindavoine, M., Letellier, L. et al. The eISP low-power and tiny silicon footprint programmable video architecture. J Real-Time Image Proc 6, 33–46 (2011). https://doi.org/10.1007/s11554-010-0163-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-010-0163-8

Keywords

Search

Navigation