skip to main content
research-article
Free access

The Frankencamera: an experimental platform for computational photography

Published: 01 November 2012 Publication History

Abstract

Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and application programming interface (API) for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond timescale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations, we demonstrate several computational photography applications: high dynamic range (HDR) viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, and inertial measurement unit (IMU)-based hand shake detection. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step toward creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.

References

[1]
Adams, A., Gelfand, N., and Pulli, K. Viewfinder alignment. 2 (2008), 597--606.
[2]
Bhat, P., Zitnick, C.L., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B., Kang, S.B. Using photographs to enhance videos of a static scene. In (2007).
[3]
Bramberger, M., Doblander, A., Maier, A., Rinner, B., Schwabach, H. Distributed embedded smart cameras for surveillance applications., 2 (2006), 68--75.
[4]
The CHDK Project, 2010.
[5]
Debevec, P.E., Malik, J. Recovering high dynamic range radiance maps from photographs. In (New York, NY, USA, 1997), ACM Press/ Addison-Wesley Publishing Co, 3 69--378.
[6]
Durand, F. private communication, 2009.
[7]
Eisemann, E., Durand, F. Flash photography enhancement via intrinsic relighting., 3 (2004), 673--678.
[8]
Hengstler, S., Prashanth, D., Fong, S., Aghajan, H. Mesheye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance. In, 360--369.
[9]
Joshi, N., Kang, S.B., Zitnick, C.L., Szeliski, R. Image deblurring using inertial measurement sensors., 3 (Aug. 2010).
[10]
Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R. High dynamic range video. In (2003), ACM, New York, NY, 319--325.
[11]
Kleihorst, R., Schueler, B., Danilin, A., Heijligers, M. Smart camera mote with high performance vision system. In ACM SenSys 2006 Workshop on Distributed Smart Cameras (DSC 2006) (Oct. 2006).
[12]
The Magic Lantern project, 2010.
[13]
Mann, S., Picard, R.W. On being 'undigital' with digital cameras: extending dynamic range by combining differently exposed pictures. In (1995), 442--448.
[14]
Mertens, T., Kautz, J., Reeth, F.V. Exposure fusion. In (2007).
[15]
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K. Digital photography with flash and no-flash image pairs. In (2004), ACM, New York, NY, 664--672.
[16]
Rahimi, M., Baer, R., Iroezi, O., Garcia, J.C., Warrior, J., Estrin, D., Srivastava, M. Cyclops: in situ image sensing and interpretation in wireless sensor networks. In (2005), 192--204.
[17]
Raskar, R., Agrawal, A., Tumblin, J. Coded exposure photography: motion deblurring using fluttered shutter. In (2006), ACM, New York, NY, 795--804.
[18]
Raskar, R., Tumblin, J. Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors, A K Peters, Natick, MA, 2010, in press.
[19]
Reinhard, E., Ward, G., Pattanaik, S., Debevec, P. High Dynamic Range Imaging - Acquisition, Display and Image-based Lighting, Morgan Kaufman Publishers, San Francisco, CA, 2006.
[20]
Rowe, A., Goode, A., Goel, D., Nourbakhsh, I. CMUcam3: An Open Programmable Embedded Vision Sensor. Technical Report RITR-07-13, Carnegie Mellon Robotics Institute, May 2007.
[21]
Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M. High performance imaging using large camera arrays. In (2005), ACM, New York, NY, 765--776.
[22]
Wolf, W., Ozer, B., Lv, T. Smart cameras as embedded systems. (2002), 48--53.

Cited By

View all
  • (2023)Attention Mechanism Architecture for Arabic Sentiment AnalysisACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357826522:4(1-26)Online publication date: 24-Mar-2023
  • (2023)Tamil Offensive Language Detection: Supervised versus Unsupervised Learning ApproachesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357586022:4(1-14)Online publication date: 24-Mar-2023
  • (2023)Dynamic Convolution-based Encoder-Decoder Framework for Image Captioning in HindiACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357389122:4(1-18)Online publication date: 24-Mar-2023
  • Show More Cited By

Index Terms

  1. The Frankencamera: an experimental platform for computational photography

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Communications of the ACM
    Communications of the ACM  Volume 55, Issue 11
    November 2012
    104 pages
    ISSN:0001-0782
    EISSN:1557-7317
    DOI:10.1145/2366316
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 01 November 2012
    Published in CACM Volume 55, Issue 11

    Permissions

    Request permissions for this article.

    Check for updates

    Qualifiers

    • Research-article
    • Popular
    • Refereed

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)459
    • Downloads (Last 6 weeks)88
    Reflects downloads up to 20 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Attention Mechanism Architecture for Arabic Sentiment AnalysisACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357826522:4(1-26)Online publication date: 24-Mar-2023
    • (2023)Tamil Offensive Language Detection: Supervised versus Unsupervised Learning ApproachesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357586022:4(1-14)Online publication date: 24-Mar-2023
    • (2023)Dynamic Convolution-based Encoder-Decoder Framework for Image Captioning in HindiACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357389122:4(1-18)Online publication date: 24-Mar-2023
    • (2023)A Comparative Study of Speaker Role Identification in Air Traffic Communication Using Deep Learning ApproachesACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357279222:4(1-17)Online publication date: 24-Mar-2023
    • (2022)Loan Default Prediction Based on Convolutional Neural Network and LightGBMInternational Journal of Data Warehousing and Mining10.4018/IJDWM.31582319:1(1-16)Online publication date: 29-Dec-2022
    • (2022)HateCircle and Unsupervised Hate Speech Detection Incorporating Emotion and Contextual SemanticsACM Transactions on Asian and Low-Resource Language Information Processing10.1145/357691322:4(1-28)Online publication date: 19-Dec-2022
    • (2022)KineCAMProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/35336135:4(1-9)Online publication date: 7-Sep-2022
    • (2022)Low-Cost Online Convolution Checksum CheckerIEEE Transactions on Very Large Scale Integration (VLSI) Systems10.1109/TVLSI.2021.311951130:2(201-212)Online publication date: Feb-2022
    • (2018)A Dataset of Flash and Ambient Illumination Pairs from the CrowdComputer Vision – ECCV 201810.1007/978-3-030-01240-3_39(644-660)Online publication date: 8-Sep-2018
    • (2015)High Dynamic Range ImagingWiley Encyclopedia of Electrical and Electronics Engineering10.1002/047134608X.W8265(1-42)Online publication date: 15-Jun-2015

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Digital Edition

    View this article in digital edition.

    Digital Edition

    Magazine Site

    View this article on the magazine site (external)

    Magazine Site

    Login options

    Full Access

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media