Andrew Adams
Mark Horowitz
Wojciech Matusik
Abstract
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address this problem, we have designed and implemented an open architecture and application programming interface (API) for such cameras: the Frankencamera. It consists of a base hardware specification, a software stack based on Linux, and an API for C++. Our architecture permits control and synchronization of the sensor and image processing pipeline at the microsecond timescale, as well as the ability to incorporate and synchronize external hardware like lenses and flashes. This paper specifies our architecture and API, and it describes two reference implementations we have built. Using these implementations, we demonstrate several computational photography applications: high dynamic range (HDR) viewfinding and capture, automated acquisition of extended dynamic range panoramas, foveal imaging, and inertial measurement unit (IMU)-based hand shake detection. Our goal is to standardize the architecture and distribute Frankencameras to researchers and students, as a step toward creating a community of photographer-programmers who develop algorithms, applications, and hardware for computational cameras.
- Adams, A., Gelfand, N., and Pulli, K. Viewfinder alignment. 2 (2008), 597--606.Google Scholar
- Bhat, P., Zitnick, C.L., Snavely, N., Agarwala, A., Agrawala, M., Cohen, M., Curless, B., Kang, S.B. Using photographs to enhance videos of a static scene. In (2007).Google Scholar
- Bramberger, M., Doblander, A., Maier, A., Rinner, B., Schwabach, H. Distributed embedded smart cameras for surveillance applications., 2 (2006), 68--75. Google ScholarDigital Library
- The CHDK Project, 2010.Google Scholar
- Debevec, P.E., Malik, J. Recovering high dynamic range radiance maps from photographs. In (New York, NY, USA, 1997), ACM Press/ Addison-Wesley Publishing Co, 3 69--378. Google ScholarDigital Library
- Durand, F. private communication, 2009.Google Scholar
- Eisemann, E., Durand, F. Flash photography enhancement via intrinsic relighting., 3 (2004), 673--678. Google ScholarDigital Library
- Hengstler, S., Prashanth, D., Fong, S., Aghajan, H. Mesheye: a hybrid-resolution smart camera mote for applications in distributed intelligent surveillance. In, 360--369. Google ScholarDigital Library
- Joshi, N., Kang, S.B., Zitnick, C.L., Szeliski, R. Image deblurring using inertial measurement sensors., 3 (Aug. 2010).Google Scholar
- Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R. High dynamic range video. In (2003), ACM, New York, NY, 319--325. Google ScholarDigital Library
- Kleihorst, R., Schueler, B., Danilin, A., Heijligers, M. Smart camera mote with high performance vision system. In ACM SenSys 2006 Workshop on Distributed Smart Cameras (DSC 2006) (Oct. 2006).Google Scholar
- The Magic Lantern project, 2010.Google Scholar
- Mann, S., Picard, R.W. On being 'undigital' with digital cameras: extending dynamic range by combining differently exposed pictures. In (1995), 442--448.Google Scholar
- Mertens, T., Kautz, J., Reeth, F.V. Exposure fusion. In (2007).Google Scholar
- Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., Toyama, K. Digital photography with flash and no-flash image pairs. In (2004), ACM, New York, NY, 664--672. Google ScholarDigital Library
- Rahimi, M., Baer, R., Iroezi, O., Garcia, J.C., Warrior, J., Estrin, D., Srivastava, M. Cyclops: in situ image sensing and interpretation in wireless sensor networks. In (2005), 192--204. Google ScholarDigital Library
- Raskar, R., Agrawal, A., Tumblin, J. Coded exposure photography: motion deblurring using fluttered shutter. In (2006), ACM, New York, NY, 795--804. Google ScholarDigital Library
- Raskar, R., Tumblin, J. Computational Photography: Mastering New Techniques for Lenses, Lighting, and Sensors, A K Peters, Natick, MA, 2010, in press. Google ScholarDigital Library
- Reinhard, E., Ward, G., Pattanaik, S., Debevec, P. High Dynamic Range Imaging - Acquisition, Display and Image-based Lighting, Morgan Kaufman Publishers, San Francisco, CA, 2006. Google ScholarDigital Library
- Rowe, A., Goode, A., Goel, D., Nourbakhsh, I. CMUcam3: An Open Programmable Embedded Vision Sensor. Technical Report RITR-07-13, Carnegie Mellon Robotics Institute, May 2007.Google Scholar
- Wilburn, B., Joshi, N., Vaish, V., Talvala, E.V., Antunez, E., Barth, A., Adams, A., Horowitz, M., Levoy, M. High performance imaging using large camera arrays. In (2005), ACM, New York, NY, 765--776. Google ScholarDigital Library
- Wolf, W., Ozer, B., Lv, T. Smart cameras as embedded systems. (2002), 48--53. Google ScholarDigital Library
Index Terms
- The Frankencamera: an experimental platform for computational photography
Recommendations
The Frankencamera: an experimental platform for computational photography
Although there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address ...
The Frankencamera: an experimental platform for computational photography
SIGGRAPH '10: ACM SIGGRAPH 2010 papersAlthough there has been much interest in computational photography within the research and photography communities, progress has been hampered by the lack of a portable, programmable camera with sufficient image quality and computing power. To address ...
Comments