Mohit Gupta
Abstract
The depth resolution achieved by a continuous wave time-of-flight (C-ToF) imaging system is determined by the coding (modulation and demodulation) functions that it uses. Almost all current C-ToF systems use sinusoid or square coding functions, resulting in a limited depth resolution. In this article, we present a mathematical framework for exploring and characterizing the space of C-ToF coding functions in a geometrically intuitive space. Using this framework, we design families of novel coding functions that are based on Hamiltonian cycles on hypercube graphs. Given a fixed total source power and acquisition time, the new Hamiltonian coding scheme can achieve up to an order of magnitude higher resolution as compared to the current state-of-the-art methods, especially in low signal-to-noise ratio (SNR) settings. We also develop a comprehensive physically-motivated simulator for C-ToF cameras that can be used to evaluate various coding schemes prior to a real hardware implementation. Since most off-the-shelf C-ToF sensors use sinusoid or square functions, we develop a hardware prototype that can implement a wide range of coding functions. Using this prototype and our software simulator, we demonstrate the performance advantages of the proposed Hamiltonian coding functions in a wide range of imaging settings.
Supplemental Material
Available for Download
Supplemental movie, appendix, image and software files for, What Are Optimal Coding Functions for Time-of-Flight Imaging?
- Amit Adam, Christoph Dann, Omer Yair, Shai Mazor, and Sebastian Nowozin. 2016. Bayesian time-of-flight for realtime shape, illumination and Albedo. IEEE PAMI 38, 5 (2016), 851--864. Google Scholar
Digital Library
- B. Buxbaum, R. Schwarte, T. Ringbeck, M. Grothof, and X. Luan. 2002. MSM-PMD as correlation receiver in a new 3D-ranging system. Proc. SPIE 4546 (2002).Google Scholar
- D. A. Carnegie, J. R. K. McClymont, A. P. P. Jongenelen, A. A. Dorrington B. Drayto and, and A. D. Payne. 2011. Design and construction of a configurable full-field range imaging system for mobile robotic applications. Lecture Notes in Electrical Engineering 83 (2011).Google Scholar
- David Droeschel, Dirk Holz, and Sven Behnke. 2010. Multi-frequency phase unwrapping for time-of-flight cameras. In Proc. IROS.Google ScholarCross Ref
- R. Ferriere, J. Cussey, and J. Dudley. 2008. Time-of-flight range detection using low frequency intensity modulation of a CW laser diode: Application to fiber length measurement. Optical Engineering 47, 9 (2008), 93602--1 to 93602--6.Google ScholarCross Ref
- D. Freedman, E. Krupka, Y. Smolin, I. Leichter, and M. Schmidt. 2014. SRA: Fast removal of general multipath for ToF sensors. In Proc. ECCV.Google Scholar
- J. P. Godbaz, A. A. Dorrington, and M. J. Cree. 2013. Understanding and ameliorating mixed pixels and multipath interference in AMCW lidar. In TOF Range-Imaging Cameras, Fabio Remondino and D. Stoppa (Eds.). Springer Berlin.Google Scholar
- B. S. Goldstein and G. F. Dalrymple. 1967. Gallium arsenide injection laser radar. Proc. of the IEEE 55, 2 (1967).Google ScholarCross Ref
- C. Gotsman and M. Lindenbaum. 1996. On the metric properties of discrete space-filling curves. IEEE TIP 5, 5 (1996), 794--797. Google ScholarDigital Library
- Frank Gray. 1953. Pulse code communication. US Patent 2,632,058 (1953).Google Scholar
- R. Grootjans, W. Van der Tempel, D. Van Nieuwenhovec, C. de Tandt, and M. Kuijk. 2006. Improved modulation techniques for time-of-flight ranging cameras using pseudo random binary sequences. In Proc. IEEE LEOS Benelux Chapter.Google Scholar
- M. Gupta, S. K. Nayar, M. Hullin, and J. Martin. 2015. Phasor imaging: A generalization of correlation-based time-of-flight imaging. ACM Trans. Graphics 34, 5 (2015), 156:1--156:18. Google ScholarDigital Library
- S. W. Hasinoff, F. Durand, and W. T. Freeman. 2010. Noise-optimal capture for high dynamic range photography. In Proc. IEEE CVPR.Google Scholar
- F. Heide, M. B. Hullin, J. Gregson, and W. Heidrich. 2013. Low-budget transient imaging using photonic mixer devices. ACM SIGGRAPH 32, 4 (2013), 45:1--45:10. Google ScholarDigital Library
- E. Horn and N. Kiryati. 1997. Toward optimal structured light patterns. In International Conference on Recent Advances in 3D Digital Imaging and Modeling. Google ScholarDigital Library
- S. Inokuchi, K. Sato, and F. Matsuda. 1984. Range imaging system for 3-D object recognition. In Proc. IEEE ICPR.Google Scholar
- A. P. P. Jongenelen, D. G. Bailey, A. D. Payne, A. A. Dorrington, and D. A. Carnegie. 2011. Analysis of errors in ToF range imaging with dual-frequency modulation. IEEE Transactions on Instrumentation and Measurement 60, 5 (2011), 1861--1868.Google ScholarCross Ref
- A. Kadambi, R. Whyte, A. Bhandari, L. Streeter, C. Barsi, A. Dorrington, and R. Raskar. 2013. Coded ToF cameras: Sparse deconvolution to address multipath interference and recover time profiles. ACM SIGGRAPH Asia 32, 6 (2013), 167:1--167:10. Google ScholarDigital Library
- M. Kawakita, K. Iizuka, R. Iwama, K. Takizawa, H. Kikuchi, and F. Sato. 2004. Gain-modulated Axi-vision camera. Opt. Express 12, 22 (2004), 5336--5344.Google Scholar
- Maik Keller and Andreas Kolb. 2009. Real-time simulation of time-of-flight sensors. Simulation Modelling Practice and Theory 17, 5 (2009), 967--978.Google Scholar
- W. Koechner. 1968. Optical ranging system employing a high power injection laser diode. IEEE Trans. AES 4, 1 (1968), 81--91.Google ScholarCross Ref
- A. Kolb, E. Barth, R. Koch, and R. Larsen. 2010. Time-of-flight cameras in computer graphics. Eurographics (2010).Google Scholar
- R. Lange. 2000. 3D ToF distance measurement with custom solid-state image sensors in CMOS-CCD-technology. Ph.D. Thesis (2000).Google Scholar
- R. Lange and P. Seitz. 2001. Solid state time-of-flight range camera. IEEE J. Quantum Electronics 37, 3 (2001), 390--397.Google ScholarCross Ref
- Robert Lange, Peter Seitz, Alice Biber, and Stefan C. Lauxtermann. 2000. Demodulation pixels in CCD and CMOS technologies for time-of-flight ranging. Proc. SPIE 3965 (2000).Google Scholar
- Stephan Meister, Rahul Nair, and Daniel Kondermann. 2013. Simulation of time-of-flight sensors using global illumination. In Vision, Modeling and Visualization, Michael Bronstein, Jean Favre, and Kai Hormann (Eds.). The Eurographics Association.Google Scholar
- Microsoft-Kinect. 2014. March NPD Results: Titanfall on Xbox One is Number One Selling Game. Retrieved from http://news.xbox.com/2014/04/xbox-one-march-npd.Google Scholar
- W. H. Mills. 1963. Some complete cycles on the n-cube. Proc. Amer. Math. Soc. 14, 4 (1963), 640--643.Google Scholar
- M. O’Toole, S. Achar, S. G. Narasimhan, and K. N. Kutulakos. 2015. Homogeneous codes for energy-efficient illumination and imaging. ACM SIGGRAPH 34, 4 (2015), 35:1--35:13. Google ScholarDigital Library
- M. O’Toole, F. Heide, L. Xiao, M. B. Hullin, W. Heidrich, and K. N. Kutulakos. 2014. Temporal frequency probing for 5D transient analysis of global light transport. ACM SIGGRAPH 33, 4 (2014), 87:1--87:11. Google ScholarDigital Library
- A. D. Payne, A. A. Dorrington, and M. J. Cree. 2010. Illumination waveform optimization for time-of-flight range imaging cameras. In Proc. SPIE 8085.Google Scholar
- J. M. Payne. 1973. An optical distance measuring instrument. Review of Scientific Instruments 44, 3 (1973).Google Scholar
- H. Sagan. 1994. Space Filling Curves. Springer, New York.Google Scholar
- Mirko Schmidt and Bernd Jähne. 2009. A Physical Model of Time-of-Flight 3D Imaging Systems, Including Suppression of Ambient Light. Springer Berlin, Berlin, Germany, 1--15.Google Scholar
- R. Schwarte, Z. Xu, H. Heinol, J. Olk, R. Klein, B. Buxbaum, H. Fischer, and J. Schulte. 1997. New electro-optical mixing and correlating sensor: Facilities and applications of the photonic mixer device. In Proc. SPIE (3100).Google Scholar
- O. Shcherbakova, L. Pancheri, G.-F. Dalla Betta, N. Massari, and D. Stoppa. 2013. 3D camera based on linear-mode gain-modulated photodiodes. In Proc. IEEE ISSCC.Google Scholar
- Shikhar Shrestha, Felix Heide, Wolfgang Heidrich, and Gordon Wetzstein. 2016. Computational imaging with multi-camera time-of-flight systems. ACM SIGGRAPH 35, 4 (2016), 33:1--33:11. Google ScholarDigital Library
- E. Tadmor, I. Bakish, S. Felzenshtein, E. Larry, G. Yahav, and D. Cohen. 2014. A fast global shutter image sensor based on the VOD mechanism. In IEEE SENSORS. 618--621.Google Scholar
- Texas-Instruments. 2017. OPT8241 3D Time-of-Flight Sensor. Retrieved from http://www.ti.com/lit/ds/sbas704b/sbas704b.pdf.Google Scholar
- A. Torralba and A. Oliva. 2003. Statistics of natural image categories. Network 14, 3 (2003), 391--412.Google ScholarCross Ref
- John F. Wakerly. 2005. Digital Design: Principles and Practices. Google ScholarDigital Library
Index Terms
- What Are Optimal Coding Functions for Time-of-Flight Imaging?
Recommendations
Doppler time-of-flight imaging
Over the last few years, depth cameras have become increasingly popular for a range of applications, including human-computer interaction and gaming, augmented reality, machine vision, and medical imaging. Many of the commercially-available devices use ...
Epipolar time-of-flight imaging
Consumer time-of-flight depth cameras like Kinect and PMD are cheap, compact and produce video-rate depth maps in short-range applications. In this paper we apply energy-efficient epipolar imaging to the ToF domain to significantly expand the ...
Differential Frequency Heterodyne Time-of-Flight Imaging for Instantaneous Depth and Velocity Estimation
In this study, we discuss the imaging of depth and velocity using heterodyne-mode time-of-flight (ToF) cameras. In particular, Doppler ToF (D-ToF) imaging utilizes heterodyne modulation to measure the velocity from the Doppler frequency shift, which ...
Comments