Thomas Hach
ABSTRACT
The combination of 2D high-resolution RGB image sensors with 3D Time-of-Flight imagers has become an increasing field of research in the past ten years. This paper describes the next step of sensor fusion by providing flawlessly matching RGB and Z images. Our novel RGB-Z camera is based on a high-quality 2.8k motion picture camera which is optically extended to provide the same perspective for a depth camera by means of a rotating mirror principle. In the scope of this paper technical details are explained and a common interface is proposed for integrating RGB-Z data in the professional post-production workflow. Thereby, this RGB-Z camera addresses typical problems in the movie production chain where depth information is essential. Finally, the Z image quality is evaluated using a cine-style full format lens.
- S. Kirchhof. Stereoskopische Langfilmproduktion. Number 5, pages 257--259. Schiele & Schöön, Berlin, 2013.Google Scholar
- A. Frick, B. Bartczack, and R. Koch. 3D-TV LDV content generation with a hybrid ToF-multicamera RIG. In 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video, pages 1--4, 2010.Google Scholar
- J. Choi and D. Min and K. Sohn. 2D-Plus-Depth Based Resolution and Frame-rate Up-conversion Technique for Depth Video. In IEEE Transactions on Consumer Electronics, volume 56, pages 2489--2497, 2010. Google ScholarDigital Library
- M. Lindner, A. Kolb, and K. Hartmann. Data-Fusion of PMD-Based Distance-Information and High-Resolution RGB-Images. In International Symposium on Signals, Circuits and Systems, pages 1--4, 2007.Google Scholar
- J. Smisek, M. Jancosek, and T. Pajdla. 3D with Kinect. In IEEE International Conference on Computer Vision Workshops, pages 1154--1160, 2011.Google ScholarCross Ref
- V. Gandhi, J. Cech, and R. Horaud. High-Resolution Depth Maps Based on TOF-Stereo Fusion. In IEEE International Conference on Robotics and Automation, pages 4742--4749, RiverCentre, Saint Paul, Minnesota, USA, 2012.Google Scholar
- J. Zhu, L. Wang, R. Yang, J. E. Davis, and Z. Pan. Reliability fusion of time-of-flight depth and stereo geometry for high quality depth maps. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 33, pages 1400--1414. 2011. Google ScholarDigital Library
- B. Boldt, S. Budge, R. Pack, and P. Israelson. A Handheld Texel Camera for Acquiring Near-Instantaneous 3D Images. In Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers, pages 953--957, 2007.Google ScholarCross Ref
- T. D. A. Prasad, K. Hartmann, W. Weihs, S. E. Ghobadi, and A. Sluiter. First Steps in Enhancing 3D Vision Technique Using 2D/3D Sensors. In Computer Vision Winter Workshop. Czech Pattern Recognition Society, 2006.Google Scholar
- P. Centen. Time of flight, the 4th dimension in imaging. In European Conference on Visual Media Production, 2011.Google Scholar
- C. Bamji and P. Zhao. United States Patent No 8,139,141 B2: Single Chip Red, Green, Blue, Distance (RGB-Z) Sensor. Microsoft Corporation, published on 20 March 2012.Google Scholar
- ARRI. "ARRI ALEXA Studio Camera Datasheet". Arnold & Richter Cine Technik GmbH & Co. Betriebs KG, 2012.Google Scholar
- D. Chan, H. Buisman, C. Theobalt, and S. Thrun. A noise-aware filter for real-time depth upsampling. In Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications, 2008.Google Scholar
- J. Park, H. Kim, Y.-W. Tai, M. S. Brown, and I. Kweon. High quality depth map upsampling for 3D-TOF cameras. In Computer Vision, 2011 IEEE International Conference on, pages 1623--1630, 2011. Google ScholarDigital Library
- M. Kiechle, S. Hawe, and M. Kleinsteuber. A joint intensity and depth co-sparse analysis model for depth map super-resolution. arXiv e-print 1304.5319, April 2013.Google Scholar
- R. Bogart F. Kainz. Technical Introduction to OpenEXR. Industrial Light & Magic, 2013.Google Scholar
- H. G. Heinol. Untersuchung und entwicklung von modulationslaufzeitbasierten 3D-Sichtsystemen. 2006.Google Scholar
- R. Lange and P. Seitz. Solid-state time-of-flight range camera. Quantum Electronics, IEEE Journal of, 37(3): 390--397, 2001.Google ScholarCross Ref
- W. Nolting. Grundkurs Theoretische Physik 3, Elektrodynamik, volume 3, chapter "Ebene Wellen", page 261 et seq. Springer-Verlag, Berlin, 9 edition, 2011.Google Scholar
- J.-Y. Bouguet. Camera Calibration Toolbox for Matlab. California Institute of Technology, 2010.Google Scholar
- S. Foix, G. Aleny, and C. Torras. Lock-in Time-of-Flight (ToF) Cameras: A Survey. IEEE Sensors Journal, 11(9): 1917--1926, 2011.Google ScholarCross Ref
Index Terms
- A novel RGB-Z camera for high-quality motion picture applications
Recommendations
Cinematic Bokeh rendering for real scenes
CVMP '15: Proceedings of the 12th European Conference on Visual Media ProductionIn this work, we present a novel approach for rendering high-quality Bokeh and depth-of-field for real, non-computer graphics scenes by synthesizing a real lens and involving depth. Although this discipline is well-known to methods in computer graphics, ...
Live RGB-D camera tracking for television production studios
Highlights A novel low-cost tool for camera tracking in broadcasting studio environments. Driftless tracking with keyframes. Real-time performance using a GPU. Allows moving actors in the scene while tracking. Comparison with Kinfu. In this work, a real-...
A Compact Tri-Modal Camera Unit for RGBDT Vision
ICMVA '22: Proceedings of the 2022 5th International Conference on Machine Vision and ApplicationsThe combination of RGBD and thermal cameras in multi-modal person-centric vision applications has great potential. As a complementary modality, thermal cameras can compensate for weaknesses such as the inability to operate in absolute darkness of ...
Comments