Pingping Cai
Skip Abstract Section
Abstract
3D Point Cloud Data (PCD) has been used in many research and commercial applications widely, such as autonomous driving, robotics, and VR/AR. But existing PCD generation systems based on RGB-D and LiDARs require robust lighting and an unobstructed field of view of the target scenes. So, they may not work properly under challenging environmental conditions. Recently, millimeter-wave (mmWave) based imaging systems have raised considerable interest due to their ability to work in dark environments. But the resolution and quality of the PCD from these mmWave imaging systems are very poor. To improve the quality of PCD, we design and implement MilliPCD, a "beyond traditional vision" PCD generation system for handheld mmWave devices, by integrating traditional signal processing with advanced deep learning based algorithms. We evaluate MilliPCD with real mmWave reflected signals collected from large, diverse indoor environments, and the results show improvements in the quality w.r.t. the existing algorithms, both quantitatively and qualitatively.
- Radu Bogdan Rusu and Zoltan Csaba Marton and Nico Blodow and Mihai Dolha and Michael Beetz, "Towards 3D Point cloud based object maps for household environments," Robotics and Autonomous Systems, vol. 56, no. 11, 2008.Google Scholar
- Yaodong Cui and Ren Chen and Wenbo Chu and Long Chen and Daxin Tian and Ying Li and Dongpu Cao, "Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review," IEEE Transactions on Intelligent Transportation Systems, pp. 1--18, 2021.Google ScholarDigital Library
- Fangping Chen and Yuheng Lu and Binbin Cai and Xiaodong Xie, "Multi-Drone Collaborative Trajectory Optimization for Large-Scale Aerial 3D Scanning," in IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), 2021.Google Scholar
- Alessio Pierluigi Placitelli and Luigi Gallo, "Low-Cost Augmented Reality Systems via 3D Point Cloud Sensors," in International Conference on Signal Image Technology Internet-Based Systems, 2011.Google Scholar
- Craig Gotsman and Xianfeng Gu and Alla Sheffer, "Fundamentals of Spherical Parameterization for 3D Meshes," in ACM SIGGRAPH, 2003.Google Scholar
- Zhirong Wu and Shuran Song and Aditya Khosla and Fisher Yu and Linguang Zhang and Xiaoou Tang and Jianxiong Xiao, "3d shapenets: A deep representation for volumetric shapes," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2015.Google Scholar
- Lars Linsen, "Point Cloud Representation," Karlsruhe Institute of Technology, Germany, Tech. Rep., 2001.Google Scholar
- Mohamed Abdelaal and Daniel Reichelt and Frank Dürr and Kurt Rothermel and Lavinia Runceanu and Susanne Becker and Dieter Fritsch, "ComNSense: Grammar-Driven Crowd-Sourcing of Point Clouds for Automatic Indoor Mapping," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 2, no. 1, 2018.Google Scholar
- Shaoshuai Shi and Xiaogang Wang and Hongsheng Li, "PointRCNN: 3D Object Proposal Generation and Detection From Point Cloud," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.Google Scholar
- Chris Xiaoxuan Lu and Stefano Rosa and Peijun Zhao and Bing Wang and Changhao Chen and John A. Stankovic and Niki Trigoni and Andrew Markham, "See through Smoke: Robust Indoor Mapping with Low-Cost MmWave Radar," in Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services, 2020.Google Scholar
- Yue Sun and Honggang Zhang and Zhuoming Huang and Benyuan Liu, "DeepPoint: A Deep Learning Model for 3D Reconstruction in Point Clouds via mmWave Radar," 2021. [Online]. Available: https://arxiv.org/abs/2109.09188Google Scholar
- Kun Qian and Zhaoyuan He and Xinyu Zhang, "3D Point Cloud Generation with Millimeter-Wave Radar," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 4, no. 4, 2020.Google Scholar
- Mingmin Zhao and Tianhong Li and Mohammad Abu Alsheikh and Yonglong Tian and Hang Zhao and Antonio Torralba and Dina Katabi, "Through-Wall Human Pose Estimation Using Radio Signals," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.Google Scholar
- Hem Regmi and Moh Sabbir Saadat and Sanjib Sur and Srihari Nelakuditi, "SquiggleMilli: Approximating SAR Imaging on Mobile Millimeter-Wave Devices," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 5, no. 3, 2021.Google Scholar
- Guangcheng Zhang and Xiaoyi Geng and Yueh-Jaw Lin, "Comprehensive mPoint: A Method for 3D Point Cloud Generation of Human Bodies Utilizing FMCW MIMO mm-Wave Radar," Sensors, vol. 21, no. 19, 2021.Google Scholar
- TI, "Point cloud visualization for robotics using TI's mmWave sensor," 2017. [Online]. Available: https://training.ti.com/point-cloud-visualization-robotics-using-tis-mmwave-sensorGoogle Scholar
- Yue Sun and Zhuoming Huang and Honggang Zhang and Zhi Cao and Deqiang Xu, "3DRIMR: 3D Reconstruction and Imaging via mmWave Radar based on Deep Learning," in IEEE International Performance, Computing, and Communications Conference (IPCCC), 2021.Google Scholar
- Junfeng Guan and Sohrab Madani and Suraj Jog and Saurabh Gupta and Haitham Hassanieh, "Through Fog High-Resolution Imaging Using Millimeter Wave Radar," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.Google Scholar
- Alexander Ganis and Enric Miralles Navarro and Bernhard Schoenlinner and Ulrich Prechtel and Askold Meusling and Christoph Heller and Thomas Spreng and Jan Mietzner and Christian Krimmer and Babette Haeberle and Steffen Lutz and Mirko Loghi and Angel Belenguer and Hector Esteban and Volker Ziegler, "A Portable 3-D Imaging FMCW MIMO Radar Demonstrator With a 24 x 24 Antenna Array for Medium-Range Applications," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 1, 2018.Google ScholarCross Ref
- Mehrdad Soumekh, Synthetic Aperture Radar Signal Processing. John Wiley & Sons, Inc., 1999.Google Scholar
- Weiyan Chen and Fusang Zhang and Tao Gu and Kexing Zhou and Zixuan Huo and Daqing Zhang, "Constructing Floor Plan through Smoke Using Ultra Wideband Radar," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 5, no. 4, 2022.Google Scholar
- Ian Goodfellow and Jean Pouget-Abadie and Mehdi Mirza and Bing Xu and David Warde-Farley and Sherjil Ozair and Aaron Courville and Yoshua Bengio, "Generative Adversarial Nets," in Advances in Neural Information Processing Systems (NIPS), 2014.Google Scholar
- Evan C. Zaugg and David G. Long, "Generalized Frequency Scaling and Backprojection for LFM-CW SAR Processing," IEEE Transactions on Geoscience and Remote Sensing, vol. 53, no. 7, 2015.Google ScholarCross Ref
- Yue Wang and Yongbin Sun and Ziwei Liu and Sanjay E. Sarma and Michael M. Bronstein and Justin M. Solomon, "Dynamic graph cnn for learning on point clouds," ACM Transactions on Graphics (TOG), 2019.Google ScholarDigital Library
- Charles R. Qi and Li Yi and Hao Su and Leonidas J. Guibas, "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space," in Advances in Neural Information Processing Systems (NIPS), 2017.Google Scholar
- Peng Xiang and Xin Wen and Yu-Shen Liu and Yan-Pei Cao and Pengfei Wan and Wen Zheng and Zhizhong Han, "SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformer," in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2021.Google Scholar
- Texas Instruments, "IWR6843 Single-Chip 60-GHz MmWave Sensor Evaluation Module," 2020. [Online]. Available: https://www.ti.com/tool/IWR6843ISKGoogle Scholar
- AsusTek Computer Inc., "Zenfone AR: Go Beyond Reality," 2021. [Online]. Available: https://www.asus.com/us/Phone/ZenFone-AR-ZS571KL/Google Scholar
- Haoqiang Fan and Hao Su and Leonidas Guibas, "A Point Set Generation Network for 3D Object Reconstruction from a Single Image," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2017.Google Scholar
- Wentao Yuan and Tejas Khot and David Held and Christoph Mertz and Martial Hebert, "PCN: Point Completion Network," in International Conference on 3D Vision (3DV), 2018.Google Scholar
- Sanjib Sur, "MilliPCD Project," 2022. [Online]. Available: https://syrex.cse.sc.edu/research/ubiquitous-sensing/millipcd/Google Scholar
- David M. Sheen and Douglas L. McMakin and Thomas E. Hall, "Three-Dimensional Millimeter-Wave Imaging for Concealed Weapon Detection," IEEE Transactions on Microwave Theory and Techniques, vol. 49, no. 9, 2001.Google ScholarCross Ref
- Muhammet Emin Yanik and Murat Torlak, "Near-Field MIMO-SAR Millimeter-Wave Imaging With Sparsely Sampled Aperture Data," IEEE Access, vol. 7, 2019.Google Scholar
- Mehrdad Soumekh, "A System Model and Inversion for Synthetic Aperture Radar Imaging," IEEE Transactions on Image Processing, vol. 1, no. 1, 1992.Google ScholarDigital Library
- Babak Mamandipoor and G. Malysa and A. Arbabian and U. Madhow and K. Noujeim, "60 GHz Synthetic Aperture Radar for Short-Range Imaging: Theory and Experiments," in IEEE Asilomar Conference on Signals, Systems and Computers, 2014.Google Scholar
- Yanzi Zhu and Yibo Zhu and Ben Y. Zhao and Haitao Zheng, "Reusing 60GHz Radios for Mobile Radar Imaging," in ACM MobiCom, 2015.Google Scholar
- Claire M. Watts and Patrick Lancaster and Andreas Pedross-Engel and Joshua R. Smith and Matthew S. Reynolds, "2D and 3D Millimeter-Wave Synthetic Aperture Radar Imaging on a PR2 Platform," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.Google Scholar
- Junfeng Guan and Arun Paidimarri and Alberto Valdes-Garcia and Bodhisatwa Sadhu, "3D Imaging using mmWave 5G Signals," in IEEE Radio Frequency Integrated Circuits Symposium (RFIC), 2020.Google Scholar
- Carlos Baquero Barneto and Taneli Riihonen and Matias Turunen and Mike Koivisto and Jukka Talvitie and Mikko Valkama, "Radio-based sensing and indoor mapping with millimeter-wave 5g nr signals," in International Conference on Localization and GNSS (ICL-GNSS), 2020.Google Scholar
- Xuyou Li and Shitong Du and Guangchun Li and Haoyu Li, "Integrate Point-Cloud Segmentation with 3D LiDAR Scan-Matching for Mobile Robot Localization and Mapping," MDPI Sensors, vol. 20, no. 1, 2020.Google Scholar
- Yiming Zeng and Yu Hu and Shice Liu and Jing Ye and Yinhe Han and Xiaowei Li and Ninghui Sun, "RT3D: Real-Time 3-D Vehicle Detection in LiDAR Point Cloud for Autonomous Driving," IEEE Robotics and Automation Letters, vol. 3, no. 4, 2018.Google ScholarCross Ref
- Mathieu Labbe and Francois Michaud, "RTAB-Map as an Open-Source Lidar and Visual Simultaneous Localization and Mapping Library for Large-Scale and Long-Term Online Operation," Journal of Field Robotics, vol. 36, no. 2, 2019.Google ScholarCross Ref
- Kun Qian and Xudong Ma and Fang Fang and Hong Yang, "3D environmental mapping of mobile robot using a low-cost depth camera," in IEEE International Conference on Mechatronics and Automation, 2013.Google Scholar
- Bishwajit Pal and Samitha Khaiyum and Y. S. Kumaraswamy, "3D point cloud generation from 2D depth camera images using successive triangulation," in International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), 2017.Google Scholar
- Vincent Fremont and Ryad Chellali, "Turntable-based 3D object reconstruction," in IEEE Conference on Cybernetics and Intelligent Systems, 2004.Google Scholar
- Darius Burschka and Ming Li and Russell Taylor and Gregory D. Hager, "Scale-invariant registration of monocular stereo images to 3D surface models," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2004.Google Scholar
- Steven M. Seitz and Brian Curless and James Diebel and Daniel Scharstein and Richard Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2006.Google Scholar
- Wei Zeng and Sezer Karaoglu and Theo Gevers, "Inferring Point Clouds from Single Monocular Images by Depth Intermediation," 2018. [Online]. Available: https://arxiv.org/abs/1812.01402Google Scholar
- K L Navaneet and Ansu Mathew and Shashank Kashyap and Wei-Chih Hung and Varun Jampani and R. Venkatesh Babu, "From Image Collections to Point Clouds with Self-supervised Shape and Pose Networks," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.Google Scholar
- Adriano Meta and Peter Hoogeboom and Leo P. Ligthart, "Signal Processing for FMCW SAR," IEEE Transactions on Geoscience and Remote Sensing, vol. 45, no. 11, 2007.Google ScholarCross Ref
- Google, "Project Soli," 2022. [Online]. Available: https://www.google.com/atap/project-soli/Google Scholar
- Jihoon Bang and Youngtaek Hong and Jaehoon Choi, "MM-wave phased array antenna for whole-metal-covered 5G mobile phone applications," in International Symposium on Antennas and Propagation (ISAP), 2017.Google Scholar
- Naser Ojaroudi Parchin and Ming Shen and Gert Fralund Pedersen, "UWB MM-Wave antenna array with quasi omnidirectional beams for 5G handheld devices," in IEEE International Conference on Ubiquitous Wireless Broadband (ICUWB), 2016.Google Scholar
- Stephan Palm and Rainer Sommer and Uwe Stilla, "Mobile Radar Mapping---Subcentimeter SAR Imaging of Roads," IEEE Transactions on Geoscience and Remote Sensing, vol. 56, no. 11, 2018.Google ScholarCross Ref
- Zheng Fang and Sifan Zhou and Yubo Cui and Sebastian Scherer, "3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud," IEEE Sensors Journal, vol. 21, no. 4, 2021.Google ScholarCross Ref
- James Diebel, "Representing attitude: Euler angles, unit quaternions, and rotation vectors," Matrix, vol. 58, no. 15--16, 2006.Google Scholar
- Shivanand Venkanna Sheshappanavar and Chandra Kambhamettu, "A Novel Local Geometry Capture in PointNet++ for 3D Classification," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020.Google Scholar
- Yanchao Lian and Tuo Feng and Jinliu Zhou, "A Dense Pointnet++ Architecture for 3D Point Cloud Semantic Segmentation," in IEEE International Geoscience and Remote Sensing Symposium, 2019.Google Scholar
- Yaoqing Yang and Chen Feng and Yiru Shen and Dong Tian, "FoldingNet: Point Cloud Auto-Encoder via Deep Grid Deformation," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.Google Scholar
- Yutaka Ohtake and Alexander Belyaev and Hans-Peter Seidel, "A multi-scale approach to 3D scattered data interpolation with compactly supported basis functions," in Shape Modeling International, 2003.Google Scholar
- Hui Huang and Shihao Wu and Minglun Gong and Daniel Cohen-Or and Uri Ascher and Hao (Richard) Zhang, "Edge-Aware Point Set Resampling," ACM Transactions on Graphics (TOG), vol. 32, no. 1, 2013.Google ScholarDigital Library
- Prudhvi Gurram and Shuowen Hu and Alex Chan, "Uniform grid upsampling of 3D lidar point cloud data," in Three-Dimensional Image Processing (3DIP) and Applications, 2013.Google Scholar
- Lequan Yu and Xianzhi Li and Chi-Wing Fu and Daniel Cohen-Or and Pheng-Ann Heng, "PU-Net: Point Cloud Upsampling Network," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.Google Scholar
- Lyne P. Tchapmi and Vineet Kosaraju and Hamid Rezatofighi and Ian Reid and Silvio Savarese, "TopNet: Structural Point Cloud Decoder," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019.Google Scholar
- Paolo Cignoni and Marco Callieri and Massimiliano Corsini and Matteo Dellepiane and Fabio Ganovelli and Guido Ranzuglia, "MeshLab: an Open-Source Mesh Processing Tool," in Eurographics Italian Chapter Conference, 2008.Google Scholar
- Open-Source, "PyTroch," 2021. [Online]. Available: https://pytorch.org/Google Scholar
- NVIDIA, "RTX A6000," 2021. [Online]. Available: https://www.nvidia.com/en-us/design-visualization/rtx-a6000/Google Scholar
- Angela Dai and Daniel Ritchie and Martin Bokeloh and Scott Reed and Jurgen Sturm and Matthias Niebner, "ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scans," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018.Google Scholar
- Pingping Cai and Sanjib Sur, "DeepPCD: Enabling AutoCompletion of Indoor Point Clouds with Deep Learning," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., vol. 6, no. 2, 2022.Google Scholar
- Ian C. McDowell and Rahul Bulusu and Sanjib Sur, "Poster: MilliDrone: A Drone Platform to Facilitate Scalable Survey of Outdoor Millimeter-Wave Signal Propagation," in Proceedings of ACM International Workshop on Mobile Computing Systems and Applications (HotMobile), 2022.Google Scholar
Index Terms
- MilliPCD: Beyond Traditional Vision Indoor Point Cloud Generation via Handheld Millimeter-Wave Devices
Recommendations
Compact Patch Antenna Array for 60 GHz Millimeter-Wave Broadband Applications
AbstractIn this study, a wide-band compact patch antenna array is developed for 60 GHz band applications. The antenna array consists of eight identical elements. Each element is a simple microstrip-fed rectangular radiating patch printed on the top side ...
Efficient Frequency Reconfigurable Antenna for 4G, Sub-6 GHz, 5G Portable Devices Applications
AbstractThis research investigates a frequency-switchable, multi-mode operation antenna contingent on ground structure slots and suitable for 4G, 5G/sub 6 GHz and other modern systems. In earlier research work, researchers used the approach of putting ...
High gain substrate integrated waveguide antenna with enhanced bandwidth for millimeter-wave wireless network applications
AbstractA high-gain slotted waveguide antenna in substrate-integrated waveguide (SIW) configuration with enhanced bandwidth is proposed. Two different cases fabricated using Rogers RO 4003 substrate over a small area of 16 × 4.5 × 0.508 mm3 are discussed. ...
Comments