Rong Ding
ABSTRACT
This paper presents EmoRI, a novel single-chip mmWave radar-based egomotion estimation approach that works in challenging scenarios where moving objects exist in radar's Field of View (FoV). Essentially, estimating a mobile platform's egomotion using an on-board mmWave radar requires inferring the relative motion between radar and the points of the stationary objects (PSOs) in the radar point cloud. However, in practice, there could be no PSOs because of the blockage of moving objects. Even if PSOs exist, precisely identifying them is still challenging due to (i) the large quantity of points generated by the moving objects, and (ii) the huge angle estimation errors of the conventional point cloud generation algorithm. We empower EmoRI to overcome the above challenges incurred by moving objects with three core techniques, which include (i) a hybrid FFT-MUSIC algorithm that improves the angle estimation accuracy of single-chip mmWave radar, (ii) a multiple stationary target consensus algorithm that precisely selects the PSOs from the radar point cloud, and (iii) a simultaneous fusion and calibration mechanism that introduces an IMU as the auxiliary sensor, meticulously calibrates IMU accelerations with radar measurements, and complimentarily fuses these two modalities to obtain the 6-DoF egomotion. Our extensive experiments validate that EmoRI pushes the limit of single-chip mmWave radar-based egomotion estimation with moving objects in radar's FoV by reducing the per-meter destination error from decimeter to centimeter level.
- M. Ouyang, X. Shi, Y. Wang, Y. Tian, Y. Shen, D. Wang, P. Wang, and Z. Cao, "A collaborative visual slam framework for service robots," in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2021.Google Scholar
- Elektrobit, "Vehicle position and ego-motion estimation for automotive applications," https://www.elektrobit.com/tech-corner/vehicle-position-and-ego-motion-estimation-for-automotive-applications/.Google Scholar
- "Ar-hud," https://www.fic.com.tw/automotive/ar-hud/.Google Scholar
- "Ar-hud: A heads up on the road ahead," https://blog.huawei.com/2022/01/04/ar-hud-road-ahead-transportation/.Google Scholar
- MBUX Augmented Reality Head-up Display, "Mercedodes-benz," https://www.mercedes-benz.com.hk/en/passengercars/mercedes-benzcars/models/eqs/saloon-v297/equipment.pi.html/mercedes-benz-cars/models/eqs/saloon-v297/equipment/individualization/mbux-ar-hud.Google Scholar
- G. Chi, Z. Yang, J. Xu, C. Wu, J. Zhang, J. Liang, and Y. Liu, "Wi-drone: wi-fi-based 6-dof tracking for indoor drone flight control," in The 20th Annual International Conference on Mobile Systems, Applications and Services (Mobisys), 2022.Google Scholar
- M. Oelsch, M. Karimi, and E. Steinbach, "R-loam: Improving lidar odometry and mapping with point-to-mesh features of a known 3d reference object," IEEE Robotics and Automation Letters, vol. 6, no. 2, pp. 2068--2075, 2021.Google ScholarCross Ref
- J. Zhang and S. Singh, "LOAM: lidar odometry and mapping in real-time," in Robotics: Science and Systems X, 2014.Google Scholar
- J. Lin and F. Zhang, "Loam livox: A fast, robust, high-precision lidar odometry and mapping package for lidars of small fov," in IEEE International Conference on Robotics and Automation, ICRA, 2020.Google Scholar
- A. Tagliabue, J. Tordesillas, X. Cai, A. Santamaria-Navarro, J. P. How, L. Carlone, and A.-a. Agha-mohammadi, "Lion: Lidar-inertial observability-aware navigator for vision-denied environments," in Experimental Robotics: The 17th International Symposium. Springer, 2021, pp. 380--390.Google Scholar
- W. Xu, Y. Cai, D. He, J. Lin, and F. Zhang, "Fast-lio2: Fast direct lidar-inertial odometry," IEEE Transactions on Robotics, vol. 38, no. 4, pp. 2053--2073, 2022.Google ScholarCross Ref
- Livox, "Livox hap," https://www.livoxtech.com/hap.Google Scholar
- A. J. Davison, I. D. Reid, N. Molton, and O. Stasse, "Monoslam: Real-time single camera SLAM," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1052--1067, 2007.Google ScholarDigital Library
- C. Wang, C. E. Thorpe, S. Thrun, M. Hebert, and H. F. Durrant-Whyte, "Simultaneous localization, mapping and moving object tracking," Int. J. Robotics Res., vol. 26, no. 9, pp. 889--916, 2007.Google ScholarDigital Library
- R. Clark, S. Wang, H. Wen, A. Markham, and N. Trigoni, "Vinet: Visual-inertial odometry as a sequence-to-sequence learning problem," in Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, S. Singh and S. Markovitch, Eds., 2017.Google Scholar
- T. Qin, P. Li, and S. Shen, "Vins-mono: A robust and versatile monocular visual-inertial state estimator," IEEE Transactions on Robotics, vol. 34, no. 4, pp. 1004--1020, 2018.Google ScholarDigital Library
- C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza, "On-manifold preintegration theory for fast and accurate visual-inertial navigation," IEEE Transactions on Robotics, pp. 1--18, 2015.Google Scholar
- Texas Instruments Incorporated, "Single-chip 60-ghz to 64-ghz automotive radar sensor integrating antenna on package, dsp and mcu," https://www.ti.com/product/IWR6843AOP.Google Scholar
- Texas Instruments, "mmwave radar sensors in robotics applications," https://www.ti.com/lit/pdf/spry311.Google Scholar
- Junko Yoshida, "Audi a8: A look under the hood," https://www.eetindia.co.in/audi-a8-a-look-under-the-hood/.Google Scholar
- Autel, "Autel evo max 4t millimeter-wave radar obstacle avoidance," https://www.autelpilot.com/blogs/news/autel-max-4t-millimeter-wave-radar-obstacle-avoidance.Google Scholar
- D. Kellner, M. Barjenbruch, J. Klappstein, J. Dickmann, and K. C. J. Dietmayer, "Instantaneous ego-motion estimation using multiple doppler radars," in IEEE International Conference on Robotics and Automation, ICRA, 2014.Google Scholar
- R. Schmidt, "Multiple emitter location and signal parameter estimation," IEEE transactions on antennas and propagation, vol. 34, no. 3, pp. 276--280, 1986.Google Scholar
- P. Zhou, M. Li, and G. Shen, "Use it free: instantly knowing your phone attitude," in The 20th Annual International Conference on Mobile Computing and Networking (MobiCom), 2014.Google ScholarDigital Library
- C. X. Lu, M. R. U. Saputra, P. Zhao, Y. Almalioglu, P. P. B. de Gusmao, C. Chen, K. Sun, N. Trigoni, and A. Markham, "milliego: single-chip mmwave radar aided egomotion estimation via deep sensor fusion," in The 18th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2020.Google ScholarDigital Library
- Y. Almalioglu, M. Turan, C. X. Lu, N. Trigoni, and A. Markham, "Milli-rio: Egomotion estimation with low-cost millimetre-wave radar," IEEE Sensors Journal., vol. 21, no. 3.Google ScholarCross Ref
- A. Kramer, C. Stahoviak, A. Santamaria-Navarro, A. Agha-mohammadi, and C. Heckman, "Radar-inertial ego-velocity estimation for visually degraded environments," in IEEE International Conference on Robotics and Automation, ICRA, 2020.Google Scholar
- S. Palipana, D. Salami, L. A. Leiva, and S. Sigg, "Pantomime: Mid-air gesture recognition with sparse millimeter-wave radar point clouds," Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 1, pp. 27:1--27:27, 2021.Google ScholarDigital Library
- K. Bansal, K. Rungta, S. Zhu, and D. Bharadia, "Pointillism: accurate 3d bounding box estimation with multi-radars," in The 18th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2020.Google Scholar
- X. Shuai, Y. Shen, Y. Tang, S. Shi, L. Ji, and G. Xing, "millieye: A lightweight mmwave radar and camera fusion system for robust object detection," in International Conference on Internet-of-Things Design and Implementation (IoTDI), 2021.Google ScholarDigital Library
- M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381--395, 1981.Google ScholarDigital Library
- Texas Instruments Incorporated, "Single-chip 76-ghz to 81-ghz mmwave sensor integrating dsp and mcu," https://www.ti.com/product/IWR1642/.Google Scholar
- Texas Instruments Incorporated, "Single-chip 76-ghz to 81-ghz mmwave sensor integrating mcu and hardware accelerator," https://www.ti.com/product/IWR1443/.Google Scholar
- M. A. Richards, Fundamentals of radar signal processing. McGraw-Hill Education, 2014.Google Scholar
- M. Ester, H. Kriegel, J. Sander, and X. Xu, "A density-based algorithm for discovering clusters in large spatial databases with noise," in Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD), 1996.Google Scholar
- J. Diebel et al., "Representing attitude: Euler angles, unit quaternions, and rotation vectors," Matrix, vol. 58, no. 15--16, pp. 1--35, 2006.Google Scholar
- WheelTac, "Ros robot," https://wheeltec.jd.com/.Google Scholar
- Volkswagen, "Volkswagen passat sedan," https://www.volkswagen.com.au/en/models/passat.html.Google Scholar
- "Matrice 210 user manual," https://dl.djicdn.com/downloads/M200/20201120/M200_User_Manual_EN_20201120.pdf.Google Scholar
- Texas Instruments Incorporated, "Real-time data-capture adapter for radar sensing evaluation module," http://www.ti.com/tool/DCA1000EVM/.Google Scholar
- Livox, "Automotive-grade lidar for serial production," https://www.livoxtech.com/.Google Scholar
- Ouster, "Os1," https://ouster.com/products/scanning-lidar/os1-sensor/.Google Scholar
- B. Sensortec, "High-performance inertial measurement unit (bmi088)," https://www.bosch-sensortec.com/media/boschsensortec/downloads/product_flyer/bst-bmi088-fl000.pdf.Google Scholar
- TDK InvenSence, "Iam-20680ht," https://invensense.tdk.com/products/iam-20680ht/.Google Scholar
- Y. S. Park, Y. Shin, J. Kim, and A. Kim, "3d ego-motion estimation using low-cost mmwave radars via radar velocity factor for pose-graph SLAM," IEEE Robotics Autom. Lett. (RA-L), vol. 6, no. 4, pp. 7691--7698, 2021.Google Scholar
- Intel, "Intel nuc 11 pro kit nuc11tnhi30z," https://ark.intel.com/content/www/us/en/ark/products/121572/intel-nuc-11-pro-kit-nuc11tnhi30z.html.Google Scholar
- C. Wu, F. Zhang, Y. Fan, and K. J. R. Liu, "Rf-based inertial measurement," in The 33th ACM Special Interest Group on Data Communication (SIGCOMM), 2019.Google Scholar
- S. Shen, M. Gowda, and R. R. Choudhury, "Closing the gaps in inertial motion tracking," in The 24th Annual International Conference on Mobile Computing and Networking (MobiCom), 2018.Google Scholar
- J. Gong, X. Zhang, Y. Huang, J. Ren, and Y. Zhang, "Robust inertial motion tracking through deep sensor fusion across smart earbuds and smartphone," Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. (IMWUT), vol. 5, no. 2, pp. 62:1--62:26, 2021.Google ScholarDigital Library
- S. H. Cen and P. Newman, "Precise ego-motion estimation with millimeter-wave radar under diverse and challenging conditions," in International Conference on Robotics and Automation, ICRA, 2018.Google Scholar
- S. H. Cen and P. Newman, "Radar-only ego-motion estimation in difficult settings via graph matching," in International Conference on Robotics and Automation, ICRA, 2019.Google Scholar
- A. Kingery and D. Song, "Improving ego-velocity estimation of low-cost doppler radars for vehicles," IEEE Robotics Autom. Lett. (RA-L), vol. 7, no. 4, pp. 9445--9452, 2022.Google ScholarCross Ref
- D. Adolfsson, M. Magnusson, A. W. Alhashimi, A. J. Lilienthal, and H. Andreasson, "Lidar-level localization with radar? the CFEAR approach to accurate, fast, and robust large-scale radar odometry in diverse environments," IEEE Trans. Robotics, vol. 39, no. 2, pp. 1476--1495, 2023.Google ScholarDigital Library
- R. Zhang, Y. Zhang, D. Fu, and K. Liu, "Scan denoising and normal distribution transform for accurate radar odometry and positioning," IEEE Robotics Autom. Lett., vol. 8, no. 3, pp. 1199--1206, 2023.Google Scholar
- R. Weston, M. Gadd, D. D. Martini, P. Newman, and I. Posner, "Fast-mbym: Leveraging translational invariance of the fourier transform for efficient and accurate radar odometry," in 2022 International Conference on Robotics and Automation, (ICRA), 2022.Google Scholar
- S.-Y. Kwon, S. Kwak, J. Kim, and S. Lee, "Radar sensor-based ego-motion estimation and indoor environment mapping," IEEE Sensors Journal, 2023.Google Scholar
- NAVTECH, "Cts350-x," https://navtechradar.com/clearway-technical-specifications/.Google Scholar
- Texas Instruments Incorporated, "mmwave cascade imaging radar rf evaluation module," https://www.ti.com/tool/MMWCAS-RF-EVM.Google Scholar
- C. Doer and G. F. Trommer, "An EKF based approach to radar inertial odometry," in IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, (MFI), 2020.Google Scholar
- J. Michalczyk, R. Jung, and S. Weiss, "Tightly-coupled ekf-based radar-inertial odometry," in IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS), 2022.Google Scholar
- H. Chen, Y. Liu, and Y. Cheng, "DRIO: robust radar-inertial odometry in dynamic environments," IEEE Robotics Autom. Lett., vol. 8, no. 9, pp. 5918--5925, 2023.Google ScholarCross Ref
- E. B. Quist, P. C. Niedfeldt, and R. W. Beard, "Radar odometry with recursiveransac," IEEE Trans. Aerosp. Electron. Syst., vol. 52, no. 4, pp. 1618--1630, 2016.Google ScholarCross Ref
- Y. Sun, W. Wang, L. Mottola, R. Wang, and Y. He, "Aim: Acoustic inertial measurement for indoor drone localization and tracking," in The 20th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2022.Google Scholar
- W. Wang, L. Mottola, Y. He, J. Li, Y. Sun, S. Li, H. Jing, and Y. Wang, "Micnest: Long-range instant acoustic localization of drones in precise landing," in The 20th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2022.Google Scholar
- "Optitrack," https://optitrack.com/.Google Scholar
- P. Zhao, C. X. Lu, B. Wang, N. Trigoni, and A. Markham, "3d motion capture of an unmodified drone with single-chip millimeter wave radar," in IEEE International Conference on Robotics and Automation (ICRA), 2021.Google Scholar
- J. Xu, G. Chi, Z. Yang, D. Li, Q. Zhang, Q. Ma, and X. Miao, "Followupar: Enabling follow-up effects in mobile ar applications," in Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (Mobisys), 2021.Google Scholar
- R. S. Ayyalasomayajula, A. Arun, C. Wu, S. Rajagopalan, S. Ganesaraman, A. Seetharaman, I. K. Jain, and D. Bharadia, "Locap: Autonomous millimeter accurate mapping of wifi infrastructure," in 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI), R. Bhagwan and G. Porter, Eds., 2020.Google Scholar
- B. Liang, P. Wang, R. Zhao, H. Guo, P. Zhang, J. Guo, S. Zhu, H. H. Liu, X. Zhang, and C. Xu, "Rf-chord: Towards deployable RFID localization system for logistic networks," in 20th USENIX Symposium on Networked Systems Design and Implementation, (NSDI), 2023.Google Scholar
- J. Liu, J. Gao, S. K. Jha, and W. Hu, "Seirios: leveraging multiple channels for lorawan indoor and outdoor localization," in The 27th Annual International Conference on Mobile Computing and Networking (MobiCom), 2021.Google Scholar
- I. Pefkianakis and K. Kim, "Accurate 3d localization for 60 ghz networks," in Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems (Sensys), 2018.Google Scholar
- A. Angeli, S. Doncieux, J. Meyer, and D. Filliat, "Real-time visual loop-closure detection," in IEEE International Conference on Robotics and Automation, (ICRA), 2008.Google Scholar
Index Terms
- Push the Limit of Single-Chip mmWave Radar-Based Egomotion Estimation with Moving Objects in FoV
Recommendations
Moving target detection in foliage using along track monopulse synthetic aperture radar imaging
This paper presents a method for detecting moving targets embedded in foliage from the monostatic and bistatic synthetic aperture radar (SAR) data obtained via two airborne radars. The two radars, which are mounted on the same aircraft, have different ...
Effective baseline estimation in dual-channel synthetic aperture radar for moving target imaging and relocation
AbstractIn a dual-channel synthetic aperture radar (SAR) system with the phase centers being positioned along the track, the motion parameters can be obtained by the interferometric phase. When the phase centers of a dual-channel system ...
MIMO Radar Moving Target Detection Against Compound-Gaussian Clutter
This paper focuses on the moving target detection (MTD) problem for multiple-input multiple-output (MIMO) radar in the presence of compound-Gaussian clutter. By exploiting the spatial and velocity diversities, we devise two different generalized ...
Comments