Conclusions
In this paper, we present a precisely synchronized event-based dataset, designed especially for multi-sensor fusion in SLAM applications, with a particular emphasis on VR and AR scenarios. Alongside setting up commonly used stereo regular cameras and an IMU, we have integrated stereo event cameras. We specialize in recording sequences to imitate real-life scenarios, while adding challenging sequences such as low light and fast motion. Consequently, it is our aspiration that this dataset will serve as a valuable resource for the advancement of research in the domain of event-based multi-sensor fusion algorithms.
摘要
近年来, 事件相机以其低延迟、 高动态范围和高时间分辨率等特点受到越来越多关注. 这些特点使它特别适合应用于虚拟和增强现实(VR/AR)领域. 为了促进事件相机在VR/AR应用中的三维感知和定位算法的发展, 我们引入用于虚拟和增强现实场景的双目事件相机数据集(SEVAR). 该数据集以头戴式设备为主体, 覆盖几种常见的室内场景序列, 包括面向事件相机的快速运动和高动态范围的挑战性情景. 我们发布了第一组VR/AR场景的感知和定位数据集, 该数据集由双目事件体相机、 30 Hz双目标准相机和1000 Hz惯性测量单元采集. 相机的放置方式、 视场和分辨率与商用头戴设备(如Meta Quest Pro)相似. 所有传感器在硬件上进行时间同步. 为更好地开展定位精度和轨迹的评估, 提供了由动作捕捉系统捕捉的位姿真值. 数据集见https://github.com/sevar-dataset/sevar.
This is a preview of subscription content, log in via an institution to check access.
Data availability
The dataset can be found at https://github.com/sevar-dataset/sevar.
References
Alzugaray I, Chli M, 2018. Asynchronous corner detection and tracking for event cameras in real time. IEEE Rob Autom Lett, 3(4):3177–3184. https://doi.org/10.1109/LRA.2018.2849882
Barranco F, Fermuller C, Aloimonos Y, et al., 2016. A dataset for visual navigation with neuromorphic methods. Front Neurosci, 10:49. https://doi.org/10.3389/fnins.2016.00049
Calabrese E, Taverni G, Easthope CA, et al., 2019. DHP19: dynamic vision sensor 3D human pose dataset. IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops, p.1695–1704. https://doi.org/10.1109/CVPRW.2019.00217
Campos C, Elvira R, Rodríguez JJG, et al., 2021. ORB-SLAM3: an accurate open-source library for visual, visualinertial, and multimap SLAM. IEEE Trans Rob, 37(6): 1874–1890. https://doi.org/10.1109/TRO.2021.3075644
Delmerico J, Cieslewski T, Rebecq H, et al., 2019. Are we ready for autonomous drone racing? The UZH-FPV drone racing dataset. Int Conf on Robotics and Automation, p.6713–6719. https://doi.org/10.1109/ICRA.2019.8793887
Furgale P, Rehder J, Siegwart R, 2013. Unified temporal and spatial calibration for multi-sensor systems. IEEE/RSJ Int Conf on Intelligent Robots and Systems, p.1280–1286. https://doi.org/10.1109/IROS.2013.6696514
Furrer F, Fehr M, Novkovic T, et al., 2018. Evaluation of combined time-offset estimation and hand-eye calibration on robotic datasets. In: Hutter M, Siegwart R (Eds.), Field and Service Robotics. Springer Proceedings in Advanced Robotics, Vol. 5. Springer, Cham, p.145–159. https://doi.org/10.1007/978-3-319-67361-5_10
Gao L, Liang YX, Yang JQ, et al., 2022. VECtor: a versatile event-centric benchmark for multi-sensor SLAM. IEEE Rob Autom Lett, 7(3):8217–8224. https://doi.org/10.1109/LRA.2022.3186770
Gehrig M, Aarents W, Gehrig D, et al., 2021. DSEC: a stereo event camera dataset for driving scenarios. IEEE Rob Autom Lett, 6(3):4947–4954. https://doi.org/10.1109/LRA.2021.3068942
Geiger A, Lenz P, Stiller C, et al., 2013. Vision meets robotics: the KITTI dataset. Int J Rob Res, 32(11):1231–1237. https://doi.org/10.1177/0278364913491297
Geneva P, Eckenhoff K, Lee W, et al., 2020. OpenVINS: a research platform for visual-inertial estimation. IEEE Int Conf on Robotics and Automation, p.4666–4672. https://doi.org/10.1109/ICRA40945.2020.9196524
Hu YH, Liu SC, Delbruck T, 2021. v2e: from video frames to realistic DVS events. IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops, p.1312–1321. https://doi.org/10.1109/CVPRW53098.2021.00144
Klenk S, Chui J, Demmel N, et al., 2021. TUM-VIE: the TUM stereo visual-inertial event dataset. IEEE/RSJ Int Conf on Intelligent Robots and Systems, p.8601–8608. https://doi.org/10.1109/IROS51168.2021.9636728
Liu YZ, Fu YJ, Chen FD, et al., 2021. Simultaneous localization and mapping related datasets: a comprehensive survey. https://arxiv.org/abs/2102.04036
Olson E, 2011. AprilTag: a robust and flexible visual fiducial system. IEEE Int Conf on Robotics and Automation, p.3400–3407. https://doi.org/10.1109/ICRA.2011.5979561
Sturm J, Engelhard N, Endres F, et al., 2012. A benchmark for the evaluation of RGB-D SLAM systems. IEEE/RSJ Int Conf on Intelligent Robots and Systems, p.573–580. https://doi.org/10.1109/IROS.2012.6385773
Weikersdorfer D, Adrian DB, Cremers D, et al., 2014. Event-based 3D SLAM with a depth-augmented dynamic vision sensor. IEEE Int Conf on Robotics and Automation, p.359–364. https://doi.org/10.1109/ICRA.2014.6906882
Zhu AZ, Thakur D, Ozaslan T, et al., 2018. The multivehicle stereo event camera dataset: an event camera dataset for 3D perception. IEEE Rob Autom Lett, 3(3):2032–2039. https://doi.org/10.1109/LRA.2018.2800793
Author information
Authors and Affiliations
Contributions
Yuda DONG designed the research. Yuda DONG, Junchi FENG, and Yinong CAO processed the data. Zichao SHU contributed to hand–eye calibration. Yuda DONG and Zetao CHEN drafted the paper. Xin HE, Jianyu WANG, and Lijun LI helped organize the paper. Yuda DONG, Chunlai LI, and Shijie LIU revised and finalized the paper. Xin HE provided research funding.
Corresponding authors
Ethics declarations
All the authors declare that they have no conflict of interest.
Additional information
Project supported by the Zhejiang Provincial Natural Science Foundation of China (No. 2023C03012), the Postdoctoral Preferential Funding Project of Zhejiang Province, China (No. ZJ2022116), and the Independent Project of Hangzhou Institute for Advanced Study, China (No. B02006C019014)
Rights and permissions
About this article
Cite this article
Dong, Y., Chen, Z., He, X. et al. SEVAR: a stereo event camera dataset for virtual and augmented reality. Front Inform Technol Electron Eng 25, 755–762 (2024). https://doi.org/10.1631/FITEE.2400011
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/FITEE.2400011