Aron Harder
ABSTRACT
A popular metric for measuring progress in autonomous driving has been the "miles per intervention". This is nowhere near a sufficient metric and it does not allow for a fair comparison between the capabilities of two autonomous vehicles (AVs). In this paper we propose Scenario2Vector - a Scenario Description Language (SDL) based embedding for traffic situations that allows us to automatically search for similar traffic situations from large AV data-sets. Our SDL embedding distills a traffic situation experienced by an AV into its canonical components - actors, actions, and the traffic scene. We can then use this embedding to evaluate similarity of different traffic situations in vector space. We have also created a first of its kind, Traffic Scenario Similarity (TSS) dataset which contains human ranking annotations for the similarity between traffic scenarios. Using the TSS data, we compare our SDL embedding -with textual caption based search methods such as Sentence2Vector. We find that Scenario2Vector outperforms Sentence2Vector by 13% ; and is a promising step towards enabling fair comparisons among AVs by inspecting how they perform in similar traffic situations. We hope that Scenario2Vector can have a similar impact to the AV community that Word2Vec/Sent2Vec have had in Natural Language Processing datasets.
- Pave poll: Americans wary of avs but say education and experience with technology can build trust. https://pavecampaign.org/, May 2020. https://pavecampaign.org/news/pave-poll-americans-wary-of-avs-but-say-education-and-experience-with-technology-can-build-trust/.Google Scholar
- Daisuke Wakabayashi. Self-driving uber car kills pedestrian in arizona, where robots roam. New York Times. https://www.nytimes.com/2018/03/19/technology/uber-driverless-fatality.html.Google Scholar
- Tesla car that crashed and killed driver was running on autopilot, firm says. The Guardian, Mar 2018. https://www.theguardian.com/technology/2018/mar/31/tesla-car-crash-autopilot-mountain-view.Google Scholar
- Walther Hans Karl Wachenfeld. How stochastic can help to introduce automated driving. PhD thesis, Technische Universität Darmstadt, 2017.Google Scholar
- Rick Salay and Krzysztof Czarnecki. Using machine learning safely in automotive software: An assessment and adaption of software process requirements in iso 26262. arXiv preprint arXiv:1808.01614, 2018.Google Scholar
- Apple Inc. Our approach to automated driving system safety. Apple ADS Safety. https://www.apple.com/ads/ADS-Safety.pdf.Google Scholar
- Aurora safety report: The new era of mobility. Aurora. https://aurora.tech/vssa/index.html.Google Scholar
- The autox safety factor. autox. https://autox.ai/safety.html.Google Scholar
- A matter of trust: Ford's approach to developing self driving vehicles. ford. https://media.ford.com/content/dam/fordmedia/pdf/Ford_AV_LLC_FINAL_HR_2.pdf.Google Scholar
- 2018 self driving safety report. General Motors. https://www.gm.com/our-stories/self-driving-cars.html.Google Scholar
- Reinventing safety: a joint approach to automated driving systems. Daimler and Bosch. https://www.daimler.com/innovation/case/autonomous/reinventing-safety-2.html.Google Scholar
- Navya safety report the autonom era. Navya. https://navya.tech/safety-report/.Google Scholar
- Delivering safety: Nuro's approach. Nuro.Google Scholar
- Safety is what drives us: Introducing the nvidia self-driving safety report. Nvidia, Oct 2018. https://blogs.nvidia.com/blog/2018/10/23/introducing-self-driving-safety-report/.Google Scholar
- Starsky robotics: Voluntary safety self-assessment. Starsky Robotics.Google Scholar
- 2019 self-driving safety report. https://www.tusimple.com/wp-content/uploads/2019/05/TuSimple-2019-Self-Driving-Safety-Report.pdf.Google Scholar
- Uber advanced technologies group a principled approach to safety. Uber, 2018. https://uber.app.box.com/v/UberATGSafetyReport.Google Scholar
- Safety report and first responders waymo safety report. Waymo. https://waymo.com/safety/.Google Scholar
- Safety is foundational to our mission. Zoox.Google Scholar
- Marc René Zofka, Sebastian Klemm, Florian Kuhnt, Thomas Schamm, and J Marius Zöllner. Testing and validating high level components for automated driving: simulation framework for traffic scenarios. In 2016 IEEE Intelligent Vehicles Symposium (IV), pages 144--150. IEEE, 2016.Google ScholarDigital Library
- Fisher Yu, Wenqi Xian, Yingying Chen, Fangchen Liu, Mike Liao, Vashisht Madhavan, and Trevor Darrell. Bdd100k: A diverse driving video database with scalable annotation tooling. arXiv preprint arXiv:1805.04687, 2(5):6, 2018.Google Scholar
- Xin Rong. word2vec parameter learning explained. arXiv preprint arXiv:1411.2738, 2014.Google Scholar
- Anthony Hu, Alex Kendall, and Roberto Cipolla. Learning a spatio-temporal embedding for video instance segmentation, 2019.Google Scholar
- S. Hu, Yikang Li, and Baoxin Li. Video2vec: Learning semantic spatio-temporal embeddings for video representation. In 2016 23rd International Conference on Pattern Recognition (ICPR), pages 811--816, 2016.Google ScholarCross Ref
- Open Autonomous Safety. Oas scenarios, 2018.Google Scholar
- Foretellix. Open m-sdl, 2019.Google Scholar
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL '02, page 311--318, USA, 2002. Association for Computational Linguistics.Google ScholarDigital Library
- Chin-Yew Lin. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74--81, Barcelona, Spain, July 2004. Association for Computational Linguistics.Google Scholar
- Satanjeev Banerjee and Alon Lavie. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65--72, Ann Arbor, Michigan, June 2005. Association for Computational Linguistics.Google Scholar
- Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. Cider: Consensus-based image description evaluation. CoRR, abs/1411.5726, 2014.Google Scholar
- Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. SPICE: semantic propositional image caption evaluation. CoRR, abs/1607.08822, 2016.Google Scholar
- Matt Kusner, Yu Sun, Nicholas Kolkin, and Kilian Weinberger. From word embeddings to document distances. volume 37 of Proceedings of Machine Learning Research, pages 957--966, Lille, France, 07--09 Jul 2015. PMLR.Google Scholar
- Nayyer Aafaq, Syed Zulqarnain Gilani, Wei Liu, and Ajmal Mian. Video description: A survey of methods, datasets and evaluation metrics. CoRR, abs/1806.00186, 2018.Google Scholar
- Jinkyu Kim, Anna Rohrbach, Trevor Darrell, John Canny, and Zeynep Akata. Textual explanations for self-driving vehicles. Proceedings of the European Conference on Computer Vision (ECCV), 2018.Google ScholarDigital Library
- Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. Unsupervised learning of sentence embeddings using compositional n-gram features. CoRR, abs/1703.02507, 2017.Google Scholar
- William Webber, Alistair Moffat, and Justin Zobel. A similarity measure for indefinite rankings. ACM Transactions on Information Systems (TOIS), 28(4):1--38, 2010.Google Scholar
Recommendations
Numerical Analysis of Tractor Accidents using Driving Simulator for Autonomous Driving Tractor
ICMRE'19: Proceedings of the 5th International Conference on Mechatronics and Robotics EngineeringAutonomous driving of automobiles is a hot research topic in recent years. The autonomous driving tractor also has been studied in the agricultural field as well as an autonomous driving automobile. On the other hand, tractor accidents frequently occur ...
Evaluation of three ionospheric delay computation methods for ground-based GNSS receivers
GNSS observables for ionospheric estimation are commonly based on carrier-to-code leveling (CCL) and precise point positioning (PPP) methods. The CCL method is a geometry-free method which uses carrier phase to level pseudorange observation for ...
Analysis of satellite-induced factors affecting the accuracy of the BDS satellite differential code bias
Differential code bias (DCB) is one of the main error sources of the positioning, navigation and timing services and slant total electron content extraction. The DCB can usually be estimated together with ionospheric model coefficients from the data of ...
Comments