Tao Fang
ABSTRACT
Evaluating the grasp stability of the robot is critical to robotic manipulation. It is very effective to combine visual and tactile information to evaluate the stability of a grip. However, most of the methods directly concatenate features of heterogeneous data that may ignore the comprehensive interaction between different modalities. Furthermore, existing methods ignore the influence of intra-modality that may degrade the performance of grasp stability assessment. To address this issue, we proposed a framework named visual and tactile deep bilinear network (VTDBN) to evaluate the grasp stability of robots by integrating visual data and tactile data. Moreover, we conduct comprehensive experiments to build a dataset that can be used for training and testing. The experiment results show that VTDBN model significantly improves the performance of robotic grasp stability assessment and outperforms traditional methods.
- Billard A, Kragic D. Trends and challenges in robot manipulation[J]. Science, 2019, 364(6446): eaat8414.Google Scholar
Cross Ref
- Levine S, Pastor P, Krizhevsky A, Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection[J]. The International journal of robotics research, 2018, 37(4-5): 421-436.Google Scholar
- Yoo Y, Lee C Y, Zhang B T. Multimodal Anomaly Detection based on Deep Auto-Encoder for Object Slip Perception of Mobile Manipulation Robots[C]//2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2021: 11443-11449.Google Scholar
- Yang C, Du P, Sun F, Predict robot grasp outcomes based on multi-modal information[C]//2018 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2018: 1563-1568.Google Scholar
- Saxena A, Driemeyer J, Ng A Y. Robotic grasping of novel objects using vision[J]. The International Journal of Robotics Research, 2008, 27(2): 157-173.Google ScholarDigital Library
- Cockbum D, Roberge J P, Maslyczyk A, Grasp stability assessment through unsupervised feature learning of tactile images[C]//2017 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2017: 2238-2244.Google Scholar
- Calandra R, Owens A, Jayaraman D, More than a feeling: Learning to grasp and regrasp using vision and touch[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 3300-3307.Google ScholarCross Ref
- Kamon I, Flash T, Edelman S. Learning to grasp using visual information[C]//Proceedings of IEEE International Conference on Robotics and Automation. IEEE, 1996, 3: 2470-2476.Google Scholar
- Cui S, Wang R, Wei J, Grasp state assessment of deformable objects using visual-tactile fusion perception[C]//2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020: 538-544.Google Scholar
- Bekiroglu Y, Kragic D, Kyrki V. Learning grasp stability based on tactile data and HMMs[C]//19th International Symposium in Robot and Human Interactive Communication. IEEE, 2010: 132-137.Google Scholar
- Gao Y, Hendricks L A, Kuchenbecker K J, Deep learning for tactile understanding from visual and haptic data[C]//2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2016: 536-543.Google Scholar
- Long X, Gan C, Melo G, Multimodal keyless attention fusion for video classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence. 2018, 32(1).Google Scholar
- Xiao F, Lee Y J, Grauman K, Audiovisual slowfast networks for video recognition[J]. arXiv preprint arXiv:2001.08740, 2020.Google Scholar
- Cheng J, Zhang J, Han Y, Integrative analysis of histopathological images and genomic data predicts clear cell renal cell carcinoma prognosis[J]. Cancer research, 2017, 77(21): e91-e100.Google Scholar
Index Terms
- Grasp stability assessment through the fusion of visual and tactile signals using deep bilinear network
Recommendations
Delay-dependent stability and stabilizability of uncertain jump bilinear stochastic systems with mode-dependent time-delays
This paper considers the problems of robust stability and robust stabilizability for a class of uncertain jump bilinear stochastic systems with mode-dependent time delays. By transforming the jump system under consideration into an equivalent descriptor ...
Robust Stability of Discrete Bilinear Uncertain Time-Delay Systems
This paper addresses the problem of robust stability for discrete homogeneous bilinear time-delay systems subjected to uncertainties. Two kinds of uncertainties are treated: (1) nonlinear uncertainties and (2) parametric uncertainties. For parametric ...
Global exponential stability and dissipativity of generalized neural networks with time-varying delay signals
This paper investigates the problems of exponential stability and dissipativity of generalized neural networks (GNNs) with time-varying delay signals. By constructing a novel LyapunovKrasovskii functionals (LKFs) with triple integral terms that contain ...
Comments