Abstract
We introduce a framework where visual features, describing the interaction among a robot hand, a tool, and an assembly fixture, can be learned efficiently using a small number of demonstrations. We illustrate the approach by torquing a bolt with the Robonaut-2 humanoid robot using a handheld ratchet. The difficulties include the uncertainty of the ratchet pose after grasping and the high precision required for mating the socket to the bolt and replacing the tool in the tool holder. Our approach learns the desired relative position between visual features on the ratchet and the bolt. It does this by identifying goal offsets from visual features that are consistently observable over a set of demonstrations. With this approach we show that Robonaut-2 is capable of grasping the ratchet, tightening a bolt, and putting the ratchet back into a tool holder. We measure the accuracy of the socket-bolt mating subtask over multiple demonstrations and show that a small set of demonstrations can decrease the error significantly.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Akgun, B., Cakmak, M., Yoo, J.W., Thomaz, A.L.: Trajectories and keyframes for kinesthetic teaching: a human-robot interaction perspective. In: Proceedings of the Seventh Annual ACM/IEEE International Conference on Human-Robot Interaction, pp. 391–398. ACM (2012)
Alexandrova, S., Cakmak, M., Hsiao, K., Takayama, L.: Robot programming by demonstration with interactive action visualizations. In: Robotics: Science and Systems (2014)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Computer Vision–ECCV 2006, pp. 404–417. Springer, Heidelberg (2006)
Calinon, S., Billard, A.: A probabilistic programming by demonstration framework handling constraints in joint space and task space. In: IEEE/RSJ International Conference on Intelligent Robots and Systems. IROS 2008, pp. 367–372. IEEE (2008)
Calinon, S., Guenter, F., Billard, A.: On learning, representing, and generalizing a task in a humanoid robot. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 37(2), 286–298 (2007)
Finn, C., Tan, X.Y., Duan, Y., Darrell, T., Levine, S., Abbeel, P.: Deep spatial autoencoders for visuomotor learning. Reconstruction 117(117), 240 (2015)
Fischer, P., Dosovitskiy, A., Brox, T.: Descriptor matching with convolutional neural networks: a comparison to sift. arXiv preprint arXiv:1405.5769 (2014)
Huang, J., Cakmak, M.: Flexible user specification of perceptual landmarks for robot manipulation. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3296–3303. IEEE (2017)
Huber, M.: A hybrid architecture for adaptive robot control. Ph.D. thesis, University of Massachusetts Amherst (2000)
Ku, L.Y., Learned-Miller, E., Grupen, R.: An aspect representation for object manipulation based on convolutional neural networks. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 794–800. IEEE (2017)
Ku, L.Y., Learned-Miller, E.G., Grupen, R.A.: Associating grasp configurations with hierarchical features in convolutional neural networks. In: 2017 IEEE International Conference on Intelligent Robots and Systems (IROS). IEEE (2017)
Ku, L.Y., Sen, S., Learned-Miller, E.G., Grupen, R.A.: Action-based models for belief-space planning. In: Workshop on Information-Based Grasp and Manipulation Planning, at Robotics: Science and Systems (2014)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Pastor, P., Hoffmann, H., Asfour, T., Schaal, S.: Learning and generalization of motor skills by learning from demonstration. In: IEEE International Conference on Robotics and Automation. ICRA 2009, pp. 763–768. IEEE (2009)
Pérez-D’Arpino, C., Shah, J.A.: C-learn: learning geometric constraints from demonstrations for multi-step manipulation in shared autonomy. In: IEEE International Conference on Robotics and Automation (2017)
Phillips, M., Hwang, V., Chitta, S., Likhachev, M.: Learning to plan for constrained manipulation from demonstrations. In: Robotics: Science and Systems, vol. 5 (2013)
Sen, S.: Bridging the gap between autonomous skill learning and task-specific planning. Ph.D. thesis, University of Massachusetts Amherst (2013)
Simo-Serra, E., Trulls, E., Ferraz, L., Kokkinos, I., Fua, P., Moreno-Noguer, F.: Discriminative learning of deep convolutional feature point descriptors. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 118–126 (2015)
Tuytelaars, T., Mikolajczyk, K.: Local invariant feature detectors: a survey. Found. Trends® Comput. Graph. Vis. 3(3), 177–280 (2007)
Acknowledgements
We are thankful to our colleagues Dirk Ruiken, Mitchell Hebert, Jay Wong, Michael Lanighan, Tiffany Liu, Takeshi Takahashi, and former members of the Laboratory for Perceptual Robotics for their contribution on the control basis framework code base. We are also grateful to Philip Strawser, Jonathan Rogers, Logan Farrell, Kimberly Vana, Evan Laske, and the Robonaut Team for their support on Robonaut-2. This material is based upon work supported under Grant NASA-GCT-NNX12AR16A, the Air Force Research Laboratory and DARPA under agreement number FA8750-18-2-0126, and a NASA Space Technology Research Fellowship. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Aeronautics and Space Administration, the Air Force Research Laboratory, DARPA, or the U.S. Government.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Ku, L.Y., Jordan, S., Badger, J., Learned-Miller, E., Grupen, R. (2020). Learning to Use a Ratchet by Modeling Spatial Relations in Demonstrations. In: Xiao, J., Kröger, T., Khatib, O. (eds) Proceedings of the 2018 International Symposium on Experimental Robotics. ISER 2018. Springer Proceedings in Advanced Robotics, vol 11. Springer, Cham. https://doi.org/10.1007/978-3-030-33950-0_35
Download citation
DOI: https://doi.org/10.1007/978-3-030-33950-0_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33949-4
Online ISBN: 978-3-030-33950-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)