Data-Driven Robotic Grasping in the Wild

Robotic grasping has seen tremendous advancements in recent years. Yet, the current paradigm of manipulation research is typically some form of table-top manipulation

in constrained setups or in simulation. Building general purpose personal robots that can autonomously grasp unknown objects in unstructured environments like homes is

an open problem. In this thesis, we explore important directions in scaling data-driven grasping to the diversity and constraints imposed by the real world. We first discuss how we can go beyond picking individual objects in isolation to 6-

DOF grasping in clutter. Most existing methods train policies on datasets collected in curated settings (in lab or simulation) and hence may not cope with the mismatch

in data distribution when deployed in the wild. We build and open-source a low-cost mobile manipulator platform to parallelize data collection in challenging settings like

homes and show that policies trained on this data generalize to novel objects in unseen homes. As a result, we also discuss ideas for scaling robot learning with several robots

and transferring policies between different hardware. Yet, we hypothesize that visual perception alone is insufficient for robustness and present a self-supervised tactile-based

re-grasping framework to close the loop on grasp execution. Lastly, we strive to go beyond robotic pick-and-place and generalize to diverse semantic manipulation tasks.

We do so by scaling task-oriented grasping datasets with crowdsourcing and learning from semantic information like knowledge graphs.