Vision-based control of an autonomous disassembly station

doi:10.1016/S0921-8890(01)00121-X

Robotics and Autonomous Systems

Volume 35, Issues 3–4, 30 June 2001, Pages 179-189

https://doi.org/10.1016/S0921-8890(01)00121-X Get rights and content

Abstract

In the EU about 10 millions of used cars have to be wrecked per year. So the disposal of these cars is of increasing importance in the next years. In a disassembly process for used cars one has to cope with the objects in an undefined state in contrast to the car production. For this purpose the use of vision-based control is motivated and a vision guided robotic system for an autonomous disassembly process is presented in this paper. An active stereo camera system is used as vision sensor. A combination of gray value and contour-based object recognition and a position measurement approach with implicit detection of occlusions is described. A system for the autonomous disassembly of wheels was successfully tested and shown in a final presentation. In this demonstrator a stereo vision system, a force–torque sensor, a task planning module, and a special unscrewing tool were integrated.

Section snippets

Introduction and state of the art

The development of intelligent or autonomous robots is one of the most quickly evolving fields in automation and robotics research. The integration of sensors, the analysis of their signals, and the control of the behavior of the robotic systems due to the sensor input leads to an action–perception cycle, in which the visual input is of special importance.

In this paper, we are going to present an active vision system for disassembling used cars. In the first step, we concentrated on automatic

System overview

In cooperation with other institutes, we have developed an autonomous disassembly station for used cars. The first task we are concentrating on is unscrewing the wheels of an arbitrary car of unknown type. This task consists of several subtasks typical for disassembly: (i) the object itself, here the wheel, has to be detected and recognized; (ii) the wheel nuts have to be detected and recognized; (iii) the wheel nuts have to be measured in all six degrees of freedom for automatic handling.

Preprocessing

Both the recognition process as well as the stereo matching algorithm are based on the same preprocessing for the gray-scale images. To facilitate the stereo matching process (see also Section 5) and to avoid the estimation of epipolar lines for each image point, we first transform the images of our convergent axis geometry into parallel axis images by rectification [1]. In this method, the images are projected onto a common rectification plane, which is placed in a distance f parallel to the

Object recognition

In the vision system, a combination of contour, gray value, and knowledge-based recognition is used. The different approaches are described in the following subsections.

3D position measurement

In order to recover the 3D structure of the scene, we use a stereo matching approach. Thus, it is necessary to determine the displacement of the corresponding points in the images. There are some reasons not to use the phase-difference of the Gabor filters directly for disparity estimation. The basis for phase-difference methods consists of the Fourier shift theorem. However, due to the local spatial support of the filters used in practice, the Fourier shift theorem does not strictly apply.

Experimental results

The system described in the previous sections was tested successfully in several demonstrations in our laboratory. The tests show that the recognition process and the position determination is as accurate as necessary for the task of dismantling wheels. The recognition process was checked at varying illuminations with wheels of different size and containing three, four, or five bolts. The wheels were always recognized in the scene, so that they could be focused. The detection of the bolts was

Acknowledgements

This paper shows the research results of the part of the Heinz Nixdorf Institute in the project DEMON, which was founded by the German Ministry for Education and Research (523-4001-01 IN 506 B 2). For the final presentation of the project, the results of the Institute for Robotics Research (IRF Dortmund), Institute for Neurocomputing (NERO Bonn), the company FER (Magdeburg), the company TZN (Unterlüß) and the Heinz Nixdorf Institute (HNI Paderborn) were integrated. The authors would like to

U. Büker, born in 1965, studied Computer Science and Mathematics at the University of Paderborn and received his Diploma in 1990. He then joined the Computer Vision Group in Paderborn and got his Doctoral Degree in Electrical Engineering in 1995. In 2001, he got his “venia legendi”. Currently he holds the position of Oberingenieur in Paderborn. His main research interests are active vision systems, knowledge-based and neural recognition strategies for hybrid systems, and the use of parallel and

References (25)

D.H. Ballard
Generalizing the Hough transform to detect arbitrary shapes
Pattern Recognition
(1981)
G. Stockman
Object recognition and localization via pose clustering
Computer Vision, Graphics, and Image Processing
(1987)
N. Ayache, C. Hansen, Rectification of images for binocular and trinocular stereovision, in: Proceedings of the Ninth...
I. Biederman
Recognition by components: A theory of human image understanding
Psychological Review
(1987)
T.O. Binford, Visual perception by computer, in: Proceedings of the IEEE Conference on Systems and Control, Miami, FL,...
G. Borgefors, Hierarchical chamfer matching: A parametric edge matching algorithm, IEEE Transactions on Pattern...
U. Büker, G. Hartmann, Knowledge based view control of a neural 3-D object recognition system, in: Proceedings of the...
U. Büker, Hybrid object models for robot vision, in: Proceedings of the 24th Annual Conference on the IEEE Industrial...
U. Büker, G. Hartmann, Object representation: On combining object-centered and viewer-centered strategies, in:...
H.H. Bülthoff, S.Y. Edelman, M.J. Tarr, How are three-dimensional objects represented in the brain? Technical Report,...

D. Gabor

Theory of communication

Journal of IEE

(1946)

W.E.L. Grimson et al.

On the sensitivity of the Hough transform for object recognition

IEEE Transactions on Pattern Analysis and Machine Intelligence

(1990)

Cited by (44)

Challenges and prospects of automated disassembly of fuel cells for a circular economy
2023, Resources, Conservation and Recycling Advances
The hydrogen economy is driven by the growing share of renewable energy and electrification of the transportation sector. The essential components of a hydrogen economy are fuel cells and electrolysis systems. The scarcity of the resources to build these components and the negative environmental impact of their mining requires a circular economy. Concerning disassembly, economical, ergonomic, and safety reasons make a higher degree of automation necessary.
Our work outlines the challenges and prospects on automated disassembly of fuel cell stacks. This is carried out by summarizing the state-of-the-art approaches in disassembly and conducting manual non-/destructive disassembly experiments of end-of-life fuel cell stacks. Based on that, a chemical and mechanical analysis of the fuel cell components is performed. From this, an automation potential for the disassembly processes is derived and possible disassembly process routes are modeled. Moreover, recommendations are given regarding disassembly system requirements using a morphological box.
Automated generation and execution of disassembly actions
2021, Robotics and Computer-Integrated Manufacturing
Citation Excerpt :
Cutting around point fasteners was used primarily to cut around screws and rivets not located near edges. In [28], it was not applied unless explicitly demonstrated. The action is defined as cutting a square with four separate cuts, centred at the fastener head, and encompassing the diameter of the fastener head with some tolerance (Fig. 8).
Product variation and the unavailability of end-of-life product and condition information makes automation of disassembly a significant challenge. Several research attempts have been made employing different techniques to approach uncertainties in the disassembly process automatically, but it has made little influence on industrial applications. This research presents a robotic disassembly system aimed at addressing these issues. The system is tested on disassembly of various models of LCD screens with promising results, limited by sensing and model inaccuracies which were able to be resolved by user intervention. Specific product information obtained from user demonstration is supplemented with background knowledge relating actions to the fasteners to be disestablished. This goal-driven approach allows the substitution of dissimilar tools––namely a screwdriver, hole saw, and angle grinder––along with flexible reactive planning in the case of operation failures. A method of assessing and selecting desirable disassembly actions based on geometrical estimations of destructiveness is implemented. Finally, an improved method for the evaluation of systems employing (semi-) destructive disassembly is proposed.
Screw detection for disassembly of electronic waste using reasoning and re-training of a deep learning model
2021, Procedia CIRP
Growing populations, increasing standards of living, and reliance on advancing technologies are resulting in an emerging abundance of electronic waste. Automating disassembly of these electronic products is a vital part of improving end-of-life product treatment where hazardous chemicals and materials are present. Furthermore, non-destructive disassembly is ideal to preserve the embodied energy in components from manufacturing. This requires the removal, and hence detection, of fasteners like screws in a disassembly environment. Crosshead screws are a common fastener type used in LCD monitors. This paper proposes a method of screw detection for disassembly and presents results of detection of crosshead screws on components of various models of LCD monitors in a disassembly cell. The system first applies gamma correction to standardize brightness––or luminance–– of images. Generic knowledge about screw features in the form of a deep learning model is then used to visually detects screws. The results are analyzed against common screw location combinations on product components in order to logically reason about possible undetected screws that are also likely to exist. Finally, true negative detections are collected as new training data and the deep learning model is re-trained on these missed detections to improve performance; tailoring and adapting to the environment of the specific disassembly cell.
Economic modelling of robotic disassembly in end-of-life product recovery for remanufacturing
2020, Computers and Industrial Engineering
Citation Excerpt :
Robotic disassembly can also be used to replace human labour in difficult and hazardous tasks. The first significant efforts in automatic disassembly were the studies by Hesselbach (1994) who recovered valuable materials from printed circuit boards in an automatic cell, and Büker et al. (2001), who used machine vision to automatically remove wheels from used cars. Later research in robotic disassembly includes the works by Gil, Pomares, Diaz, Candelas, and Torres (2007), defining a robotic system using several robots in a parallel and a cooperative way, and Torres, Puente, and Díaz (2009), designing an automatic cooperative disassembly robotic system using a task planner based on decision trees.
The key to fully achieving the benefits of remanufacturing lies in the efficient and cost-effective reuse of components from end-of-life (EoL) products. Unlike recycling and disposal, remanufacturing contributes to a firm’s profits and reduces environmental impacts. Maximising the economic value of recovery options while meeting environmental regulations is of prime importance. This paper proposes a novel model to design the robotic disassembly process (RDP) for EoL products. An optimisation decision-making model has been designed to find the near-optimal solution that achieves the best economic performance of the process while simultaneously yielding the optimal disassembly level, or “stopping point”, disassembly sequence plan, and recovery option for the components in a framework. Furthermore, the model helps the robotic cell to re-plan the ongoing disassembly process. To do so, it recalculates the economic outputs, making decisions about the re-planned optimal disassembly level and the reassigned recovery option for the components: reuse, remanufacturing, recycling or disposal. The model was been tested using a case study based on a gear pump. The results demonstrate the effectiveness of the proposed model and provide insights into recovery practices for remanufacturing.
Robot assisted disassembly for the recycling of electric vehicle batteries
2015, Procedia CIRP
The rising number of electric vehicles (EV) will eventually lead to a comparable number of EV batteries reaching their end-of-life (EOL). Efforts are therefore being made to develop technologies and processes for recycling, remanufacturing and reusing EV batteries. One important step of many such processes is the disassembly of EOL EV batteries, which poses a challenging task due to unpredictable lot sizes and volumes, as well as significant variations in battery design between different car models. In response to these challenges and the increasing demand, we present a concept for a battery disassembly workstation where a human is assisted by a robot. While the human performs the more complex tasks, the proposed robot performs simple, repetitive tasks such as removing screws and bolts. Such a robot requires 1) a suitable procedure for the unscrewing task, 2) a means of autonomously changing screwdriver bit in accordance with the variety of screws and bolts found in EV batteries, and 3) some means of acquiring information regarding the location of these fasteners. This paper summarises the results of our preliminary investigations.
Enhancing Disassembly Practices for Electric Vehicle Battery Packs: A Narrative Comprehensive Review
2023, Designs

View all citing articles on Scopus

S. Drüe received his Diploma and Doctoral Degree in Electrical Engineering from the University of Paderborn in 1983 and 1988, respectively. Since 1988, he is Akademischer Oberrat at the Department of Electrical Engineering. His research interests include computer vision, active vision systems, artificial neural networks and their applications.

N. Götze, born in 1969, received his diploma in Business Administration and Electrical Engineering from the University of Paderborn in 1995. He then worked at the Computer Vision Group in Paderborn, where he got his Doctoral Degree in Electrical Engineering in 2000. Now Dr. Goetze is with the Signum Bildtechnik GmbH in Munich. His main research interests are active vision systems and real time appearance-based object learning.

G. Hartmann was born in Fürth, Germany on 7 April 1937. He received his M.S. degree in Physics in 1962 from the University of Erlangen and after postgraduate studies of Nuclear Physics he received his Ph.D. degree in 1968. He was Development Engineer in the field of nuclear instrumentation, radiometric measurement, and industrial automation between 1968 and 1976, and Head of development until 1979. At this time, he received a call to the Faculty of Electrical Engineering at the University of Paderborn. Between 1983 and 1987, he served as Vice-President at this university. From this time on, he is a member of the governing board of the Heinz Nixdorf Institute, a center of interdisciplinary research in the field of computer science at the Paderborn University. His field of research is computer vision, and from 1994 to 2000 he was the President of the German Association for Pattern Recognition.

B. Kalkreuter, born in 1969, received his Diploma in Electrical Engineering from the University of Paderborn in 1998. He then worked at the Computer Vision Group in Paderborn in the field of vision systems for autonomous robots. In 2000, he joined the Orga Kartensysteme GmbH in Paderborn.

R. Stemmer, born in 1969, studied Electrical Engineering at the University of Paderborn and received his Diploma in 1995. He then joined the Computer Vision Group in Paderborn. His main research interests are active stereovision systems, robot vision and hand-eye calibration.

R. Trapp, born in 1968, received his Diploma in Electrical Engineering from the University of Paderborn, Germany, in 1994. He then held a scholarship at the Graduate Center of the Heinz Nixdorf Institute, interdisciplinary research center for Computer Science and Technology within the University of Paderborn. During this time he has worked on active vision systems, stereo-image processing, image filtering and camera calibration methods. He has published several research papers in these fields and he is the co-author of the On-line Compendium of Computer Vision CVonline. He received his Ph.D. degree in electrical engineering from the University of Paderborn in 1998.

View full text