Elsevier

Pattern Recognition Letters

Volume 34, Issue 15, 1 November 2013, Pages 1928-1935
Pattern Recognition Letters

Shape recognition of laser beam trace for human–robot interface

https://doi.org/10.1016/j.patrec.2013.03.023Get rights and content

Highlights

  • This paper proposed the concept of the shape identification method of laser beam trace.

  • We constructed the classification algorithm of figures drawn by a laser pointer.

  • Experiments confirmed our system classified typical figures in accuracy of more than 86%.

  • We applied the system to steer the mobile robot only with a laser beam.

Abstract

A robot navigation system using the pattern recognition of figures drawn by a laser pointer has been proposed. Typical figures are registered and assigned to individual robot commands. Each figure is identified based on the feature of its edges. This system detects trace of laser beam and calculates its optical flow vectors. Each figure has its own characteristic distribution pattern of vector inclinations. By evaluating the optical flow pattern of displayed laser spot, the system distinguishes the shape of a laser beam trace and provides the command to a robot corresponding to the drawn figure. The proposed system has been applied to the mobile robot, and shows its effectiveness by steering successfully.

Introduction

Robots are expected to be introduced not only in factories but also at home. Welfare robots are especially required to be navigated easily and simply by unskilled operators. Considering the use of the aged or injured persons in the hospital, robots are needed to be controlled for themselves while they are taken to their bed. It is desirable to use simple equipment for easy operation in robot navigation. Innumerable methods are reported to navigate robots by means of joy sticks (Honga et al., 1998, Zhu et al., 2011), vocal guidance (Buddhika et al., 2010), and so on (Mikawa et al., 2010, Karkoub et al., 2012, Lapointe and Savard, 2011).

Laser pointer is one of the simplest ways to provide visual instruction (Yamazaki et al., 2002, Oyama et al., 2000, Shibata et al., 2011, Seko et al., 2006). It is not as contrived as joysticks. Its operation is less complicated than vocal guidance. The robot operating methods using laser beam have been proposed in previous studies as intuitive device (Suzuki, 2005, Stopp et al., 2002). They used the laser spot to give orders to mobile robots because one can indicate the arbitrary point from away. A laser pointer has been used only to designate the robot destination in previous studies (Fukuda et al., 2009, Ishii et al., 2009, Kurata et al., 2004, Paromtchik and Asama, 2001, Baczynski and Baczynski, 2004, Iwasaki et al., 2006). Other devices are additionally required to give more complicated commands to robots (Kemp et al., 2008, Choi et al., 2008). The sole report is on the blinking pattern of laser luminescence which adds the supplementary instruction way to a laser targetting system (Mizukawa et al., 2003).

The authors have proposed the navigation system which instructs not only the robot destination but also more complicated command using laser luminescent trace (Minato et al., 2011). When a user draws a figure on a wall or on a floor with a laser pointer, the system identifies its shape and directs a robot to act the intended behavior corresponding to the drawn figure. The image of the figure is acquired by CCD camera, and the figure’s edges are extracted as optical flows by image processing. We proposed the method which detects the figure’s characteristic distribution pattern of optical flow inclinations to classify several kinds of figures.

This paper presents the application of robot operating method using the identification of shape drawn by a laser pointer. The concept of robot navigation system is proposed first, while three typical figures are assumed in this study. Identification method of laser trail is investigated by presenting the pre-processing technique for laser beam spot, by analyzing the geometrical features of the laser trail figures, and by determining the criteria of figure identification. It has been confirmed that a number of laser trail figures can be classified based on the criteria. Finally, we design the shape identification algorithm, construct the prototype system, and evaluate it by applying to the mobile robot navigation to give robot commands by laser drawing.

Section snippets

Concept of robot navigation systems

The conceptual diagram of proposed system is shown in Fig. 1(a). When making a mobile robot approach its target, an operator indicates the target by flashing a laser beam to it. Minuter or complicated instruction is given by laser trail figures to employ the robot for intricate tasks This system acquires motion of the laser spot by a CCD camera, obtains geometric features by calculating optical flows, identifies its shape and gives a robot the command corresponding to the drawn figure. Simple

Optical flow of laser spot trace

When a 650 nm laser beam has been shot on the indoor white wall, a luminescent spot is appeared on the surface. Fig. 2(a) indicates an image taken by a CCD camera. Day light or reflection from walls often prevents CCD cameras from detecting the spot. As we use a red laser beam, extraction of the red component helps the spot to stand out against the background. The RGB color space has been evaluated to extract the red elements, and the camera image has been digitally processed. Both the image

Mobile robot navigation

A mobile robot navigation system has been constructed by applying the laser trace shape identification. Based on the above discussion, an algorithm of robot navigation is designed as shown in Fig. 5. Human operators give commands by drawing figures on the wall. Each figure connoted a specified robot motion. The robot is navigated with a laser pointer when the system distinguishes the drawn shape among candidate ones.

Discussions

This paper deals with only three simple figures to show the typical example of shape identification. The same principle can be applied to intricate figures with discretion. The proposed method evaluates only the histogram of the vector incidence. It pays no attention to the order or the origin of stroke. Therefore, it is impossible to classify different figures that consist of the same types of vectors, e. g., a triangle and a hexagon. The identification algorithm needs to evaluate the order

Conclusion

This paper has studied the shape recognition method of the laser beam trace and its application for navigation of the mobile robots controlled by unskilled operators. The concept of geometric feature extraction of laser beam trace figure has been proposed first to apply to figure identification. Edges of figures are focused on instead of the corners.

Next, we have formulized and constructed image processing techniques such as image denoising and optical flow extraction. These methods are applied

References (28)

  • Suzuki, T., Ohya, A., Yuta, S., 2005. Operation direction to a mobile robot by projection lights. In: Proc. IEEE Int....
  • Stopp, A., Baldauf, T., Hantsche, R., Horstmann, S., Kristensen, S., Lohnert, F., Priem, C., Ruscher, B., 2002. The...
  • Fukuda, Y., Kurihara, Y., Kobayashi, K., Watanabe, K., 2009. Development of electric wheelchair interface based on...
  • Ishii, K., Zhao, S., Inami, M., Igarashi, T., Imai, M., 2009. Designing laser gesture interface for robot control. In:...
  • View full text