1 Introduction

Slip recognition represents an important sense that must be generated at the moment of slip occurrence due to manipulation or to any external disturbance on the grasped object. This operation provides a grasping force optimization to prevent the object from slipping at a low grasping force, and hence preventing damage to the object. The recognition algorithm consists of detection and prevention of slip, as slip detection is based on a multisensory system to indicate the earliest stage of slip before gross slipping has occurred; after that, the incipient slippage must be prevented through translating the generated signal to the actuating response as a feedback to increase the grasping force. To achieve this operation, artificial intelligent algorithms are implemented for acquiring the multisensory signals as input information, and then generating the feedback response. In this field, Fujimoto et al. [3], designed an artificial finger skin with a vibro-tactile sensing element to detect the slipping occurrence through using a multilayered artificial neural network (ANN) technique. Briglen and Gosselin [2] introduced a real-time controller based on a fuzzy force control method by using tactile and position information to detect slipping from observing the variation in the normal force value, actuator torque, and motion of the finger. Tada and Hosoda [17] proposed a network to acquire the multisensory signals of slips by using three different sensation devices – vibration, pressure, and vision devices; this network is trained through the grasping experiences to sense the microslip occurrence. Lopez et al. [9] proposed a new strategy to detect slip by building blocks to be able to detect the very low frequency signals like those generated by the slip, and then these signals are encoded as a response of the skin by using the neural network. Mazid and Fakhrul Islam [10] suggested a mathematical model for the computation of scattered energy of vibrations, which are sensed by the stylus during the slippage of an object from the robot hand, with the help of ANN to estimate the optimal grasping force of the object according to its physiomechanical properties. Soliman et al. [16] developed a simple fuzzy logic controller to control the grasping process through receiving the object’s acceleration, acceleration rate, pushing force, and applied force to adjust the finger motion. Schopfer et al. [14] presented an approach to detect incipient slip with training of ANN on the data from the fast piezo-resistive sensor to detect slipping. Shirafuji and Hosoda [15] presented a method for slip detection by using polyvinylidene fluoride films, and slip prevention by using strain gauges with neural network processing that controls the actuators of the robotic hand. Herrera [5] presented a tactile control model for lifting unfamiliar objects with grasp stability and human-like behavior by using the ANN to estimate the friction coefficient and to detect incipient slipping. Jamali and Sammut [7] used a hidden Markov model classifier that was learned by taking the multidimensional tactile sensor data after transforming it into a symbolic sequence to predict slip.

In summary, most previous studies tackle the problem of slipping as a result static or quasi-static load on the grasped object, and that was because the implemented sensors or the recognition techniques were unable to recognize the difference between the associated oscillation with slippage case and that produced by external excitation on the grasped object or on any part of the robotic system. In this paper, the main advantage of the proposed method is represented by the ability to detect slipping without the need for any knowledge about the characteristics of the grasped object, such as the surface nature, weight, and nature of its excitation.

2 System Simulation

The system shown in Figure 1 represents a section of one finger of the underactuated hand, which is composed of three fingers; each finger has one degree of freedom and actuated by the pulleys-tendon mechanism with flexible elements (tension springs) [1, 12].

Figure 1:

The Artificial Hand Mechanism.

The fingertip has a separate design from the finger structure, but it is connected to the finger through a limited motion joint to allow the contact force to be transmitted to the force sensor. A layer of viscoelastic material covers the fingertip to increase the robustness of the grasping; both the finger and the fingertip can be considered as one part governed by the classical Lagrange formulation as follows:

For simplification, the left side of Eq. (1) can be neglected because it is considered very small in comparison with the right side, specifically after assuming the absence of any relative motion between the object and the hand before slip occurrence. Also, the tendon friction and elasticity can be neglected; hence, Eq. (1) becomes

(τ_i) can be coupled to (Fti) as follows:

and

From the setting of D-H parameters for the finger mechanism, the Jacobian matrix can be derived and shown as follows:

Taking only the effect of contact force components (normal and tangential) and substituting Eqs. (3)–(5) into Eq. (2) yields

where Lni = Licosθi and Lti = Lisinθi.

The equilibrium of the fingertip part about its joint can be shown as

By rewriting Eqs. (6) and (7) in matrix notation:

After solving Eq. (8), the force components can be formulated as follows:

From modeling of the pulleys-tendon system, (Fti) can be formulated as

The ratio of the contact force components is given by

From the definition of friction cone and for stable grasp, the contact force at the fingertip must be inside the friction cone:

However, if the grasped object is with unknown surface properties, namely coefficient of friction, then Eq. (13) is incapable of recognizing the slippage occurrence. For this reason, the process of slip detection must be achieved through monitoring of the result behavior of Eq. (12) instantaneously with time. Having followed the cited approach, there is a need to understand the slippage mechanism that shows the variation of the coefficient of friction value during the slip-dynamic contact problem. The mentioned variation in (μ_i) value is attributed to the discontinuous sudden decay from static (μ_si) to dynamic or kinematic value (μ_di) [6]. From this fact, one can recognize the onset of slippage by recognizing the onset of variation in the output result of Eq. (12).

3 System Description

The experimental setup consists of three systems as described below.

3.1 Artificial Arm-Hand

The implemented artificial arm-hand was designed especially for this work; the mechanism of the hand is composed of a power screw that is coupled to a direct current (DC) geared motor to transform the rotational motion from the actuator to linear motion for the group of nut and pulleys, and then the tendons are pulled to close the three fingers on the grasped object. These tendons are connected between tensile springs and the mechanism of the finger; this allows the actuator to pull the tendons after grasping to increase the contact force by expanding the springs. The fingertip is designed, as clarified in Figure 1, to be capable of measuring both components of contact force and to satisfy the conditions of sensor mounting [18]. This hand is grouped with the artificial arm, which transforms the excitation from the minishaker to the hand, as shown in Figure 2.

Figure 2:

The Arm-Hand System.

3.2 Sensory System

The sensory system is used to measure all parameters that produce the output result of Eq. (12), as the FlexiForce sensor is mounted at the combination of the fingertip as shown in Figure 3; from this sensor, the (Fsi) value is provided. For measuring the joint angle (θ_i) at each finger, the limited rotational potentiometer is used with resistance (10 kΩ) and angle range (270°); however, the tendon tension (Fti) is measured indirectly by applying Eq. (11) with measurement of the joint angle and the multiturn potentiometer, which is calibrated to measure the stroke distance of the nut (y). This sensor has resistance (5 kΩ) and angle range (10 turns); in addition, it is rotated with the power screw (motor axis) through gear engagement to increase its sensitivity, as shown in Figure 3.

Figure 3:

Sensors Mounting.

The hall-effect sensor [4] is used to describe the slippage situation by sensing the variation in distance between the grasped object, where this sensor is mounted, and the magnetic piece is fixed on the center of the palm of the hand, as shown in Figure 3. The signal of this sensor is not calibrated because it is used just for slip observation. All the signals of the mentioned sensors are acquired by an analog to digital converter, and then transformed to a personal computer for processing and generating the signal, which is used for slip detection via Matlab-Simulink software.

3.3 Grasped Object

The grasped object is designed in a prism shape as shown in Figure 4A, and it is provided by three DC motors that have unbalance rotating mass to generate the excitation in the object; these motors are distributed symmetrically on the faces of the object. Also, it is connected to a pulling system to force the object to slip through the tendon that is connected to the grasped object in one side, and the other side is turned around the rotated drum by a geared DC motor, as shown in Figure 4B,C. The contact region at each face is coated with the layer of spongy rubber with ability to replace this layer with other materials (glass and wood) as the test required. The weight of this object with the sensors, motors, and the connected wires is 253.5 g with spongy rubber coating, 336 g with glass coating, and 280 g with wood coating.

Figure 4:

(A) The Grasped Object. (B, C) The Components of the Pulling System.

4 Neural Network Control for Slip Prevention

After understanding the slippage mechanism, the slippage recognizer has been created with the aid of artificial intelligent algorithms in order to control the grasping force. In other words, the slippage is detected and prevented without the need for any knowledge about the properties of the grasped object, such as weight, surface texture, and external disturbance on the object. In this work, the Matlab Neural Network Toolbox has been used for pattern recognition. This network is a feedforward back-propagation neural network with two hidden layers, as shown in Figure 5; the first hidden layer consists of 10 neurons, and the second hidden layer consist of 2 neurons with tansigmoid transfer function for each layer. A scaled conjugate gradient algorithm [11] is utilized for numerical optimization of the process.

Figure 5:

The ANN Structure.

4.1 Data Set for Network Training

The network has been trained under specific conditions, such as covering the object by a spongy rubber layer, with the effect of the quasi-static and dynamic load conditions. Thus, there are two types of tests to force the grasped object to slip, and then the reading data are recorded from the sensory system. The first test process starts by grasping the object, which is coated with spongy rubber, until reaching a stable grasp situation; in this case, the object is free from any external load except its weight, till the pulling system is turned on, and thus, the object starts to slip. In this process, there are many examples of results [1, 12] that were recorded in real-time. Figures 6 and 7 show a selective test from these samples. The second test is achieved with the same primary grasping conditions as in the first test, except that the grasped object is excited by the embedded unbalance motors. It is worthwhile to mention here that the object starts slipping at the moment of excitation. Many samples about the second test are given in Refs. [1, 12]. One of these samples is shown in Figure 8.

Figure 6:

The Variation of Force Ratio (fti/fni) (First Test).

Figure 7:

The Applied Load on the Grasped Object (First Test).

Figure 8:

The Variation of Force Ratio (fti/fni) (Second Test).

The results of the first test presents the behavior of the contact force component ratio (fti/fni) versus real time, as shown in Figure 6. As the applied load on the grasped object increases, the hall-effect sensor signal indicates a sudden change in its behavior, namely, the object is starting to slip; at this moment, the force ratio (fti/fni) also starts to vary almost in a similar pattern, and this pattern can be explained by the gradual increasing in force ratio from the beginning of the slip. At this moment, the real coefficient of friction is decreased to kinetic coefficient of friction (μ_di); at this condition, the contact force is out of the friction cone as in Eq. (13).

For the other test, the results show the variation in the value of force ratio (fti/fni) in alternating sequence at the moment of the beginning of the slip; this is due to the variation of the magnitude and direction of contact force components. The range of cited variation in force ratio (fti/fni) is increased at the moment of slip in comparison with the decrease in the coefficient of friction to the kinetic coefficient of friction (μ_di). This means that the friction cone condition is not satisfied [1, 12].

The signals of force ratio (fti/fni), as shown in Figures 6–8, are divided into subsignals for a specific time interval, and each subsignal is converted to vector form by using Matlab-Simulink software through buffering the main signal by a specific buffer size to prepare the input vector of the neural network, as shown in Figure 9.

Figure 9:

Schematic Diagram for the Buffering Process.

After the buffering process, the associated target vector with an input vector of the neural network is prepared in binary mode (0, 1) according to the force ratio signal behavior; the target vector [0, 1]^T at the small range of variation in force ratio subsignal is considered as a stable grasp case, while the target vector [1, 0]^T for the large variation in subsignal is considered as a slippage case. Figures 10 and 11 show the best neural network training performance with 20 neurons in the first hidden layer; when the grasped object was covered by a spongy rubber layer, it is also grasped under quasi-static and dynamic load conditions. Figures 12 and 13 show the effect of the first hidden layer size on the neural network training performance.

Figure 10:

The Performance of ANN with (20) First Hidden Layer Size Under Quasi-static Load Conditions.

Figure 11:

The Performance of ANN with (20) First Hidden Layer Size Under Dynamic Load Conditions.

Figure 12:

The Performance of ANN with a Different First Hidden Layer Size Under Quasi-static Load Conditions.

Figure 13:

The Performance of ANN with a Different First Hidden Layer Size Under Dynamic Load Conditions.

4.2 Enhancement of Neural Network Performance by Using Input Normalization

The normalization process for the ANN inputs has a noticeable effect on preparing the data to be suitable for the training process. Before this normalization, training the neural networks would have been very slow, as seen from the number of epochs [8]. According to that, the input samples have been normalized, namely are in the range between (–1 and 1). This is done through subtracting the main signal of force ratio from the instantaneous mean value of the same signal, as shown in Figure 14.

Figure 14:

Schematic Diagram of the Normalization Process.

This aids the network to train with best performance, as described below (Figures 15–18).

Figure 15:

The Performance of ANN with Quasi-static Load Conditions Before Normalization.

Figure 16:

The Performance of ANN with Quasi-static Load Conditions After Normalization.

Figure 17:

The Performance of ANN with Dynamic Load Conditions Before Normalization.

Figure 18:

The Performance of ANN with Dynamic Load Conditions After Normalization.

4.3 Neural Network Test

The network is tested under different conditions, in which the grasped object is covered by glass, wood, and spongy rubber, once when the robotic arm is excited and when the grasped object is excited both quasi-statically and dynamically. Tests have been conducted with more than one network having a different input layer size (buffer size). This is to study the effect of the accumulation process for the force ratio signal on the response time of the controller, as the output of this network represents the switching sequence to the actuator of the hand, namely [0, 1] denotes (OFF) and [1, 0] denotes (ON). Figures 19–21 represent the results of these tests with the (10) input layer size.

Figure 19:

(A) The Controller Response Under Quasi-static Load and Object Covered by a Spongy Rubber Layer with (10) Input Layer Size. (B) The Controller Response Under Quasi-static Load and Object Covered by Glass Layer with (10) Input Layer Size. (C) The Controller Response Under Quasi-static Load and Object Covered by Wood Layer with (10) Input Layer Size. (D) Average Delay in Response Time for Different Contact Materials at Quasi-static Load with (10) Input Layer Size.

Figure 20:

(A) The Controller Response Under Dynamic Load (Vibrating Arm) and Object Covered by a Spongy Rubber Layer with (10) Input Layer Size. (B) The Controller Response Under Dynamic Load (Vibrating Arm) and Object Covered by the Glass Layer with (10) Input Layer Size. (C) The Controller Response Under Dynamic Load (Vibrating Arm) and Object Covered by Wood Layer with (10) Input Layer Size. (D) Average Delay in Response Time for Different Contact Materials at Dynamic Load (Vibrating Arm) with (10) Input Layer Size.

Figure 21:

(A) The Controller Response Under Dynamic Load (Vibrating Object) and Object Covered by a Spongy Rubber Layer with (10) Input Layer Size. (B) The Controller Response Under Dynamic Load (Vibrating Object) and Object Covered by the Glass Layer with (10) Input Layer Size. (C) The Controller Response Under Dynamic Load (Vibrating Object) and Object Covered by Wood Layer with (10) Input Layer Size. (D) Average Delay in Response Time for Different Contact Materials at Dynamic Load (Vibrating Object) with (10) Input Layer Size.

5 Discussion

The training and testing processes represent the main operations for evaluating the abilities of the ANN to accomplish the desired task with efficient performance as a grasping controller.

The training process has been done according to understanding the slippage phenomenon, as the network shows good ability to learn with the prepared samples of force ratio signal; in other words, it has minimum error percentage and maximum classification percentage, especially when the number of hidden layer neurons is equal to (20) and (30) for quasi-static load conditions as shown in Figure 12, and that for hidden layer size was equal to (20) and (40) at dynamic load conditions as shown in Figure 13. This learning ability has been enhanced when the training samples of force ratio have been normalized, as at (10) hidden layer size the mean squared error at best validation performance is decreased from (0.035827) to (2.2455e–8) and also the percentage of the classified patterns is increased from (95.8%) to (100%).

In addition, the normalized input network is able to be treated with different types of contact surface and nature of the load on the grasped object. This appears in the testing process when the object is covered with glass and wood layers under the effect of different load conditions, as shown in Figures 19–21.

The diversity in pairs of contact materials represents one of the testing conditions to evaluate the controller operation, as this condition have differed from that used in the training process. Each material has special behavior during the contact and slippage occurrence. According to that, one can discover the diversity in the controller response with changing in the pairs of the contact material, as the experiments that used the spongy rubber as a covering layer for the grasped object show more stability and reliability than the other materials. This can be justified by the compliant nature of the spongy rubber beside the cancellous nature of its surface. The spongy rubber compliant with the compliance of the fingertips covers will increase the contact area, and then the friction force also will increase. As a result, rubber generates the friction force mainly by an adhesion friction mechanism [13]. There is also another reason for this behavior: as it is known that spongy rubber is the covering material used in the training process, by its nature, the testing process with the spongy rubber has a better response than that with the other contact materials.

Also in this field, the contact between the rubber of the fingertips cover and the glass layers shows a stable behavior because the adhesion friction mechanism is in play [13]. However, the experiments with contact of pairs of rubber and wood layers have different responses as they were noisy and confusing; this is due to the diversity in friction mechanism of rubber on wood from the other cited contact materials. Where, in this case, the deformation friction mechanism comes into play, especially the implemented wood layers have a fibrous surface that is mounted in contrary with the direction of slippage.

It is worthwhile to mention here that the cited test of the ANN represents one of the many tests to optimize the required value of the average delay in response time. This average delay is an important parameter that is considered in this study to evaluate the ANN performance as a controller, as this parameter depends on the input layer size of the network. Hence, increasing the size of the input layer will increase the delay time, as shown in Figures 19D, 20D, and 21D. However, increasing the size of the input layer will enhance the network’s ability to recognize the moment of slippage occurrence; this appeared especially in the test with an object covered with wood, as shown in Figures 19C, 20C, and 21C, as the network is confused at the (10) input layer size but it is working better at the (20) input layer size and more, as shown in Figure 22A–C.

Figure 22:

(A) The Controller Response Under Quasi-static Load and Object Covered by Wood Layer with (20) Input Layer Size. (B) The Controller Response Under Dynamic Load (Vibrating Object) and Object Covered by Wood Layer with (20) Input Layer Size. (C) The Controller Response Under Dynamic Load (Vibrating Arm) and Object Covered by Wood Layer with (20) Input Layer Size.

6 Conclusion

The proposed ANN is trained on the basis of the patterns of slip signals with a novel design of fingertips to detect slipping of a grasped object under dynamic load conditions. In the mathematical model, the slip detection signal has been generated by conventional sensors. The generated signal of slip detection shows good agreement with the physical definition of the slippage phenomena, despite the associated noise effect on the contact force component ratio signal. This noise affects the signal analysis eventually. This problem and the effect of changing the surface material of the grasped object on the behavior of the slip signal has been solved through normalizing the slip signal to enhance the detection process. In the future, this work must be developed by designing an ANN that is capable of adapting to any change in test parameters; in other words, the network will be able to update its memory (weights and bias) during the operation in real environment.