Keywords

1 Introduction

This research work is part of a project whose aim is to develop an innovative design framework for lower limb prosthesis design [1]. In particular, we refer to the module, named Socket Modelling Assistant (SMA), that permits to replicate the traditional, handmade socket design process and embeds design rules and orthopedic technicians’ knowledge.

One of the most important steps of socket design is the identification of critical zones that have to be modified to ensure the realization of a comfortable socket. In fact, due to the anatomical characteristic of the residuum, the appropriate pressure over specific areas is crucial to provide the right fit and prevent pain. We focused the attention on below knee residuum, which is characterized by the following critical zones (Fig. 1): patella, medial tibia, patella tendon, lateral femoral condyle, crest of the tibia, tibia terminal, head of the fibula, lateral tibia, popliteal depression and fibula end [2, 3].

Fig. 1.
figure 1

Anatomical zones of below knee residuum

Full size image

Therefore, our main goal has been to develop a module to be integrated within SMA that can identify automatically the mentioned areas. Once they are recognized, SMA can perform the corresponding deformations, in automatic way, in order to create the so-called load and off-load socket zones [1].

The implemented recognition algorithm exploits a supervised learning approach [4], in which a number of artificial neural networks have been trained for the recognition of a specific anatomical zone. In particular, we focused on the automatic recognition of the patella, the patella tendon and the tibia terminal.

2 Training Dataset

Artificial Neural Networks (ANN) are machine learning computational models inspired by the biological brain and are useful to approximate complex functions that depend on a large number of inputs [5].

In supervised learning, a given ANN is trained with a samples dataset consisting of a number of input data and their corresponding correct output (usually computed/labeled by human experts). After the training, the ANN is able to estimate the transfer function between input and output and to generalize its response.

In our specific case, the training dataset is composed by a set of 3D models of the residuum (STL files), on which prosthetists have labeled the anatomical zones. In particular, in order to carry out preliminary tests, experts marked the patella center, the patella tendon and the tibia terminal (Fig. 2).

Fig. 2.
figure 2

Patella, patella tendon and tibia terminal labelled by experts

Full size image

One of the drawbacks of supervised learning is that it requires a good number of training data. This is an important issue because, due the nature of our context, it’s quite challenging to get a sufficient amount of lower limb residuum models.

We used 5 models as seed shapes to generate new residuum models. Performing random scaling and skewing geometric operations on each seed have generated a new set of models. The generated models are synthetic but are useful to train a recognition system, because they respect the proportions and shapes of a real human limb. Generation of new synthetic data by performing geometric transformations on real data is a useful technique used, for example, in optical character recognition systems [8].

Models generated from the first 4 seeds have been used to train the neural network (with a proportion of 80 % for the training set, and 20 % for the validation set), while the models generated from the fifth seed have been used to test the performance of the networks on new cases. The models distribution for the training, validation and test dataset is represented in Table 1.

Table 1. Dataset creation
Full size table

3 Features Extraction

In order to convert the 3D geometric data in an input suitable for a neural network, we created a grid of 20 × 25 points in correspondence of the frontal view. Then, we casted a ray, perpendicular to the grid (Z axis), from each of the grid points, computing its intersection on the residuum model surface (Fig. 3).

Fig. 3.
figure 3

Ray casting on 3d model surface

Full size image

The distances between the grid points plane and the intersection points represent our features, thus the neural network input is represented by a matrix of 20 × 25 distances. Distances have been normalized in −1.0 /+ 1.0 range (Fig. 4).

Fig. 4.
figure 4

Features extraction

Full size image

In a given dataset sample, a correct output corresponds to each input. In this case, the output consists of the position of the marker that experts assigned during the manual labeling phase.

Specifically, the output is represented as X and Y coordinates on the grid, in correspondence of the centre of a given anatomical zone (assuming the Z axis perpendicular to the grid).

4 Neural Network Architecture and Training

The neural network type used to recognize the anatomical zones is a feed-forward network [6] composed by three layers (Fig. 5). The input layer has 500 neurons (20 × 25 grid points), the hidden layer has 30 neurons and the output layer has 2 neurons, that represent the X and Y coordinates of the centre of the anatomical zone to be identified.

Fig. 5.
figure 5

Neural network architecture

Full size image

The activation function of the hidden neurons is the sigmoid function, while the input and output neurons have a linear activation function. The network has been trained with the back propagation algorithm [6].

For each anatomical area to be identified (i.e., patella, patella tendon and tibia terminal), a specific neural network has been trained with the back propagation algorithm.

Since the size of the training dataset was small, to avoid over-fitting we adopted the early-stopping approach. We trained several networks, and stopped the training when the validation error started to increase [7]. Then we kept the network with the minimum validation error. Figures 6, 7, 8 show the training error and validation error during the training epochs, respectively for the patella, patella tendon and tibia terminal networks.

Fig. 6.
figure 6

Neural network for patella area recognition training and validation error graph

Full size image
Fig. 7.
figure 7

Neural network for patella tendon area recognition training and validation error graph

Full size image
Fig. 8.
figure 8

Neural network for tibia terminal area recognition training and validation error graph

Full size image

5 Results

Once the neural networks have been trained, we tested their performance on models generated from the seed 5. The test error represents the distance between a given anatomical area marker position, estimated by the network, and the real marker position, labeled by human experts. Table 2 shows the test errors, that is a measure of the performance of the networks. The test dataset has been composed by 10 models labeled by human experts, and not used during the training.

Table 2. Test errors on unseen cases
Full size table

Figure 9 shows the markers positions of each anatomical area: red markers have been placed by humans experts, while green markers have been placed on the coordinates estimated by the neural networks. Considering the application field, the performances of the neural network are good, as stated by domain experts.

Fig. 9.
figure 9

Patella, patella tendon and tibia terminal predicted (green) and real (red) marker positions.

Full size image

6 Conclusions

In this paper, we presented a system able to automatically recognize the important anatomical areas of a below-knee residuum. In particular, the system deals with the patella, the patella tendon and the tibia terminal zones. Once the anatomical areas have been identified, correct geometric modification can be applied, in order to create load and off-load zones on the residuum.

This is a preliminary work to test the validity of the approach, especially concerning the extraction of the learning features from the 3D model of the residual limb.

Results are promising, however we have planned to develop further the learning model, expanding the training dataset and adding new learning features. It could be worth to investigate the generation of new synthetic data by merging operation: creating a new residuum model by blending the geometries of two existing models.

Once the recognition system will be embedded into the Socket Modelling Assistant, new learning data can be acquired. During the design process, the system identifies and marks the anatomical areas position; the user can accept the automatic choice of the system, or can manually refine the position of the markers, in order to get more precision. This operation generates a new sample learning data, useful to train further the recognition algorithm. We have planned to apply the recognition algorithm also to above knee residuum.