Keywords

1 Introduction

By accumulation of a body serum, the subject is suffered from edema with swelling of a leg, hand, or face. Many healthy people have a minor symptom of edema. Patients who are treated for the cancer have a serious symptom of edema. It is required to perform continual long-term treatments such a massage or exercise to change for the better [1, 2]. Simultaneously, a quantitative evaluation and visualization of edema conditions are also very important to motivate the patient by preventing from getting bored with treatments.

For quantitative evaluation of edema, volume measurement is one of practical methods [3]. Due to the volume change of lymph fluid, a volume of a limb increases or decreases. These increase/decrease of the volume means that the symptom of edema is getting worse/better. Conventionally, a measuring tape is used for a diagnosis in hospitals to measure a circumference of patient’s limb instead of its volume value [3,4,5]. However, this method has poor repeatability and accuracy.

In recent years, the methods using a specific depth camera are proposed to obtain volume of the leg [6, 7]. The first step of these methods is an acquisition of a 3D model of subject’s leg. The next step is a calculation of its volume value by computing the 3D model. There are some previous methods which use a depth camera for a volume measurement. However, it is difficult to use it as home care system because most of depth cameras are expensive to be used in home care system.

Kiyomitsu et al. [8] proposed to evaluate the edema of leg by using Microsoft Kinect with a volume measurement. Kinect is a RGB-D camera shown in Fig. 1 which can obtain a depth map as well as a RGB image [9]. Kinect is already used as a game controller in the world. Furthermore, “Kinect for Windows SDK” contains Kinect Fusion [10, 11] which is an application to obtain a 3D model easily.

Fig. 1.
figure 1

Microsoft Kinect sensor.

Full size image

It is necessary to measure the amount of change for the volume precisely so that the purpose of our study is quantitative evaluation of edema. Due to this, Kiyomitsu et al. constructed a reproducible measurement system whose output value has less dispersion in repetitive measurements. However, visualization technique for change of edema condition was not established in the previous researches.

In this paper, therefore, based on the research of Kiyomitsu et al., we propose a new visualization technique to visualize the change in shape of leg. We only measured the leg since the leg is important part for the patients to be better. In Sect. 2, we describe our proposed method to measure the volume and visualization for shape change of the leg. In Sect. 3, we describe the experiments and results of this study. In Sect. 4, we describe the conclusion of our study.

2 Proposed Method

In this study, we first measure the 3D shape of the leg by using Kinect based on the research of Kiyomitsu et al. Firstly, we scan the leg with Kinect Fusion technique and points of the legs are acquired in real time. Secondly, the point cloud is processed to remove unnecessary parts. To decide the range of the measurement, we set the coordinate axis along the leg length. Finally, in this research, we visualize the shape change. The previous and current shape models are overlapped by registration, then the change amounts are colorized.

2.1 Acquisition of a Point Cloud

We use Kinect Fusion [10, 11] for the acquisition of a point cloud of the leg. Kinect Fusion is one of the applications contained in Kinect for Windows SDK. We can obtain a point cloud in real time by scanning around the leg with Kinect sensor which used as a hand-held depth camera. Figure 2 shows the screen shot of recording by Kinect Fusion. We can move Kinect sensor while looking at the reconstructed point cloud on the monitor in real time, so that it is possible to scan it with checking any error such a lack or hole of a point cloud.

Fig. 2.
figure 2

Screen shot of recording by Kinect Fusion.

Full size image

2.2 Removal of Unnecessary Parts

A point cloud obtained by Kinect Fusion includes not only the leg but also other parts like a chair and a floor. For this reason, we have to remove these unnecessary parts. Firstly, we remove the points except the leg and the floor. For this procedure, we use “MeshLab” [12] which is free software to edit a 3D model that we can implement the removal, translation, rotation, and filling a hole of a point cloud. After this procedure, the floor still remain. It is difficult to remove floor parts from a point cloud because an arch of a foot is attached to the floor widely. Therefore we remove the floor parts automatically by plane model segmentation algorithm. Simultaneously, we translate and rotate a point cloud to reset its coordinates by setting the XZ-plane along the floor, thus Y-axis become the length of the leg. This is preprocessing for following procedure, which is shown in Fig. 3.

Fig. 3.
figure 3

Removal of unnecessary parts and the floor.

Full size image

2.3 Setting of the Stable Range of the Measurement

We decide the stable range of the measurements with the height of the leg. The purpose of our study is the measurement of the change in edema, therefore it is important to set a stable range of the measurement. In this method, we extract the intended point cloud by masking along the height of the leg. It is noted that the direction of the leg length corresponds to Y-axis due to the preprocessing which was discussed in the previous sub-section. Thus we conduct masking with Y-coordinate to set the range of the measurement.

2.4 Visualization of Shape Change Amount

It is impossible to confirm the partial change of edema by measuring the increase and decrease of the volume. Therefore, we visualize partial changes in shape.

First, we registrate the acquired previous shape model and the current shape model. In this research, we use GO-ICP algorithm [13] for registration. The GO-ICP algorithm is the first method to propose a global optimal solution for the Euclidean registration problem defined by ICP in 3D. Based on the established branch-and-bound theory [14] for global optimization, Yang et al. [13] realize a superior registration that is always accurate and does not fall into a local solution. Figure 4 shows the registrations of two point clouds using these methods. It can be seen that the Go-ICP method yields the accurate registration, although the registration with the conventional ICP method is insufficient.

Fig. 4.
figure 4

Registration by ICP method and GO-ICP method.

Full size image

Next, the amount of the shape change is visualized. In this method, the partial distance between the two point clouds describe the amount of the shape change. Therefore, we obtain the partial distance between two points clouds with the fast ray-triangle intersection algorithm proposed by Moller and Trumbore [15]. The ray-triangle intersection of Tomas Moller et al. is an algorithm for intersection determination between a ray and its corresponding triangle.

Firstly, we define the current shape model as point cloud X and the previous shape model as point cloud Y. The surface normals are obtained at all points of the point cloud X which has been registered with the point cloud Y. We estimate the surface normal from 50 neighbor points of the interest point, the surface normal obtained at this procedure is taken as the vertex normal to the interest point. We introduce a viewpoint in order to unify the directions of normals. Next, we make it sure that the normals face the opposite of the viewpoint in order to the normals face the outside of the legs. Even if you set the viewpoint to the centroid of the model and point the opposite of the viewpoint, not all normals face outside the model of the legs. To avoid this problem, we divide the shape model into upper model and lower model at the ankle and set two viewpoints on the upper model and lower model. Figure 5 shows the set position of the viewpoint. Then the obtained normals are facing outside of the shape model as shown in Fig. 6.

Fig. 5.
figure 5

The setting position of the viewpoint.

Full size image
Fig. 6.
figure 6

Comparison of the obtained normals with single viewpoint and double viewpoints.

Full size image

Secondly, we construct a surface on point cloud Y. This surface is constructed with a triangular mesh. Next, intersection determination is performed on the normal of the point cloud X and the triangular mesh of the point cloud Y. The distance between the point and the triangular mesh obtained by the intersection determination is taken as a partial distance in the two shape models. The state of the intersection determination is shown in Fig. 7.

Fig. 7.
figure 7

The state of the intersection determination.

Full size image

Finally, the amount of partial shape change can be visualized by coloring the point cloud X according to the obtained distance as shown in Fig. 8.

Fig. 8.
figure 8

Color coding according to distance. (Color figure online)

Full size image

3 Experiment

In this study, we first shows the result of registration. Next, we obtained the amount of shape change and visualized the change amount of the edema reconstruction model.

3.1 Registration Result of Two 3D Models

Figure 9(a) shows the initial position before registration with the leg of a healthy subject and the legs with reproduced edema by winding bandages. Figure 9(b) shows the results of registration. As shown in the expanded view of Fig. 9(b), the height is shifted just by go-ICP once. It is impossible to make a correct comparison if there is a deviation in height. Therefore, it is necessary to correct the height by the proposal method as shown in the Fig. 10. After Go-ICP, the portion below the ankle extracted from the point cloud X is defined as a point cloud X’. A transformation matrix is acquired by executing the GO-ICP again for the point cloud X’ and the point cloud Y. The heights of the point cloud X and the point cloud Y are made to be equal by applying the acquired transformation matrix to the point cloud X. It was possible to correct the height, then the two 3D models can be correctly overlapped as shown in Fig. 9(c).

Fig. 9.
figure 9

Registration result and height correction result.

Full size image
Fig. 10.
figure 10

The overview of height correction.

Full size image

3.2 Visualization Result of Partial Shape Change

The result of color-coded display of the amount of shape change of the edema is shown in Fig. 11. Figure 11(a) shows the case where the edema is improved, on the other hand, Fig. 11(b) shows the case where the edema worsens. It can be seen that the amount of shape change of edema could be visualized. Looking above the ankle. Figure 11(a) shows the blue color as a whole, indicating that the swelling is improved. On the other hand, Fig. 11(b) shows the red color as a whole, indicating that the swelling is getting worse. Also, since both figures are green below the ankle, we can see that there is no change. The black portion could not be colored because the intersection determination was not performed. From this result, it is possible to know the position where the volume change occurred.

Fig. 11.
figure 11

Color-coded display of the amount of shape change. (Color figure online)

Full size image

4 Conclusion

In this paper, we propose a visualization method of change amount of shape using Microsoft Kinect for an evaluation of a leg edema. We obtain a point cloud of the leg by Kinect Fusion technique and the partial change of the shape is displayed with a color-coded manner according to the change amount. Our experimental results show that the proposed technique was able to visualize the state of edema. Our future work is measurement of a leg of a patient with edema. Additionally, we aim to realize registration of 3D models independent of patient’s posture at measurement. This technique will make it possible to deal with bedridden people. Furthermore, not only the shape but also multi-faceted edema symptom such as skin color and tightness can be evaluated, aiming for a practical edema evaluation system.