Keywords

1 Introduction

In neuroscientific studies, fiber tracking, also known as tractography, is an important way to explore white matter structures. Special techniques have been developed for this purpose, including diffusion tensor imaging (DTI) [1], high angular resolution diffusion imaging (HARDI) [2], etc. Based on these techniques, many fiber tracking algorithms were proposed, such as streamline tractography [3, 4], probabilistic tractography [5], etc. The streamline tractography based on DTI is the most commonly used one in neurosciences and clinical applications. But, due to the single-tensor model, DTI-based streamline tractography can not handle partial volume effects [6]. To handle complex fiber configurations, Malcolm et al. proposed the filtered multitensor tractography [7], where diffusion-weighted imaging (DWI) data are treated as a mixture of Gaussion tensors whose parameters are recursively estimated by unscented Kalman filter (UKF). So the method is also called UKF tractography. Recently, Chen et al. further showed the advantages of UKF tractography in processing crossing fibers and edema regions [8].

Besides fiber tracking, visualization of fiber tracts and diffusion tensors is also a key issue of tractography. In general, fiber tracts are rendered as curve lines or thin tubes with colors encoding fiber segment orientations [9, 10]. And diffusion tensors are represented as geometric glyphs in three dimensional spaces. The commonly used glyph is ellipsoid whose shape just fits the eigensystem of diffusion tensor [11]. But, from the view of data analysis, it is not enough to get full understanding of tractography, because many other properties or measures, such as volume ratio, mean diffusivity, fractional anisotropy, etc., are not included in the glyph visualization. As an alternative, such measures as fractional anisotropy are often visualized as the background or side-by-side views. However, this method affects the efficiency of visual analysis, since we have to switch and make alignment between different views. If we can integrate tensor measures or properties into tensor glyph visualization, it would be better to explore tractography data simultaneously and interactively.

With the above motivation, we design a new solution to visualize UKF tractography data, where tracts, tensors and related scalar measures are integrated and visualized in the same framework, providing global and local views of tractography data for comparisons. The key of the solution is the tensor glyph design. For UKF tractography, the tensor’s second eigenvalue is assumed to equal to the third one [7]. This means that the third eigenvalue is redundant and leaves space to integrate other information. Based on the idea, we design a new glyph whose shape is similar to cylinder but the lateral surface is cut into small rectangles. It is a cylindrical polygon to some extent, so we refer it to cylingon glyph for convenience. The lateral faces of cylingon are used to encode tensor measures through color mapping. And cylingons are then attached to fiber tracts. By this way, fiber tracts, diffusion tensors, and derived tensor measures are integrated in a unified view. With initial tests and applications, we found that the newly proposed solution provided more successful visualization of tractography data, and improved the efficiency of data analysis. It also helped to get more insight into UKF tractography to inspire further study.

In the remainder of the paper, we first introduce the two-tensor model used by UKF tractography in Sect. 2, and then briefly review related work of tractography visualization in Sect. 3. The newly proposed visualization is addressed in Sect. 4, including the glyph design, the visualization scheme, and the interactive analysis. Section 5 provides our initial tests and applications. Section 6 closes the paper with conclusions.

2 Two-Tensor Model

As mentioned in Sect. 1, UKF tractography is a kind of filtered multitensor tractography. The local fiber model could be single-tensor, two-tensor, or more complicated ones. In practice, the two-tensor model is the most common choice since it provides better results as reported in [7, 8]. For this reason, this paper focuses on the visualization of data generated with the two-tensor model. Here, we introduce the two-tensor model as the background.

For a general diffusion tensor, there are three eigenvalues (\( \lambda_{1} \), \( \lambda_{2} \), and \( \lambda_{3} \)) and three corresponding eigenvectors (\( e_{1} \), \( e_{2} \), and \( e_{3} \)). But, tensors in the two-tensor model are simplified. Firstly, each tensor is assumed to has a dominant major diffusion direction \( e_{1} \) with the major eigenvalue \( \lambda_{1} \), and the other two minor eigenvalues are the same, i.e., minor eigenvalues \( \lambda_{2} = \lambda_{3} \). Moreover, both tensors have the same contribution to the signal. As a result, diffusion weighted imaging is formulated as:

$$ s_{i} = \frac{{s_{0} }}{2}(e^{{ - bu_{i}^{T} D_{1} u_{i} }} + e^{{ - bu_{i}^{T} D_{2} u_{i} }} ), $$
(1)

where \( s_{0} \) is the baseline signal, \( b \) is the constant for signal acquisition, \( u_{i} \) is the gradient, \( D_{1} \) and \( D_{2} \) are tensor matrices. With the assumptions, \( D_{1} \) is characterized by its major eigenvector \( v_{1} \), major eigenvalue \( \lambda_{11} \), and minor eigenvalue \( \lambda_{12} \), and \( D_{2} \) has the similar characteristic parameters \( v_{2} \), \( \lambda_{21} \), and \( \lambda_{22} \). These model parameters can be recursively estimated in the framework of UKF tractography [7].

Although they are simplified, tensors used in UKF tractography still have the same properties as general full tensors. There are dozens of properties or measures derived from them, which provide different ways to evaluate the underlying diffusion. The frequently used measures in literature include but not limit to: Mean Diffusivity (MD), Apparent Diffusion Coefficient (ADC), Volume Ratio (VR), Fractional Anisotropy (FA), Rational Anisotropy (RA), General Anisotropy (GA), Linear anisotropy (CL), Planar anisotropy (CP), Spherical anisotropy (CS), Trace (TR), Mode (M), etc. Their full meanings and definitions can be found in [11, 12]. Here, we only list parts of them for later references.

3 Related Work

This paper mainly concerns the visualization of tractography data that consist of fiber tracts, tensors and associated measures. Currently, these data are almost processed independently in literature.

For fiber tracts, it is simple and intuitive to render them as curved lines or thin tubes. Lines or tubes are then colored to display segment orientations or tensor eigenvectors [9, 10, 13]. By this way, global view of connectivity is disclosed, but information conveyed by fiber colors is rather limited because only one color mapping could be used.

For diffusion tensors, glyphs are widely used to display their information. Pierpaoli et al. first used ellipsoids to show diffusion tensors [11]. In their method, ellipsoid’s size and shape are determined by three eigenvectors and their corresponding eigenvalues. Ellipsoidal glyph has six degrees of freedom in shape and orientation that just fit in the six parameters of diffusion tensors. It seems perfect, but ellipsoid may cause visual ambiguity since some of them may have similar image-space shapes for different tensors [14]. Boxes and cylinders are also used to visualize diffusion tensors. While cylinders have the problem of discontinuity in shape, cuboids may be confused in showing orientations. To deal with these issues, Kindlmann proposed superquadric tensor glyphs which provide better performance in representing tensor shape and orientation [14]. Recently, Chitiboi et al. proposed 3D superquadric glyphs to visualize myocardial motion [15]. Zhang et al. proposed Tender glyphs for the purpose of comparing orientation and shape differences between tensors [16]. To deal with general tensors, Gerrits et al. constructed a new kind of glyphs based on piecewise rational curves and surfaces [17]. One of them is the flat-shaped glyph with eigenstick.

Reviewing existing tensor glyphs, we can find that most of them convey information through visual geometric properties that mainly include the size and shape of glyphs. As we know, however, the space of geometric properties is rather limited for small three-dimensional objects. For this reason, most of them do not take tensor measures into consideration. For example, ellipsoid can encode three eigenvectors and three eigenvalues. But there is no space for it to hold additional tensor measures except for a scalar color mapping on the surface. As mentioned in Sect. 2, tensor measures play important roles in understanding diffusion weighted imaging. The limited capacity of existing glyphs would be a drawback for current tractography visualization to some extent.

4 Our Approach

For general tractography data, it would be hard to show tracts, tensors, and scalar measures together to provide a full view. When turning to UKF tractography data, we find there is an opportunity to do that. The opportunity lies in the fact that each tensor in UKF tractography has only two different eigenvalues \( \lambda_{1} \) and \( \lambda_{2} \), leaving a degree of freedom that is often occupied by the third eigenvalue \( \lambda_{3} \). The released freedom can be used to encode other tensor information. With this motivation, we designed a new tensor glyph, and consequently proposed a new visualization scheme for UKF tractography data.

4.1 Glyph Design

The key of our approach is the glyph design. The new glyph must make full use of the freedom released by \( \lambda_{3} \). If we simply replace \( \lambda_{3} \) with another scalar measure (for example, FA), we could get nothing but a strange object that wastes the valuable degree of freedom. As an alternative, we remove the \( \lambda_{3} \) from the shape control of glyph, and take a cylinder to represent a tensor, as shown in Fig. 1(a), where the height and diameter of the cylinder are determined by eigenvalues \( \lambda_{1} \) and \( \lambda_{2} \), respectively, and the orientation of the cylinder is aligned with the major eigenvector \( e_{1} \). Here, it should be noted that the other two eigenvectors \( e_{2} \) and \( e_{3} \) are definitely existed and different although their eigenvalues are the same. The eigenvectors \( e_{1} \), \( e_{2} \), and \( e_{3} \) build the eigensystem. The cylinder glyph is centered at the eigensystem with the axis of rotation aligned to \( e_{1} \). Due to the round cross-section of cylinder, we can not distinguish the directions of \( e_{2} \) and \( e_{3} \) from the shape of cylinder. We only know that they are in the same plane and perpendicular to each other. To remove this ambiguity, we assume that \( e_{1} \) coincides with the axis Z of Cartesian coordinate system OXYZ, and \( e_{2} \) points to the positive axis X, as shown in Fig. 1(b). In the latter color scheme, \( e_{2} \) is further indicated by a special color hue, as shown in Fig. 4(a).

Fig. 1.
figure 1

Cylinder glyph encoding essential tensor information.

Full size image

By now, we have encoded essential tensor information using the geometric properties of cylinder glyph. When considering tensor measures, we can color the faces of the glyph. For example, color the lateral surface using FA, or color the top or the bottom using other measures. By this way, the cylinder glyph can convey three additional kinds of information, yet it is up to three at most there. Obviously, it is not enough for real applications, because there are dozens of measures that have defined for tensors. In order to break the limitation, we change the cylinder to a new one by cutting the lateral surface into rectangles. Here, we assume that the cutting starts from \( e_{2} \), and \( e_{2} \) is the normal of the first rectangle. As shown in Fig. 2, the cylinder is cut into a hexahedron, which has the capacity to hold more measures using its additional six lateral faces.

Fig. 2.
figure 2

Hexahedron glyph that not only encodes essential tensor information, but also have additional faces to hold more tensor measures.

Full size image

By changing the number of lateral faces τ, we can get a serial of similar glyphs such as prism, cuboid, pentagon, hexagon, heptagon, octagon, etc. For convenience, we refer to them as cylingons in this paper, because they are all polyhedral glyphs derived from cylinders. The parameter τ can be set according to real requirements. In general, it should be not less than 3, otherwise the glyph would collapse. On the other hand, if τ is too big, it would be hard to distinguish the lateral faces. Therefore, the proper setting for τ in practice would be from 3 to 10.

4.2 Visualization Scheme

Cylingon glyph integrates essential tensor information as well as derived measures. However, a glyph can only provide the local information about the underlying tensor. To learn more about the tensor field, we have to place lots of glyphs. Placing glyph is a challenge problem due to visual cluttering. In this paper, we take a relatively easy way to build global tensor view. As described in [18], we attach glyphs to fiber tracts according to a predefined down sampling rate. For a fiber tract, as shown in Fig. 3, cylingon glyphs are evenly placed along the fiber. Here the space of glyphs α is measured in samples, and can be adjusted as desired. By this way, fiber tracts, diffusion tensors, and derived measures are integrated and visualized together, providing local and global information about the underlying white matter. This is the main framework of our visualization scheme for UKF tractography data.

Fig. 3.
figure 3

Cylingon glyphs evenly placed along a fiber tract.

Full size image

In the framework, fiber tracts are rendered as usual, whose colors are set according to the orientation of line segments.

For cylingon glyph, we need to design a color scheme for faces to encode derived measures. When the parameter τ is determined, each face can present a tensor measure using its color property. We can set the color mapping to any desired results. But, we had better select colors that are easy to distinguish, because measures attached to faces would have completely different meanings. For this consideration, we provide a default color scheme for cylingon glyph based on the HSV (Hue-Saturation-Value) color model [13]. Firstly, we assign a unique label for each lateral face. As shown in Fig. 4(a), starting from \( e_{2} \) in the top face, we name the lateral faces in the clockwise \( f_{1} \), \( f_{2} \), \( f_{3} \), etc. Then we superimpose the color wheel onto the top face, and sample hues for each lateral face according to the rule: the hue perpendicular to the face is set for that face. For example, in Fig. 4(a), the color hue indicated by \( e_{2} \) is set for the right face \( f_{1} \). Moreover, we assign a fixed full value to the saturation, and set the third color component value proportionally varying with the scalar measure assigned for the face. By this simple but intuitive way, derived tensor measures are attached to lateral faces.

Fig. 4.
figure 4

Default color scheme for cylingon glyph.

Full size image

Besides lateral faces, the top and bottom faces can also be used to encode information through color mapping. In our default color scheme, however, we simply fill the top face with triangular color patches that coincides with lateral faces, and keep the bottom face black, as shown in Fig. 4(b). The first purpose is to make the direction of major eigenvector clear, which points from the bottom to the top. The second is to provide an opportunity to view all measures on the top face, making up for the three dimensional occlusion to some extent.

In summary, the visualization scheme consists of the following parts:

  • Glyph shape coding that encodes essential tensor information;

  • Glyph color mapping that associates multiple tensor measures with glyphs;

  • Fiber color mapping that shows the orientation of line segments as usual;

  • Linkage of tensor glyphs and fiber tracts that provides integrated global and local views.

4.3 Interactive Analysis

Diffusion tensors and fiber tracts are all defined in 3D space, and projected to 2D view plane. To compensate the information loss caused by the projection, user interactions are necessary. Translation, rotation, and zooming in or out, are general interactions to help view 3D objects. Besides these, we provide more ways to analyze tractography data interactively.

Firstly, cylingon glyphs can rotate around their major eigenvectors under user control. The goal of carrying derived measures is to compare them in field. If only a part of glyph surfaces can be viewed, the comparison would be affected. The glyph rotation enables to view full information interactively. This further relieves the effect of occlusion. Secondly, users can control the visibility of objects to build different views. For UKF tractography data, there are fiber tracts, Tensor 1 (\( D_{1} \)), and Tensor 2 (\( D_{2} \)). We can choose to display any one or more of them. In addition, we can select parts of fiber tracts and their associated glyphs to be viewed. Finally, the cylingon glyph can be customized by changing the parameters and options: the lateral face number τ, the space distance α, the derived tensor measures assigned to lateral faces, and the color mapping scheme for lateral faces.

All of these user interactions improve the performance of visualization, and make it more feasible and valuable to applications.

5 Applications

With the visualization scheme and user interactions, we can do most of visual analysis for UKF tractography data. Here we only provide two examples of tests and applications due to the paper length.

5.1 Phantom Data

Our approach was firstly tested with a phantom data that was made from a helix flow field according to the method in [19]. Besides basic diffusion tensors, the dataset obtained from the UKF tractography with the phantom data include the following derived measures: Trace (TR), Volume Ratio (VR), Fractional Anisotropy (FA), General Anisotropy (GA), Normalized Mean Square Error (NMSE).

To visualize this dataset, we set the parameter τ = 5, and assigned the measures TR, VR, FA, GA, and NMSE to glyph faces \( f_{1} \), \( f_{2} \), \( f_{3} \), \( f_{4} \), and \( f_{5} \) in accordance with the order. The fiber color mapping and the glyph color mapping were set to default. When changing the visibility of Tensor 1 and Tensor 2, we obtained three different views, as shown in Fig. 5. On this basis, we can further observe and compare them by selecting fibers, rotating glyphs, adjusting space, zooming, and changing view points. Such examples are given in Fig. 6, where a fiber selected from Fig. 5(a) and its associated glyphs for Tensor 1 are displayed with different settings. In Fig. 6(a), glyphs are displayed with a small space to build a tube style view. In Fig. 6(b), the glyphs are rotated around their rotation axis to view the hidden faces. Figure 6(c) shows the relationship of Tensor 1 and Tensor 2 clearly.

Fig. 5.
figure 5

Visualized results of the phantom dataset.

Full size image
Fig. 6.
figure 6

Visualized results in different views for the phantom dataset.

Full size image

5.2 Real Data

The basic usage of our approach is demonstrated with the phantom dataset. In this section, we provide a real application that involves the public DWI sample dataset available from 3D Slicer [20]. There are 42 gradients in the sample dataset, and the b value is 1000. The voxel size is 1.5 mm * 1.5 mm * 1.5 mm, and the data size is 128 * 128 * 94. By seeding in corpus callosum and running UKF tractography algorithm, we got a collect of tractography data that included fiber tracts, tensors, and derived measures. For this example, concerned tensor measures were: Trace (TR), Fractional Anisotropy (FA), General Anisotropy (GA), Normalized Mean Square Error (NMSE), Linear anisotropy (CL), Spherical anisotropy (CS). So the cylingon parameter τ was set to 6, and the above measures were assign to faces \( f_{1} \), \( f_{2} \), \( f_{3} \), \( f_{4} \), \( f_{5} \) and \( f_{6} \) in accordance with the order.

Similar to the phantom data, we can get various views of the dataset by changing the visualization settings. Some of them are shown in Fig. 7, where we can view fibers, tensor glyphs, and tensor measures attached to glyphs. With the help of these views, some interesting findings about the tractography can be observed. For example, as shown in Figs. 7(a), (b) and (d), Tensor 1 and Tensor 2 almost coincide with each other in the middle part of corpus callosum, while they are crossing with wide-angle in both end sides. In addition, some glyphs in the middle part are much smaller than their neighbors. In Fig. 7(c), the glyphs in the middle have higher CL values (shown in purple colors) than those near the ends. Both Figs. 7(d) and (e) have a glyph with bright yellow, meaning higher FA values, but they occur at different parts. Such trends and findings disclosed by the visualization give us cues to further investigate the tracotography. Although the investigation of such things would depend on specialists in neurosciences and clinical applications, they proved the value and significance of our visualization.

Fig. 7.
figure 7

Visualized results for the DWI dataset (Color figure online)

Full size image

6 Conclusions

General tractography data include fiber tracts, diffusion tensors, and derived tensor measures. These data are often visualized individually due to the limitation of tensor glyphs. In order to get more insight into UKF tractography data, which are generated by UKF tractography with the simplified two-tensor model, we proposed a joint visualization scheme in this paper. In the scheme, fiber tracts, diffusion tensors, and multiple tensor measures are integrated and visualized at the same time. The key of the visualization is the newly designed tensor glyph: cylingon, which is similar to cylinder, but removes the ambiguity of orientation and have more lateral faces to hold multiple scalar measures through color mapping. With the joint visualization scheme, we can explore UKF tracrography data efficiently through various user settings, observing and disclosing what are hidden in the dataset, just as initial applications demonstrated. Currently, our approach is limited to UKF tractography data that use simplified tensor models. It would be a future work to extend the approach to general tractography data. In addition, the application of the approach is worthy of further exploration, especially working with domain specialists to obtain further understanding of white matter in brain.