Keywords

1 Introduction

In past decades, high precise and high realistic three-dimensional crop canopy models were becoming more important in many applications, such as scientific researches in agriculture, cultivars estimation and phenotype analysis, virtual technological training, etc. [1,2,3,4,5,6,7,8]. One of the most important applications of 3D quantitative virtual models is calculation of light distribution within a crop canopy [6, 7]. Wheat is a main crop both in China and whole world, and many researchers have made fruitful works about cultivars evaluation, phenotype analysis and planting technologies [8,9,10,11,12,13].

The main problem in 3D virtual canopy model applications is reconstruction, in both precision and efficiency. There are several effective reconstruction ways: measure-based, 3D digitizer, 3D scanning and vision-based technologies. Each method has its advantages and limitations. For example, traditional measure-based way is generally low efficient and precise, but it could be used in more widely scene without complex influences. Nowadays, with the fast development of 3D laser scanning technology and popularizing of scanning devices, people have made use of scanning to reconstruct crops’ canopies models. Hosoi and Omasa [13] estimated vertical plant area density and analyzed growth parameters of a wheat canopy by using 3D portable lidar imaging. LiDAR is becoming a new important technology and tool in next-generation plant phenotyping applications [14]. Multi-sensor systems have been also developed for high throughput field phenotyping, including video camera, NDVI, distance sensor, infrared radiometers, and so on [15]. Benefit from that, image-based 3D canopy reconstruction technology has been used in many applications, such as potential productivity [16].

To sum up, the main problems of 3D canopy modeling and analysis in applications is twofold. One is how to reconstruct precise 3D canopy models. Another is how to make use of virtual models to analyze the features and differences between cultivars. The first problem is more important than the second one, and it is the basic condition for any applications. Especially for wheat, because of its complex structures of canopy, precise reconstruction and modeling with all 3D models of organ details are still challenges. Existing works have only focus on large scale simulation and approximative simplified morphologic modeling for organ details. To tackle the problems, we present a 3D canopy reconstruction and phenotype analysis method for wheat in this paper. Figure 1 shows the flowchart of this paper. First, we acquire multi-type 3D structure, shape and distribution data and 2D images, as well as synchronous light intensity and leaf area index (LAI). And then, we reconstruct canopy models of wheat based on processed data. By use of canopy models, we conduct phenotype analysis to calculate light distribution and parameters.

Fig. 1.
figure 1

The flowchart of this method

2 Materials and Method

We have planted two varieties of wheat with two densities: Jingdong18 Jing8043, in our experimental farm in Beijing Academy Agriculture and Forest Science located at (39°57′52″N, 116°17′84″E). The line distance is 300 mm, and the sowing date is Oct 30, 2016. In our experimenting field, we have installed a group of cameras for wheat growth status monitoring. The images are automatically sent to service machine and image collecting frequency is 1 per hour (Fig. 2).

Fig. 2.
figure 2

Wheat planting in experiment field

2.1 Data Acquisition

Here, our 3D reconstruction method depends on real 3D data of shape, structure and distribution of wheat organs in the canopy. So we make use a 3D digitizer named FastRak produced by Polhemus Company to obtain feature points of wheat plants in colony. For leaf shape, we use an image-based way to measure leaf width. On the other hand, we also acquire real light intensity and LAI values by a PAR device to examine reconstruction results.

For 3D digitization, we have picked up all feature points on wheat plant, including all node joints of stalks, and all leaf nervation curve feature points. Generally, we took 5 to 8 feature points on a leaf mid curve, in turn from base to tip. The FastRak 3D digitizer is shown in Fig. 3. Figure 4(a) shows a scene of data collecting and an example of captured frame data are shown in Fig. 4(b). For each blocks, we selected 3–5 plants including 10–30 tillers. By 3D digitizer, we collected the real shape, structure and organ distribution information.

Fig. 3.
figure 3

Wheat planting in experiment field

Fig. 4.
figure 4

3D data collecting and example data

To obtain precise leaf width parameters, we captured images of all leaves fixed on a plotting paper, and then we measured values of leaf width and fitted a leaf width value sequence for each leaf. We have investigated the leaf width values of different plants in a same canopy, and the results have shown that the differences of values with same leaf order between different plants were very small. So we assumed the distribution of leaf width value was uniform and chose a fixed leaf width function to generate parameters according to leaf length values (Fig. 5).

Fig. 5.
figure 5

Images of wheat plant and group of leaves.

2.2 Three-Dimensional Canopy Reconstruction

Based on the multi-type acquired data, we processed and got key parameters of 3D models of wheat organs including stalks, leaves and ears. We presented a multi-scale and multi-layer 3D modeling method to reconstruct virtual wheat models. The organ modeling is shown in Fig. 6. For each organ’s feature points, we constructed a cubic spline interpolation curve (Eq. 1) to generate the smooth nervation of leaf. And then, we construct a local coordinate on each point of nervation, as shown in Fig. 6, and the local coordinate determines the deformation of corresponding point set both on two edges.

$$ \begin{aligned} S_{k} (x) & = H_{3}^{(k)} (x) = y_{k} \alpha_{0}^{(k)} (x) + y_{k + 1} \alpha_{1}^{(k)} (x) + m_{k} \beta_{0}^{(k)} (x) + m_{k + 1} \beta_{1}^{(k)} (x) \\ & = y_{k} (1 + 2\frac{{x - x_{k} }}{{x_{k + 1} - x_{k} }})(\frac{{x - x_{k + 1} }}{{x_{k} - x_{k + 1} }})^{2} + y_{k + 1} (1 + 2\frac{{x - x_{k + 1} }}{{x_{k} - x_{k + 1} }})(\frac{{x - x_{k} }}{{x_{k + 1} - x_{k} }})^{2} \\ & + \;m_{k} (x - x_{k} )(\frac{{x - x_{k + 1} }}{{x_{k} - x_{k + 1} }})^{2} + m_{k + 1} (x - x_{k + 1} )(\frac{{x - x_{k} }}{{x_{k + 1} - x_{k} }})^{2} \\ \end{aligned} $$
(1)
Fig. 6.
figure 6

Wheat leaf generation algorithm, (a) is the local coordinate on mid-point, (b) deformation.

We translate each row of control points to according to the middle point of them, and the translation can be expressed by Eqs. (2) and (3) where N is the new model.

$$ \left\{ {\begin{array}{*{20}c} {Trans_{i} = Vc[i] - Ve[i]} \\ {N_{i} = Trans_{i} \bullet E_{i}^{\prime } } \\ \end{array} } \right.\quad i = 1,2, \ldots ,n $$
(2)
$$ N = \left[ {\begin{array}{*{20}c} {N_{x} } \\ {N_{y} } \\ {N_{z} } \\ 1 \\ \end{array} } \right] = Trans \bullet E^{\prime } = \left[ {\begin{array}{*{20}c} 1 & 0 & 0 & {Trans_{x} } \\ 0 & 1 & 0 & {Trans_{y} } \\ 0 & 0 & 1 & {Trans_{z} } \\ 0 & 0 & 0 & 1 \\ \end{array} } \right] \bullet \left[ {\begin{array}{*{20}c} {E_{x}^{\prime } } \\ {E_{y}^{\prime } } \\ {E_{z}^{\prime } } \\ 1 \\ \end{array} } \right] $$
(3)

Finally, we recalculate the surface model according to Eq. (1) and a new leaf model is generated, as shown in Fig. 6. The same process is conducted on stalk and ear feature points, we use a simplified whole cylinder ear model to get a higher efficiency. Some modeling results are illustrated in Fig. 7, here (a) and (b) are frame and 3D models of a tiller of wheat respectively, (c) and (d) are small canopy’s frame and 3D models including 5 plants of wheat.

Fig. 7.
figure 7

3D reconstruction of a tiller of wheat and a small colony.

3 Experimental Results

We have conducted various experiments by different scene of wheat canopies. To implement our proposed method, we have developed a prototype system for wheat’s three-dimensional reconstruction and phenotype analysis and the system is named as “PlantCAD Wheat”. The software system consists of functional modules: 3D data view and processing, wheat organs shape reconstruction by free-form curves and surface, 3D canopy and scene generation and management, light distribution computing, canopy parameters phenotype analysis, and other additional modules such as 3D scene rendering, view operating, data I/O, and so on. To examine the proposed method, we have performed 4 canopies models on a DELL mobile workstation with i7 2.8 GHz CPU, 16 GB memory and NVIDIA Quadro K2100M graphics card.

3.1 Results of Three-Dimensional Reconstruction

By use of modeling and reconstruction algorithm introduced in Sect. 2.2, we analyze local features of acquired data and generate an assigned wheat colony by group copy both in row and line directions. Figure 8 shows an example of 3D reconstruction of wheat canopy with 5 rows including about 180 plants. We also add land and light sensor models into canopy to support additionally light distribution calculation. The visualization is implemented by OpenGL Library.

Fig. 8.
figure 8

3D reconstruction of wheat canopy with 5 rows including about 180 plants.

3.2 Results of Light Distribution Computing

Quantitative light distribution within a wheat canopy is very important for simulation of canopy photosynthesis, and precise 3D virtual models could support effective and accurate calculation of light distribution. We take a light distribution algorithm introduced in literatures [17] to calculate both direct sun light and scattering of sky. We also combine a visualization module based on OpenGL Library to render the light distribution results, as shown in Fig. 8(b).

4 Conclusions

In this paper, we have proposed a 3D canopy reconstruction and phenotype analysis method for wheat. By means of a set of multiple type real data measured from wheat canopy in field, we present a 3D reconstruction algorithm for all wheat organs including stalks, leaves and ear. And then, we construct canopy models by acquired frame feature points. Based on reconstructed 3D wheat canopy, we conduct light distribution calculation and phenotype characters calculation. The main contribution is that our data-based 3D reconstruction method provides a feasible and effective way for wheat canopy modeling and phenotype analysis. On the other hand, the main limitation of our method is data acquisition capacity and efficiency, because 3D feature points are manually collected for aim of high precision and completeness. So we have to make use a local distribution to express the expanded canopy. Involving 3D laser scanning and LiDAR technologies as well as high throughput phenotype analyzing approaches, automatically 3D canopy model reconstruction should be a potential trend for wheat and other crops in future.