Combine harvester remote monitoring system based on multi-source information fusion

https://doi.org/10.1016/j.compag.2022.106771Get rights and content

Highlights

Abstract

Combine harvesters are prone to blockage, belt burnout and maintenance problems due to their complex transmission structure and variable operating environment. Therefore, a remote monitoring system of combine harvesters based on multi-source information fusion was designed, which could not only realize effective monitoring of combine harvesters, but also realize the functions of fault diagnosis and remote dispatching guidance. By analyzing the working principle and fault mechanism of combine harvester, a fault diagnosis algorithm based on speed fusion index, component slip rate and adaptive threshold discrimination was proposed. Users could obtain the real-time operation status and fault records of the combine harvester anytime and anywhere through the browser. The performance of the combine harvester remote monitoring system was verified through simulation tests and indoor tests. The test results showed that the system met the requirements of combine harvester remote monitoring, and the accurate recognition rate of combine harvester working condition is 97.46%, which has the advantages of high judgment accuracy, fast recognition speed and robustness.

Introduction

The development of remote monitoring of agricultural machinery is an important direction for the transformation and upgrading of agricultural machinery and equipment. While promoting the mechanization of the whole process of agricultural production, the realization of digital, informatzed digitalization, information technology, intelligent management of agricultural production and operation and intelligent agricultural production and management through information technology equipment to achieve is of great significance to improve the quality and efficiency of agricultural development and the construction of modern agriculture. (Kamilaris and Prenafeta-Boldu, 2018, Ding et al., 2018, Wei et al., 2017). Combine harvester is widely used as an important equipment of harvesting agricultural machinery, and its health is directly related to the efficiency and quality of harvesting. Due to the complex working environment and mechanical structure of combine harvester, the untimely maintenance of combine harvesters during the operation leads to frequent shutdown during the operation, which seriouslyaffects the harvesting efficiency (Fountas et al., 2015, Ylmaz and Gkduman, 2020, Shang et al., 2020). In order to ensure the performance and stability of combine harvester and prolong its service life, domestic and foreign scholars have conducted extensive research on real-time condition monitor and fault diagnosis of combine harvester.

General fault diagnosis methods are divided into model-based method, signal-based method, knowledge-based method and composite method (Lei et al., 2014). The traditional fault diagnosis method includes three different steps: signal acquisition, feature extraction and fault classification. (Gao et al., 2015, Zhang et al., 2018). Due to the reliance on operational skills and experience, it is ambiguous to use manual monitoring for combine harvesters with complex structure and variable working conditions.

In recent years, machine learning method has attracted much attention of scholars because it can stack multiple-layer nonlinear information processing modules into a hierarchical structure to stimulate the high-level representation behind the data, and has been successfully applied in the field of mechanical fault monitoring and diagnosis (Tang et al., 2019, Lei et al., 2015, Chan et al., 2018). Wattanajitsiri et al. (2020) used FMEA technology to assess the risk of key components of combine harvester, and indicate the causes and effects of faults. The risk priority number was rated and preventive maintenance strategies were proposed. Xi et al. (2020) proposed a method (SDAE-BP) of monitoring and diagnosing the operation faults of combine harvester based on the fusion of Stack Denoising Auto Encoder (SDAE) and Back Propagation neural network (BP), which realized the diagnosis of common blockage faults in combine harvesters. Wu et al. (2021). proposed a convolutional neural network fault diagnosis method based on improved LeNet-5, for the incompleteness of rolling bearing fault samples. Xiao et al. (2020) proposed a multi-group co-evolutionary particle swarm optimization BP neural network method for diesel engine fault diagnosis in view of the complexity, the correlation and the concurrency of high-power tractor diesel engine faults. Dong et al. (2018) developed an expert system for agricultural machinery fault diagnosis and safety assessment based on antecedent reasoning.

It is obvious that the needs of collaborative agricultural machinery operation cannot be met by relying on manual experience to achieve farm machinery scheduling at this stage. For remote monitoring of agricultural machine, Xie et al. (2020) developed a remote monitoring system for corn sowing parameters based on Android and wireless communication to solve the limitation of monitoring distance and transmission method of corn sowing monitoring device and realized remote and synchronous monitoring of corn sowing parameters for different distances. In order to realize remote monitoring and management of cotton pickers, Wang et al. (2019) designed a wireless sensor flow monitoring device based on laser and sensor, and proposed a method of image data processing. The tracking and production statistics of cotton pickers were realized for remote management, scheduling, and visual imaging. Remote monitoring has also been widely used in biological health monitoring, solar equipment monitoring, and farmland remote monitoring (Ma et al., 2021, Cheragee et al., 2021, Yin et al., 2017). At present, it is an important development direction of machinery monitoring to use monitoring data to assess machine health status and condition-based maintenance. Zhao et al. (2019), reviewed and summarized the emerging research work of deep learning in machinery health monitoring, and proposed a new trend of machinery health monitoring methods based on deep learning. Zhong et al. (2021) proposed statistical modeling and statistical analysis of the normalized squared envelope spectrum to construct a nonparametric health index and its associated statistical threshold of significance level for machine condition monitoring.

The problem of low fault diagnosis accuracy of combine harvesters can be solved by high precision fault diagnosis models, but the harsh operating environment of combine harvesters increases the difficulty of collecting large amount of quality data required for model training. Secondly, the local monitoring system for combine harvesters can not realize efficient dispatching, repair and maintenance of vehicles during cluster operations. To solve the above problems, a remote monitoring system of combine harvester with multi-source information fusion was proposed in this paper. The system used CAN bus to read combine engine parameters, collected combine component speed, top engine temperature and other related parameter information through sensor network, and sent the collected data to the industrial computer after pre-processing. The industrial computer sent the combine harvester operation information to the cloud server through the integrated terminal with built-in 4G DTU module, which realized the functions of combine harvester status monitoring, fault inquiry and maintenance information inquiry. Users can access the remote monitoring system through mobile phone and computer browser, so as to effectively improve the operation and maintenance efficiency of combine harvester.

Section snippets

The architecture of the system

The architecture of combine harvester remote monitoring system adopted the classic three-layer structure of IoT: perception layer, network layer and application layer. The perception layer is mainly responsible for the collection and pre-processing of combine information, the network layer mainly completes the information transmission, and the application layer is mainly responsible for the display of combine harvester operation status information, fault diagnosis and operation and maintenance

Fault diagnosis test of harvester

Since the wheat harvesting period was missed during the design of this system, a large field experiment could not be conducted to test the accuracy of this system identification. Data originating from a test field collected in June 2020 in Matun Town, Mengjin County, Luoyang City, were used as data input to validate the combine harvester adaptive fault monitoring method.Fig. 5 shows the picture of pre-test commissioning work for data acquisition. During the data acquisition test, the protective

Discussion

When the combine harvester is harvesting, the operator needs to adjust the parameters such as harvesting speed and fan speed according to different operating conditions. With the emergence of the combine harvester fault alarm system, operators have technical support to operate the combine harvester. However, the traditional combine harvester fault alarm system diagnoses the fault by setting the speed threshold, which has the problem of low fault judgment accuracy, and will affect the judgment

Conclusion

A remote monitoring system of combine harvester based on multi-source information fusion was designed in this paper to realize the fault diagnosis and remote operation monitoring of combine harvester by collecting the information of combine harvester engine, sensors and positioning, and fusing the proposed adaptive fault diagnosis algorithm and edge computing concept.

  • (1)

    The fault diagnosis algorithm of combine harvester was proposed through organically combining the speed fusion index and slip

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

The work was sponsored by the Intelligent Manufacturing Comprehensive Standardization Project (No. 2018GXZ1101011), the National Key Research and Development Program of China Sub-project (No. 2016YFD0701802), the Natural Science Foundation of Henan (No. 202300410124).

References (24)

  • Kamilaris, A., Prenafeta-Boldu, F.X., 2018. Deep learning in agric.: a survey. Comput. Electron. Agric. 147....
  • Lei, Y.G., Jia, F., Zhou, X., Lin, J., 2015. A deep learning-based method for machinery health monitoring with big...
  • Cited by (13)

    • Impurity monitoring study for corn kernel harvesting based on machine vision and CPU-Net

      2022, Computers and Electronics in Agriculture
      Citation Excerpt :

      Corn is the most productive grain crop in the world and consists of three main types of harvesting: ear harvesting, ear and stalk harvesting, and kernel harvesting (Zhao et al., 2022; Cerdeira-Pena et al., 2017; Li et al., 2022). The kernel harvesting method can complete a series of work, such as picking, threshing, separation, cleaning, and returning to the field, with high efficiency, a short harvesting period, and low-cost advantages (Qiu et al., 2022; Ma et al., 2022). This method has become the main development direction of corn harvesting.

    • Design of feed rate monitoring system and estimation method for yield distribution information on combine harvester

      2022, Computers and Electronics in Agriculture
      Citation Excerpt :

      In addition, accurately grasping the distribution information of crop yield along the field is of great significance for guiding agricultural production management to increase crop yield, reduce the input of chemical fertilizers and pesticides, and improve farmers’ income (Bramley and Ouzman, 2018). The methods for obtaining information on crop yield distribution can mainly be divided into two categories: one is to predict crop yield through remote sensing, soil analysis, and other methods during the growing period of crops (Klompenburg et al., 2020; Dobermann and Ping, 2004; Mu et al., 2019; Shahrin et al., 2020; Mariano and Monica, 2021); the other one is to install sensors on the harvester to monitor crop yield, moisture content, and other information in real time during the crop harvest period and match the harvester positional information to generate a yield distribution map (Brusa et al., 2018; Gerstmann et al., 2016; Wang et al., 2017; Qiu et al., 2022; Yin et al., 2021; Liu et al., 2022). The first type of methods chooses to predict the yield at the stage when the crop is not fully mature.

    View all citing articles on Scopus
    View full text