Abstract
Drilling for gas extraction, a common method in coal mine gas control, involves tedious loading and uploading of drill pipes. This study aims to design a method for detecting and measuring pose drill pipes using point cloud data. We present an experimental platform for acquiring drill pipe point cloud data under various lights. Additionally, we propose a GA-PointNet + + model, enhanced with an adversarial generation network. The pose of the drill pipe was calculated from the segmented pipe and pin point clouds. Results indicate that the intersection-over-union (IoU) values for pipe and pin, based on GA-PointNet + + , are 0.824 and 0.472, respectively. Evaluating the model's performance in recognizing the pin using the ROC curve yielded an AUC of 0.87. The combination of GA-Pointnet + + and RGB-D camera was used to pose drill pipes, achieving an average accuracy of 82.5% under different lighting conditions. Under lighting conditions of 25–35 lx with an added diffuser film and 10–15 lx, the accuracy reaches 90%, with average distance errors of 1.4 cm and 2.5 cm, and average angle errors of 3.5° and 3.7°, respectively. This has significant implications for the use of LED lights in underground environments. Therefore, the proposed drill pipe pose measurement method is of great significance for the intelligentization of coal mine drilling operations.
Graphical Abstract
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
Data Availability Data will be made available on request.
Code availability
All code was implemented in Python using PyTorch as the primary deep-learning library. Code is available at https://github.com/caijinyu0609/GA-Point.
References
He S, Ou S, Lu Y, Jin L, Chen T, Ma Y (2022) Failure mechanism of methane drainage borehole in soft coal seams: Insights from simulation, theoretical analysis and in-borehole imaging. Process Saf Environ Prot 168:410–421
Wang F, Yang Y, Zhou J, Zhang W (2023) An Onboard Point Cloud Semantic Segmentation System for Robotic Platforms. Machines 11(5):571
Yang L, Ma H, Nie Z, Zhang H, Wang Z, Wang C (2023) 3D LiDAR Point Cloud Registration Based on IMU Preintegration in Coal Mine Roadways. Sensors 23(7):3473
Lin G, Tang Y, Zou X, Li J, Xiong J (2019) In-field citrus detection and localisation based on RGB-D image analysis. Biosys Eng 186:34–44
Huang Z, Wang Y, Wen J, Wang P, Cai X (2023) An object detection algorithm combining semantic and geometric information of the 3D point cloud. Adv Eng Inform 56:101971
Si L, Wang Z, Liu P, Tan C, Chen H, Wei D (2021) A Novel Coal-Rock Recognition Method for Coal Mining Working Face Based on Laser Point Cloud Data. IEEE Trans Instrum Meas 70:1–18
Xing Z, Zhao S, Guo W, Guo X, Wang S, Li M, Wang Y, He H (2023) Analyzing point cloud of coal mining process in much dust environment based on dynamic graph convolution neural network. Environ Sci Pollut Res Int 30(2):4044–4061
Xu F, Wang Z, Wang H, Lin L, Liang H (2023) Dynamic vehicle pose estimation and tracking based on motion feedback for LiDARs. Appl Intell 53(2):2362–2390
Yang Z, Zeng J, De La Cruz A (2023) Learning and outperforming the model predictive control with a linear ϵ-support vector machine for power converter. IEEE Trans Ind Electron 71(1):5–13
Jhaldiyal A, Chaudhary N (2023) Semantic segmentation of 3D LiDAR data using deep learning: a review of projection-based methods. Appl Intell 53(6):6844–6855
Mostafa G, Mahmoud H, Abd El-Hafeez T, ElAraby ME (2024) Feature reduction for hepatocellular carcinoma prediction using machine learning algorithms. J Big Data 11(1):88
Dutagaci H, Rasti P, Galopin G, Rousseau D (2020) ROSE-X: an annotated data set for evaluation of 3D plant organ segmentation methods. Plant Methods 16:28
Jing Y, Sheil B, Acikgoz S (2024) A lightweight Transformer-based neural network for large-scale masonry arch bridge point cloud segmentation. Comput Aided Civ Infrastruct Eng 39(16):2427–2438
Qi CR, Yi L, Su H, Guibas LJ (2017) Pointnet++: Deep hierarchical feature learning on point sets in a metric space. Adv Neural Inf Process Syst 30
Li M, Xie Y, Ma L (2022) Paying attention for adjacent areas: Learning discriminative features for large-scale 3D scene segmentation. Pattern Recogn 129:108722
Deng L, Yang M, Liang Z, He Y, Wang C (2020) Fusing geometrical and visual information via superpoints for the semantic segmentation of 3D road scenes. Tsinghua Sci Technol 25(4):498–507
Chen S, Miao Z, Chen H, Mukherjee M, Zhang Y (2023) Point-attention Net: a graph attention convolution network for point cloudsegmentation. Appl Intell 53(9):11344–11356
Wang Z, Zhang X, Ran C, Yu H, Wang S, Zhang Q, Zhou X (2024) Deep Learning-based Semantic Segmentation and Surface Reconstruction for Point Clouds of Offshore Oil Production Equipment. IEEE Trans Geosci Remote Sens 62(2024):1–18
Wei H, Xu E, Zhang J, Meng Y, Wei J, Dong Z, Li Z (2022) BushNet: Effective semantic segmentation of bush in large-scale point clouds. Comput Electron Agric 193:106653
Xing Z, Zhao S, Guo W, Meng F, Guo X, Wang S, He H (2023) Coal resources under carbon peak: Segmentation of massive laser point clouds for coal mining in underground dusty environments using integrated graph deep learning model. Energy 285:128771
Lu X, Li Y, Xie M (2023) Preliminary study for motion pose of inshore ships based on point cloud: Estimation of ship berthing angle. Measurement 214:112836
Lin YH, Wu TL, Yu CM, Wu IC (2023) Development of an intelligent underwater recognition system based on the deep reinforcement learning algorithm in an autonomous underwater vehicle. Measurement 214:112844
Liu X, Wang H, Chen X, Chen W, Xie Z (2022) Position Awareness Network for Noncooperative Spacecraft Pose Estimation Based on Point Cloud. IEEE Trans Aerosp Electron Syst 59(1):507–518
Li Y, He L, Jia J, Lv J, Chen J, Qiao X, Wu C (2021) In-field tea shoot detection and 3D localization using an RGB-D camera. Comput Electron Agric 185:106149
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, ... Bengio Y (2020) Generative adversarial networks. Commun ACM 63(11):139–144
Nguyen DP, Berg P, Debbabi B, Nguyen TN, Tran VD, Nguyen HQ, ... Dao TT (2023) Automatic part segmentation of facial anatomies using geometric deep learning toward a computer-aided facial rehabilitation. Eng Appl Artif Intell 119:105832
Lei H, Akhtar N, Mian A (2020) Seggcn: Efficient 3d point cloud segmentation with fuzzy spherical kernel. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition pp 11611–11620
Zhao H, Jiang L, Jia J, Torr PH, Koltun V (2021) Point transformer. In Proceedings of the IEEE/CVF international conference on computer vision pp 16259–16268
Elmessery WM, Maklakov DV, El-Messery TM, Baranenko DA, Gutiérrez J, Shams MY, ... Elwakeel AE (2024) Semantic segmentation of microbial alterations based on SegFormer. Front Plant Sci 15:1352935
Snoek J, Larochelle H, Adams RP (2012) Practical bayesian optimization of machine learning algorithms. Adv Neural Inf Process Syst 25
Funding
This research was supported by the Priority Academic Program Development of Jiangsu Higher Education Institutions (PAPD) and the National Natural Science Foundation of China (grant number 52304147).
Author information
Authors and Affiliations
Contributions
All authors contributed to the study conception and design. Jiangnan Luo: Conceptualization, Methodology, Writing review & editing, Software. Jinyu Cai: Investigation, Formal analysis, Visualization, Writing–original draft. Jianping Li: Supervision, Review manuscript. Deyi Zhang: Conceptualization, found. Jiuhua Gao: Manufacture drill pipes. Yuze Li, Liu Lei and Mengda Hao: Dataset collection and processing.
Corresponding author
Ethics declarations
Ethical and informed consent for data used
Not applicable.
Competing interests
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Luo, J., Cai, J., Li, J. et al. Detection and pose measurement of underground drill pipes based on GA-PointNet++ . Appl Intell 55, 132 (2025). https://doi.org/10.1007/s10489-024-05925-w
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05925-w