Framework for a closed-loop cooperative human Cyber-Physical System for the mining industry driven by VR and AR: MHCPS
Section snippets
Sources of problems and literature review
CPS are multi-dimensional and complex systems that integrate computing, networks, and the physical environment (Sonkoly et al., 2020). Using organic integration and deep collaboration of 3C (computing, communication, and control) technologies, the real-time perception, dynamic control, and information services of some large-scale engineering systems has been realized (Shahin et al., 2020). Proposed as early as 2005, CPS did not have a significant effect at the time due to technical limitations (
Introduction of mining work-face
Automatic mining work-faces, located in underground coal seams 200–1000 m below the ground, are mainly composed of shearers, scraper conveyors, and hydraulic supports. They operate under conditions where coal seams exhibit frequent and irregular fluctuations. The shearer cuts coal onto the scraper conveyor which transports the falling coal out of the mine. The hydraulic support groups support the roof of the mine as a whole to ensure the safety of operators. Limited by cost and technological
Overall operating framework
We adopted the concept of a human–machine integrated system to construct CPS operating framework. The goal was to form a new intelligent mining complex with high levels of integration among human beings, computing systems, and physical system in the network environment utilizing the HCPS concept. Mining systems are made more dependable and efficient through the integrated design of computing, communication, and physical systems.
The traditional HCPS includes human beings, information systems,
Three key technologies and system implementation
The information space includes the VR and AR systems and represents the precise synchronization and modeling of all elements and individuals in physical space as the basis of the HCPS. To achieve this, there are four basic requirements. (1) The physical equipment should be contain a certain intelligence level; (2) the VR system needs to construct high-precision simulated virtual image, and its intrinsic knowledge model must be enriched; (3) the AR system must obtain additional sensory
3C flow directions and fusion methods
After explaining the key technologies and establishing the relationship between operators and the three systems, the issues present among the 3C components must then be explained. These interconnected relationships are shown by the connecting arrows in Fig. 3.
The goal of a 3C operation is the seamless integration of humans with 3C, the achievement of which has three main requirements. (1) A communication platform must be used to ensure perfect information interactions between modules. (2) Based
Prototype system design
The prototype system was designed in the laboratory and the functions of the system were assessed based on the communication network layout. The attitude measurement sensors and Ultra-Wide Band (UWB) positioning base station were installed on hydraulic support groups. The inertial navigation system (INS) was arranged on the shearer and the attitude data was transmitted to the industrial computer of the control center via wireless transmission. The distributed VR monitoring system was installed
Conclusion
- (1)
The MHCPS is proposed for complex underground mining working conditions to meet the demand for automatic operations and serve as the foundation for the design of a CPS operation framework.
- (2)
In this large framework, AR and VR technologies are fused in real time to make up an information system that drives the 3C system and forms a closed information loop. The system facilitates the perfect fusion of virtual simulation information, real-time sensing information, and visual information, all to
Funding
This work was supported by the National Natural Science Foundation of China [grant number 52004174], the Fund for ShanXi “1331” Project, Key project of the Chinese Society of Academic Degrees and Graduate Education [grant number 2020ZDA12], Key Research and Development Program of Shanxi [grant number 201903D121141], Natural Science Foundation of Shanxi Province [grant number 201901D211022] and Scientific and Technological Innovation Programs of Higher Education Institutions in ShanXi [grant
CRediT authorship contribution statement
Jiacheng Xie: Writing – original draft, Methodology, Software. Shuguang Liu: Data curation, Visualization, Software, Investigation. Xuewen Wang: Conceptualization, Supervision, Validation.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
References (48)
- et al.
Digital Twin as a Service (DTaaS) in Industry 4.0: An Architecture Reference Model
Advanced Engineering Informatics
(2021) - et al.
Using augmented reality to build digital twin for reconfigurable additive manufacturing system
Journal of Manufacturing Systems
(2020) - et al.
Augmented reality in support of Industry 4.0-Implementation challenges and success factors
Robotics & Computer Integrated Manufacturing
(2019) - et al.
Multi robots interactive control using mixed reality
IFAC-PapersOnLine
(2019) - et al.
Human–Cyber–Physical Systems (HCPSs) in the Context of New-Generation Intelligent Manufacturing
Engineering
(2019) - et al.
Cyber Physical and Social Networks in IoV (CPSN-IoV): A Multimodal Architecture in Edge-Based Networks for Optimal Route Selection Using 5G Technologies
IEEE Access
(2020) - Bataille G,Gouranton V,Lacoche J,Pelé D,Arnaldi B. (2020) A Unified Design & Development Framework for Mixed...
- et al.
Synchronizing Digital Process Twins between virtual products and resources – A virtual design method
Procedia CIRP
(2019) - et al.
Evaluating Centralized and Heterarchical Control of Smart Manufacturing Systems in the Era of Industry 4.0
Applied Sciences
(2020) - et al.
Cyber-Physical Ubiquitous Cycling With Fuzzy-Controlled Panorama Manifestation Based on Internet of Things Technologies
IEEE Sensors Journal
(2020)