In the past decades, the removal of the semi-solid metal oxide slag has raised a significant concern in industrial manufacturing field. The presence of oxide slag may lead to a decrease in the quality of metal products, affecting their mechanical properties and visual appearance, which causes serious quality problems in manufacturing industry. Traditional method removes the oxide slag without concerning the actual depth of the slag, which leads to poor removal effectiveness and unnecessary waste of the products. This article proposed a novel method to improve the removal accuracy and efficiency. In this work, statistical features are obtained using the Gabor transformation and the Johnson SB distribution. The statistical features and temporal correlations features are fused, and a self-attention mechanism is applied to reduce redundancy. Joint learning network is used to effectively identify the oxidation slag thickness based on the fused features. To validate the efficiency of the proposed method, a removal experiment is carried out to compare the accuracy of the proposed method and the traditional method, by removing the oxide slag of a 500 mm ingot sample. As a result, the proposed method shows a significant improvement on the accuracy by reducing the traditional removal error from 6% to less than 1%. The experiment has been repeated 100 times to remove the accidental error.