In the textile and garments industry, minimizing fabric defects and production downtime can be efficiently handled by proper fault detection and fault prediction. This research work presents an approach using sound-based analysis to detect fault in knitting machines, which is a vital machine for widely used knit fabric production. Sound data was captured using an electret condenser microphone, processed and recorded through Arduino Uno and CoolTerm Software. This sound data then converted into voltage-time graph images which were afterwards trained as a dataset on the VGG16 image classifier model and the csv format of sound data was trained with other classifiers such as SGD, AdaBoost, Bagging, Decision Tree, Random Forest, Transformer and others for detecting faults such as Faulty Needle or Faulty Sinker. The setup was tested on single jersey knitting machine in both faulty and healthy conditions and end result of the experiment lead to a remarkable 73.8% accuracy for needle or sinker fault detection. Comparing with other fault detection methods for textile machines such as vibration based, photo-electric based or mechanical based, our sound-based analysis system with image classifier machine learning model is proved to be both cost effective and of higher accuracy. This research depicts the potential of sound-based analysis for real time and automatic fault detection of knitting machines with the chance of improvement in production efficiency and wastage reduction.
1 Introduction
In the vast arena of the Textile and Garments industries across the world, Bangladesh stands as the second largest exporter of RMG world-wide contributing by earning about 29 billion US$ [8, 12]. In this growing sector the knitting industries plays an impactful position as the growth rate of knitwear export was about 12% [6]. As for the increase in demand of knitwear throughout the world, the demand and production of different knitting machines also increased and so did the faults occur in those machines.
One of the main challenges about knitting machines are the fault detecting. Major faults are likely to occur on the needles, sinkers, cam or cam-boxes which lead to fabric faults creating wastage and production loss [18]. Eliminating these faults by different means of research and applications have been done for ages. Controlling, mitigating or eradicating the knitting machine faults are done through laser detection, machine vision, mechanical means like needle detector, vibration-based fault prediction, sound frequency based statistical analysis and by many other means [4, 5, 17, 18].
In this research, we conduct fault detection of knitting machines by measuring sound and converting it to graphical image then analyze and train a dataset for detecting needle or sinker faults. Here an electret condenser microphone was used along with Arduino uno setup to measure and store the sound of the different faulty situation along with machine faultless condition as voltage value. Then this sound voltage value data was transformed as a voltage time graph image and a large dataset consisting about 1270 data was trained with VGG16 image classifier which led to detecting needle or sinker fault in a knitting machine with great accuracy. This experiment was done over single jersey knitting machine on both noisy and quiet environment for the further applicability of this research on industrial environment.
Our contributions in this research are at first measuring and collecting minimal amount of sound using the LM393 based sound sensor through the Arduino uno platform and an electronic device (laptop). After getting the sound value as voltage with help of CoolTerm the data was converted into voltage vs time graph. Each graph was counted as 7 second data. These data were taken for both quiet environment and noisy environment (all machine running along with people sound). Data were collected by creating real faults like sinker broken, needle latch broken, needle hook broken, bended sinker or bended needle along with data from fault free condition of the machine. The created dataset was trained with image classifier VGG16 machine learning model and successfully detected whether there was needle fault or sinker fault in the machine instantaneously with a 73.8% accuracy. This fault detection method has higher accuracy and is cost effective than about any other automated methods which use vibration, sound frequency or other means.
2 Background and Related Work
Circular knitting machines have complex design. Because of the design of circular knitting machines, flaws or any defects on the cloth are not visible as soon as they appear. Faults appear only after a length of cloth, approximately 50 to 60 cm long after is knit. This condition results in a loss of time and material for producers. Some article describes a technique for detecting defects that may arise during production on circular knitting machines [14]. Studies on the inspection of the faults in the woven fabrics, which are the most applied areas of image processing methods [11]. But in knitting machines this process is not that used. Again, Photoelectric fault detection systems are costly to implement in the textile sector. Furthermore, implementing these problem detection systems is hard and requires highly skilled staff [17]. We find that there is a change in the machine frequency and vibration respectively during different fault [5]. Altinors et al. proposed a sound-based fault detection system for UAV (Unmanned Aerial Vehicle) motors, including statistical feature extraction [2]. Zhou et al. suggested a sound sensor-based method for monitoring tool wear. However, only 138 samples were collected in this investigation [19]. Shiri et al. introduced an acoustic signal-based fault detection system for belt conveyor idlers. They created an inspection robotic UGV platform. However, the system is somewhat expensive for industry applications [13]. Lu et al. suggested a technique for detecting bearing faults using sound-aided vibrations and weak signals. However, the experiment was conducted on a single bearing, making the device unsuitable for Industrial machinery [9]. Bandara et al. suggested an automatic system for detecting fabric defects and machine faults. The system detects fabric problems using picture pre-processing and Neural Networks [3]. Zhang et al. suggested a highly accurate machine vision-based needle flaw detection method. However, this system requires a high-brightness linear auxiliary lighting source. This makes the technology unsuitable for use in a real-factory setting [18]. Nisha et al. developed a fabric defect detecting system using image pre-processing, feature extraction, and the multi-SVM method [10]. The discussed fault detection systems are effective, but only detects certain issues and lacks integration and value-added information. For example, the most commonly used system, the needle detector, can only detect closed latches or raised needles (stimulated by broken butt) [6]. Damaged sinkers, hooks, and needles are not detected. The technology for detecting flaws in knitted fabric has limitations due to machine speed and certain patterns that may go undetected. Many machines do not communicate with the production information system, especially when it comes to failures [6]. Implementing a single force sensor in each feeder presents a significant hurdle and also costly. So, in this project we took the different sound of knitting machine with sound sensor. Then we got graphical values and convert them into data set. A new transfer learning algorithm based on pre-trained VGG-19 (TranVGG-19) is suggested for defect diagnosis. First, a method for converting time-domain signals into RGB images is proposed. The pre-trained VGG-19 is then used as a feature extractor to acquire the features of the transformed images. Finally, a SoftMax classifier is trained using the features [16]. The time-domain vibration signal and the frequency domain stator current signal are combined to diagnose IDF (Irreversible Demagnetization Fault) and BF (Bearing Fault) using the 16-layer visual geometry group (VGG) architecture. The use of several signals makes the FDI (Fault Detection and Identification) more resilient. The transfer learning strategy is used to alleviate the problem of overfitting while training VGG-16 with a short dataset [15]. Furthermore, a novel data preprocessing technique is proposed that turns two-dimensional current and vibration signals into RGB images without knowing any parameters. This is a simple signal processing method that does not require advanced knowledge. The proposed model for the combination of current and vibration data is experimentally evaluated and compared to their individual use for fault classification [1, 7]. The main issues with the previously discussed methods are that they can only detect machine flaws after a defective output is produced. Furthermore, they do not provide details about which machine portion is defective. In contrast, our system can detect machine defects. For generating inaccurate output because it directly detects the machine’s fault. This paper focuses on image-based machine learning for textile inspection, with a focus on benchmarking techniques, defect types, performance, metrics, and datasets. It aims to add value to previous surveys.
Figure 1:
Figure 1: Illustration of our proposed methodology flow diagram
3 Proposed Methodology
In this section we will be discussing the method of approach for this research and explain the process flow sequentially along with the machine learning image-based classification. As for the methodology we used an electret condenser microphone module with an LM393 operational amplifier to collect sound data from knitting machine through the Arduino uno platform. The knitting machine was purposefully equipped with faulty parts such as broken latch needle, broken hook needle, broken butt needle, bent needle, broken sinker or bent sinker before taking the sound value. Sound value of the knitting machine in all okay condition was also taken with help of the sound sensor. The data got from the sensor was processed through Arduino and converted into digital voltage values which were recorded as txt or CSV file with the CoolTerm software. With the further collaboration of CoolTerm and Microsoft excel graph of voltage-time image was extracted, compared for distinct differences among faulty and faultless condition of the machine and created dataset was trained with VGG16 image classifier machine learning model. These procedures then facilitated with the needle fault or sinker fault detection of the machine. The illustration of our proposed methodology flow diagram is shown in Figure1.
3.1 Data Collection and Preprocessing
Here in the experiment, we used a microphone module to detect sound intensity. It has an electret microphone that picks up sound and an LM393 operational amplifier processes the sound signals and converts it into electrical signals. This sensor also has analog output that provides continuous signal proportional to the sound level to measure real time sound intensity. The sound electric signal then was read by Arduino microcontroller which was shown on laptop monitor as voltage output. The sensor setup was deployed to the needle latch areas, cambox areas and sinker ring areas by simply placing them near these positions.
The sound data captured in real time from the Arduino is stored through CoolTerm software. This is a cross-platform terminal application that monitor and capture real time data sent over serial ports, store and analyze data from connected devices and useful for sensor reading debugging and storing. This software saves the sound voltage data as txt or CSV file format.
With the collaboration of Microsoft Excel and CoolTerm software graphs of sound voltage data vs time are extracted. The txt or CSV file from CoolTerm is used as the readable content for Excel and Excel converts the data into graphical images as jpeg format. The voltage vs time graph is extracted at a constant interval of time so that the analysis of the machine learning can gain higher accuracy.
The sound data was collected for fault-free machine condition, needle butt broken, hook broken and faulty sinker condition. All these data were converted to voltage-time graph and a dataset is created based on different faulty conditions like needle fault and sinker fault. About 1270 graph image was added to dataset for training the VGG16 model.
3.2 Train Machine Learning Model
We used the VGG16 image classifier machine learning model where all the 1270 datasets were fed and trained. VGG16 is a deep convolutional neural network architecture that has 16 layers with 3x3 convolution filters with amazing image classification task success rate. The code was written in Python language and google colab was used because of insufficient hardware. After training with VGG16, CSV file of dataset was trained with other classifiers for comparison and effectiveness of the fault detection.
3.3 Evaluation of Model and Fault Detection
After the training process the trained model detects and predicts if there are any faults in the machine or not. If faulty condition occurs it gives result and specifies as needle fault or sinker fault. The accuracy of the model was 73.8%.
Figure 2:
Figure 2: Graph of sound data in machine all okay condition
Figure 3:
Figure 3: Graph of sound data for sinker fault condition
Figure 4:
Figure 4: Graph of sound data for needle butt fault condition
Figure 5:
Figure 5: Graph of sound data for needle hook fault condition
4 Experiment Evaluation
Here in this section the experimented data collection process details along with the different sound data analysis and differentiation with fault detecting ability and accuracy are discussed.
4.1 Data Collection
In the beginning of the experiment, on the lab we turned off all the machines except the experimental subject machine, a Single Jersey Knitting machine of FUKUHARA brand. We captured sound data from the all-okay condition first, then we sequentially collected faulty machine sound data by creating artificial real time faults (Needle butt broken, Needle hook broken, Faulty sinker). Quiet place was ensured for pure sound intensity and achieve higher accuracy. Collecting all the sound data we executed all the methodology works step by step and got success on needle fault or sinker fault detection in the knitting machine.
4.1.1 All Okay Condition.
At first sound data in voltage value is collected through the sensor for the machine with all okay and healthy condition. Then with collaboration of CoolTerm and Excel graph of voltage vs time image was extracted for further comparison with the faulty sound graphs. Such a sample graph is shown in Figure 2.
4.1.2 Faulty Sinker Condition.
A very important element of knitting machine specially for the circular single jersey knitting machines is a sinker that holds down the fabric, forms loops, clearing of needle and controls the loop length. Because of a faulty needle drop stitch, fabric press off, loop length variation, sinker line in fabric, puckering and many more faults occur. To identify sinker fault based on sound we conduct this experiment with pre-installed bent or broken sinkers and store the sound data which afterwards converted into voltage-time graph image and count as dataset element. The sample graph is shown in Figure 3.
4.1.3 Faulty Needle Condition.
Needle is one of the main elements for a Knitting machine and a faulty one can generate irreparable faults in the fabrics such as dropped stitch, missed stitch, float stitch, snagging, barre marks or tension issues. For resolving the fault issue these needle faults need to be found out and it is the motive of our research to do so. Faulty needles like broken butt needle or broken hook or latch needle was inputted into the knitting machine and sound data was captured. After that the data was turned into graph image for further analysis. The sound graph for Needle Butt Fault and Needle Hook Fault for the knitting machine are shown in Figure 4 and Figure 5 respectively.
Figure 6:
Figure 6: All okay and needle hook fault graph comparison
Figure 7:
Figure 7: All okay and sinker fault graph comparison
Figure 8:
Figure 8: Needle fault and sinker fault graph comparison
Figure 9:
Figure 9: All okay and needle butt fault graph comparison
Figure 10:
Figure 10: All okay and all faults’ graphs comparison
4.2 Data Analysis
This section contains the comparison and distinct difference discussions among the various faulty conditions and okay or healthy condition of the machine. The processed sound data gives voltage-time graph images and by comparing them with simple overlapping of graphs a clear confirmation was found that there remains sound difference among the various faulty conditions and okay condition. Furthermore, among the needle butt fault, needle hook fault and sinker fault there remains distinct difference on the graph amplitude or voltage. As the difference between different faults are confirmed, we were able to train the machine learning model and got fault detection success.
As for the all-okay condition, the amplitude or voltage peaks at about 0.2 volt and most of the time deflects between 0.07 to 0.12 volt. On the other hand, for Faulty needle butt condition the sound intensity or voltage increases and peaks at about 0.4 volt and deflects between 0.1 to 0.3 volt mostly. For the Needle hook defect condition amplitude ranges from 0.1 to 0.5 volt with peak voltage of about 0.6 volts. In comparison with sinker fault range is spread through 0.1 to 0.25 volt with different pattern with highest amplitude with 0.35 volt.
These comparisons contribute as the confirmation of difference between sound intensity between faulty machine and okay condition machine and also confirms distinct difference among needle fault and sinker fault. All the comparison graphs of sound data are shown in Figure 6,7,8,9 and 10.
Figure 11:
Figure 11: Proposed VGG16 model architecture
4.3 Findings
Here in the section discussion is made based on the findings of the trained model, result of prediction and detection of sinker fault or needle fault along with their accuracy and training loss.
4.3.1 Result from Trained Model.
VGG16 image classifier machine learning model was used for the sound data graph image data set training and implementing the model for fault detection. The dataset of all types of needle fault and sinker fault were trained with VGG16 and an accuracy of 73.8% was obtained. The model was performed for 50 epochs or 50 complete cycles over the dataset. We got the results of these epochs as loss of the model, loss on the validation data, accuracy of the model and validation data accuracy. The proposed model architecture is shown in Figure 11.
Table 1:
Classes
Precision (%)
Recall (%)
F1 Score (%)
Needle butt fault
66.7
72.1
70.5
Needle hook fault
65.2
77.3
72.5
Needle latch fault
67.5
78.0
72.2
Sinker fault
75.7
73.4
75.2
All okay condition
72.4
79.2
78.1
Average
69.5
76.0
73.8
Table 1: Accuracy result of VGG16 test model
Table 2:
Algorithm
Precision(%)
Recall(%)
F1 score(%)
Nearest Centroid
53.8
54.5
53.7
Gaussian NB
53.3
54.5
53.5
SGD Classifier
53.6
54.3
53.2
AdaBoost Classifier
72.3
70.8
70.6
KNeighbors Classifier
55.3
54.8
55.2
Bagging Classifier
47.1
46.0
44.8
Decision Tree Classifier
61.7
59.9
59.6
Random Forest Classifier
61.5
59.1
59.3
Gradient Boosting
55.6
54.9
54.4
Extra Trees Classifier
49.8
49.2
49.6
VGG16 Classifier
69.5
76.0
73.8
Transformer Classifier
68.2
88.7
76.1
Table 2: Accuracy of different machine learning models
This model uses binary cross-entropy as the loss function suitable for binary classification as detection of faulty needle and faulty sinker. Total 1270 data is used as dataset and among them 890 is training set and 190 for both validation and testing set. The dataset summery is shown in Table 3.
As for other classifiers like nearest centroid, gaussian NB, SGD, AdaBoost, KNeighbors, Bagging, Decision tree, Random Forest, Gradient Bosting, Extra Trees and lastly Transformer classifier was used to train the dataset and get the accuracy results for comparison and understanding which model performs well for our created dataset and detect fault flawlessly.
Table 3:
Classes
Number of samples in different dataset
Training set
Validation set
Testing set
All okay condition
178
38
38
Needle butt fault
178
38
38
Needle hook fault
178
38
38
Needle latch fault
178
38
38
Sinker fault
178
38
38
Total
890
190
190
Table 3: Class-wise dataset summary
4.3.2 Result of Fault Prediction.
In this code segment the main motive of the experiment is conducted. The accuracy for all the test models is evaluated. As for the result, the trained model successfully detected needle or sinker fault at about 76.1% accuracy.
The test sound data graph image was also resized by the code into 224x224 pixels and prediction was analyzed on which class (needle or sinker fault) the data belongs for the VGG16 model. Accuracy results as precision, recall and f1 score are depicted on Table 1. VGG16 model average accuracy was about 73.8%. Confusion matrix for the model gives a clear image of the correct and misguided data which is shown in Figure 12.
In case of the other classifiers their accuracy result was also evaluated as their precision, recall and f1 score. It is shown on Table 2. It clearly shows that the Transformer classifier gives the highest 76.1% accuracy for our dataset. The VGG16 also keeps up with a 73.8% accuracy. Hence chance for correctly detecting of needle and sinker fault increases.
Figure 12:
Figure 12: Confusion matrix
5 Conclusion
In this research, we proposed and successfully implemented a sound-based fault detection system to detect needle fault or sinker fault in knitting machines. By using a sound sensor, we took the sound data and through Arduino uno we converted the sound intensity signal into electric signal as voltage value which were afterwards recorded by the CoolTerm software. By the collaboration of CoolTerm and Microsoft Excel the sound data was converted into voltage-time graph image and also CSV file and after creating dataset it was trained with VGG16 machine learning image classifier model along with Transformer and other classifiers. This model was test run and successfully detected fault from the running knitting machine.
The experimental results showed that our method performed remarkably well and gave a high accuracy of 73.8% in distinguishing between needle fault and sinker fault in a knitting machine. Such high accuracy depicts our research success and efficiency of sound analysis approach in detecting faults. Comparing to other fault detection methods this sound based image classifying approach is both cost effective and highly accurate. This research could be helpful for real-time automated fault detection in knitting machines. Future work could be increasing the dataset and include more fault types with system optimization.
References
[1]
Muhammad Altaf, Muhammad Uzair, Muhammad Naeem, Ayaz Ahmad, Saeed Badshah, Jawad Ali Shah, and Almas Anjum. 2019. Automatic and efficient fault detection in rotating machinery using sound signals. Acoustics Australia 47 (2019), 125–139.
Ayhan Altinors, Ferhat Yol, and Orhan Yaman. 2021. A sound based method for fault detection with statistical feature extraction in UAV motors. Applied Acoustics 183 (2021), 108325.
Prasanna Bandara, Thilan Bandara, Tharaka Ranatunga, Vibodha Vimarshana, Sulochana Sooriyaarachchi, and Chathura De Silva. 2018. Automated fabric defect detection. In 2018 18th International Conference on Advances in ICT for Emerging Regions (ICTer). IEEE, 119–125.
Md Harunur Rashid Bhuiyan, Iftekhar Morshed Arafat, Masfiqur Rahaman, Tarik Reza Toha, and Shaikh Md Mominul Alam. 2022. Devising a Vibration-Based Fault Detection System for Textile Machinery. In International Conference on Mobile and Ubiquitous Systems: Computing, Networking, and Services. Springer, 3–20.
Md Harunur Rashid Bhuiyan, Muhammad Tafsirul Islam, Nazmul Islam, Mynul Islam, Anupom Mondol, Tarik Reza Toha, and Shaikh Md Mominul Alam. 2022. Sound-Based Fault Detection For Textile Machinery. In Proceedings of the 9th International Conference on Networking, Systems and Security. 53–62.
André Catarino, Ana Rocha, João L Monteiro, and Filomena Soares. 2004. A system for knitting process monitoring and fault detection on weft circular knitting machines. proceedings of AUTEX (2004).
Yuan Fu, Xiang Chen, Yu Liu, Chan Son, and Yan Yang. 2022. Multi-source information fusion fault diagnosis for gearboxes based on SDP and VGG. Applied Sciences 12, 13 (2022), 6323.
Md Khalilur Rahman, Mst Tasqurun Nessa, and Md Tarek Rahman. 2019. Export Performance of Readymade Garments Sector in Bangladesh. Journal of Economics and Development Studies 8, 1 (2019), 27–41.
Hamid Shiri, Jacek Wodecki, Bartłomiej Ziętek, and Radosław Zimroz. 2021. Inspection robotic UGV platform and the procedure for an acoustic signal-based fault detection in belt conveyor idler. Energies 14, 22 (2021), 7646.
Zia Ullah, Bilal Ahmad Lodhi, and Jin Hur. 2020. Detection and identification of demagnetization and bearing faults in PMSM using transfer learning-based VGG. Energies 13, 15 (2020), 3834.
Long Wen, X Li, Xinyu Li, and Liang Gao. 2019. A new transfer learning based on VGG-19 network for fault diagnosis. In 2019 IEEE 23rd international conference on computer supported cooperative work in design (CSCWD). IEEE, 205–209.
Zhouqiang Zhang, Sihao Bai, Guang-shen Xu, Xuejing Liu, Feilei Wang, Jiangtao Jia, and Zhi Feng. 2020. Research on the knitting needle detection system of a hosiery machine based on machine vision. Textile Research Journal 90, 15-16 (2020), 1730–1740.
NSysS '22: Proceedings of the 9th International Conference on Networking, Systems and Security
The textile sector is one of the vital driving forces in the economy of south Asian countries like Bangladesh, India, Pakistan, etc. However, most of the textile industries suffer from frequent machinery faults everyday which reduces their productivity,...
ICTCE '24: Proceedings of the 2024 6th International Conference on Telecommunications and Communication Engineering
The importance of detecting motor faults to ensure the stable operation of industrial equipment and reduce maintenance costs cannot be overstated. This research introduces a motor fault detection technology based on analyzing acoustic signals. The ...
Special issue on independent components analysis and beyond
Through the use of DSP chips and multiple microphones, hearing aids now offer the possibility of performing signal-to-noise enhancement. Evaluating different algorithms before they are instantiated on a hearing aid is essential. However, commercially ...
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Md. Washiur Rahman, 
Tarik Reza Toha, 
