Skip to main content

Advertisement

Springer Nature Link
Log in

Convolutional neural network based multi-input multi-output model for multi-sensor multivariate virtual metrology in semiconductor manufacturing

  • Original Research
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Virtual metrology (VM) in semiconductor manufacturing is to predict product physical quality measurements using the processing information at a wafer fabrication process. The process state parameters from various sensors on production equipment are used for the accurate and reliable prediction of process outcomes, and several deep learning methods have been considered, including convolutional neural network (CNN). Recent studies in the literature have demonstrated the successful VM modeling with CNN for univariate-response prediction. Multivariate process outputs, however, are overlooked in deep learning-based VM modeling although the joint information among the process outputs can be used to improve the prediction performance. In this work, we propose a CNN-based multivariate VM model using multi-sensors process sensor data. We evaluate the proposed model on a real-life case for VM modeling at an etching process in wafer fabrication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Berndt, D. J., & Clifford, J. (1994). Using dynamic time warping to find patterns in time series. In Proceedings of the 3rd international conference on knowledge discovery and data mining (pp. 359–370).

  • Bishop, C. M. (1995). Training with noise is equivalent to Tikhonov regularization. Neural Computation, 7(1), 108–116.

    Google Scholar 

  • Borchani, H., Varando, G., Bielza, C., & Larranaga, P. (2015). A survey on multi-output regression. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 5(5), 216–233.

    Google Scholar 

  • Breiman, L., & Friedman, J. H. (1997). Predicting multivariate responses in multiple linear regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 59(1), 3–54.

    Google Scholar 

  • Burnham, A. J., MacGregor, J. F., & Viveros, R. (1999). Latent variable multivariate regression modeling. Chemometrics and Intelligent Laboratory Systems, 48(2), 167–180.

    Google Scholar 

  • Caruana, R. (1997). Multitask learning. Machine Learning, 28, 41–75.

    Google Scholar 

  • Chan, L. L. T., Wu, X., Chen, J., Xie, L., & Chen, C. I. (2018). Just-in-time modeling with variable shrinkage based on Gaussian processes for semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 31(3), 335–342.

    Google Scholar 

  • Chen, P., Wu, S., Lin, J., Ko, F., Lo, H., Wang, J., Yu, C.H., & Liang, M. (2005). Virtual metrology: A solution for wafer to wafer advanced process control. In ISSM 2005, IEEE international symposium on semiconductor manufacturing, 2005 (pp. 155–157).

  • Cheng, F. T., Chen, Y. T., Su, Y. C., & Zeng, D. L. (2008). Evaluating reliance level of a virtual metrology system. IEEE Transactions on Semiconductor Manufacturing, 21(1), 92–103.

    Google Scholar 

  • Cheng, F. T., Chang, J. Y. C., Huang, H. C., Kao, C. A., Chen, Y. L., & Peng, J. L. (2011). Benefit model of virtual metrology and integrating AVM into MES. IEEE Transactions on Semiconductor Manufacturing, 24(2), 261–272.

    Google Scholar 

  • Chien, J. C., Wu, M. T., & Lee, J. D. (2020). Inspection and classification of semiconductor wafer surface defects using CNN deep learning networks. Applied Sciences, 10(15), 5340.

    Google Scholar 

  • Choi, J., & Jeong, M. K. (2018). Deep autoencoder with clipping fusion regularization on multistep process signals for virtual metrology. IEEE Sensors Letters, 3(1), 1–4.

    Google Scholar 

  • Chollet, F. (2015). Keras. https://www.keras.io.

    Google Scholar 

  • Djenouri, Y., Srivastava, G., & Lin, J. C. W. (2020). Fast and accurate convolution neural network for detecting manufacturing data. IEEE Transactions on Industrial Informatics, 17(4), 2947–2955.

    Google Scholar 

  • Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the 13th international conference on artificial intelligence and statistics (pp. 249–256).

  • Goodfellow, I., Bengio, Y., & Courville, A. (2016). Deep Learning. MIT Press.

    Google Scholar 

  • Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., & Chen, T. (2018). Recent advances in convolutional neural networks. Pattern Recognition, 77, 354–377.

    Google Scholar 

  • Han, Z., Liu, Y., Zhao, J., & Wang, W. (2012). Real time prediction for converter gas tank levels based on multi-output least square support vector regressor. Control Engineering Practice, 20(12), 1400–1409.

    Google Scholar 

  • Hirai, T., & Kano, M. (2015). Adaptive virtual metrology design for semiconductor dry etching process through locally weighted partial least squares. IEEE Transactions on Semiconductor Manufacturing, 28, 137–144.

    Google Scholar 

  • Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural Computation, 9(8), 1735–1780.

    Google Scholar 

  • Hsieh, Y. M., Wang, T. J., Lin, C. Y., Peng, L. H., Cheng, F. T., & Shang, S. Y. (2021). Convolutional neural networks for automatic virtual metrology. IEEE Robotics and Automation Letters, 6(3), 5720–5727.

    Google Scholar 

  • Hsu, C. Y., & Liu, W. C. (2021). Multiple time-series convolutional neural network for fault detection and diagnosis and empirical study in semiconductor manufacturing. Journal of Intelligent Manufacturing, 32, 823–836.

    Google Scholar 

  • Huang, Z., Zhu, J., Lei, J., Li, X., & Tian, F. (2020). Tool wear predicting based on multi-domain feature fusion by deep convolutional neural network in milling operations. Journal of Intelligent Manufacturing, 31, 953–966.

    Google Scholar 

  • Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167.

  • Jia, X., Di, Y., Feng, J., Yang, Q., Dai, H., & Lee, J. (2018). Adaptive virtual metrology for semiconductor chemical mechanical planarization process using GMDH-type polynomial neural networks. Journal of Process Control, 62, 44–54.

    Google Scholar 

  • Kang, P., Lee, H. J., Cho, S., Kim, D., Park, J., Park, C. K., & Doh, S. (2009). A virtual metrology system for semiconductor manufacturing. Expert Systems with Applications, 36(10), 12554–12561.

    Google Scholar 

  • Kang, S., & Kang, P. (2017). An intelligent virtual metrology system with adaptive update for semiconductor manufacturing. Journal of Process Control, 52, 66–74.

    Google Scholar 

  • Kang, S., Kim, D., & Cho, S. (2016). Efficient feature selection-based on random forward search for virtual metrology modeling. IEEE Transactions on Semiconductor Manufacturing, 29(4), 391–398.

    Google Scholar 

  • Kano, M., & Fujiwara, K. (2013). Virtual sensing technology in process industries: Trends and challenges revealed by recent industrial applications. Journal of Chemical Engineering of Japan, 46(1), 1–17.

    Google Scholar 

  • Khan, A. A., Moyne, J. R., & Tilbury, D. M. (2007). An approach for factory-wide control utilizing virtual metrology. IEEE Transactions on Semiconductor Manufacturing, 20(4), 364–375.

    Google Scholar 

  • Khan, A. A., Moyne, J. R., & Tilbury, D. M. (2008). Virtual metrology and feedback control for semiconductor manufacturing processes using recursive partial least squares. Journal of Process Control, 18(10), 961–974.

    Google Scholar 

  • Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020). A survey of the recent architectures of deep convolutional neural networks. Artificial Intelligence Review, 53(8), 5455–5516.

    Google Scholar 

  • Kim, P. (2017). MATLAB deep learning with machine learning, neural networks and artificial intelligence. Springer.

    Google Scholar 

  • Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.

  • Kocev, D., Džeroski, S., White, M. D., Newell, G. R., & Griffioen, P. (2009). Using single-and multi-target regression trees and ensembles to model a compound index of vegetation condition. Ecological Modelling, 220(8), 1159–1168.

    Google Scholar 

  • Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, Vol. 25.

  • Kurz, D., De Luca, C., & Pilz, J. (2014). A sampling decision system for virtual metrology in semiconductor manufacturing. IEEE Transactions on Automation Science and Engineering, 12(1), 75–83.

    Google Scholar 

  • Kuznar, D., Mozina, M., & Bratko, I. (2009). Curve prediction with kernel regression. In Proceedings of the 1st workshop on learning from multi-label data (pp. 61–68).

  • LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2324.

    Google Scholar 

  • Lee, K. B., Cheon, S., & Kim, C. O. (2017). A convolutional neural network for fault classification and diagnosis in semiconductor manufacturing processes. IEEE Transactions on Semiconductor Manufacturing, 30(2), 135–142.

    Google Scholar 

  • Lee, K. B., & Kim, C. O. (2020). Recurrent feature-incorporated convolutional neural network for virtual metrology of the chemical mechanical planarization process. Journal of Intelligent Manufacturing, 31(1), 73–86.

    Google Scholar 

  • Lee, W., & Liu, Y. (2012). Simultaneous multiple response regression and inverse covariance matrix estimation via penalized Gaussian maximum likelihood. Journal of Multivariate Analysis, 111, 241–255.

    Google Scholar 

  • Li, Z., Liu, F., Yang, W., Peng, S., & Zhou, J. (2021). A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Transactions on Neural Networks and Learning Systems, 33, 6999–7019.

    Google Scholar 

  • Lin, T. H., Hung, M. H., Lin, R. C., & Cheng, F. T. (2006). A virtual metrology scheme for predicting CVD thickness in semiconductor manufacturing. In Proceedings 2006 IEEE international conference on robotics and automation, 2006. ICRA 2006. (pp. 1054–1059).

  • Liu, R., Yang, B., & Hauptmann, A. G. (2019). Simultaneous bearing fault recognition and remaining useful life prediction using joint-loss convolutional neural network. IEEE Transactions on Industrial Informatics, 16(1), 87–96.

    Google Scholar 

  • Lu, B., Stuber, J., & Edgar, T. F. (2014). Integrated online virtual metrology and fault detection in plasma etch tools. Industrial & Engineering Chemistry Research, 53(13), 5172–5181.

    Google Scholar 

  • Maggipinto, M., Beghi, A., & Susto, G. A. (2022). A deep convolutional autoencoder-based approach for anomaly detection with industrial, non-images, 2-dimensional data: A semiconductor manufacturing case study. IEEE Transactions on Automation Science and Engineering, 19(3), 1477–1490.

    Google Scholar 

  • Maggipinto, M., Terzi, M., Masiero, C., Beghi, A., & Susto, G. A. (2018). A computer vision-inspired deep learning architecture for virtual metrology modeling with 2-dimensional data. IEEE Transactions on Semiconductor Manufacturing, 31(3), 376–384.

    Google Scholar 

  • Mei, J., Liu, M., Wang, Y. F., & Gao, H. (2015). Learning a mahalanobis distance-based dynamic time warping measure for multivariate time series classification. IEEE Transactions on Cybernetics, 46(6), 1363–1374.

    Google Scholar 

  • Mo, Z., Zhang, Z., & Tsui, K. L. (2021). The variational kernel-based 1-D convolutional neural network for machinery fault diagnosis. IEEE Transactions on Instrumentation and Measurement, 70, 1–10.

    Google Scholar 

  • Müller, M. (2007). Dynamic time warping. Information Retrieval for Music and Motion. https://doi.org/10.1007/978-3-540-74048-3_4

    Article  Google Scholar 

  • Nair, V., & Hinton, G. E. (2010). Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on international conference on machine learning (pp. 807–814).

  • Nakazawa, T., & Kulkarni, D. V. (2019). Anomaly detection and segmentation for wafer defect patterns using deep convolutional encoder–decoder neural network architectures in semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 32(2), 250–256.

    Google Scholar 

  • Olson, M., Wyner, A. J., & Berk, R. (2018). Modern neural networks generalize on small data sets. In Proceedings of the 32nd international conference on neural information processing systems (pp. 3623–3632).

  • Parkinson, B. R., Lee, H., Funk, M., Prager, D., Yamashita, A., Sundararajan, R., & Edgar, T. F. (2010). Addressing dynamic process changes in high volume plasma etch manufacturing by using multivariate process control. IEEE Transactions on Semiconductor Manufacturing, 23(2), 185–193.

    Google Scholar 

  • Peres, F. A. P., & Fogliatto, F. S. (2018). Variable selection methods in multivariate statistical process control: A systematic literature review. Computers & Industrial Engineering, 115, 603–619.

    Google Scholar 

  • Qin, S. J., Cherry, G., Good, R., Wang, J., & Harrison, C. A. (2006). Semiconductor manufacturing process control and monitoring: A fab-wide framework. Journal of Process Control, 16(3), 179–191.

    Google Scholar 

  • Ren, M., Song, Y., & Chu, W. (2019). An improved locally weighted PLS based on particle swarm optimization for industrial soft sensor modeling. Sensors, 19(19), 4099.

    Google Scholar 

  • Saqlain, M., Abbas, Q., & Lee, J. Y. (2020). A deep convolutional neural network for wafer defect identification on an imbalanced dataset in semiconductor manufacturing processes. IEEE Transactions on Semiconductor Manufacturing, 33(3), 436–444.

    Google Scholar 

  • Schlosser, T., Beuth, F., Friedrich, M., & Kowerko, D. (2019). A novel visual fault detection and classification system for semiconductor manufacturing using stacked hybrid convolutional neural networks. In 2019 24th IEEE international conference on emerging technologies and factory automation (ETFA) (pp. 1511–1514). IEEE.

  • Senin, P. (2008). Dynamic time warping algorithm review. Information and Computer Science Department University of Hawaii at Manoa Honolulu, USA, 855(1–23), 40.

    Google Scholar 

  • Similä, T., & Tikka, J. (2007). Input selection and shrinkage in multiresponse linear regression. Computational Statistics & Data Analysis, 52(1), 406–422.

    Google Scholar 

  • Su, A. J., Jeng, J. C., Huang, H. P., Yu, C. C., Hung, S. Y., & Chao, C. K. (2007). Control relevant issues in semiconductor manufacturing: Overview with some new results. Control Engineering Practice, 15(10), 1268–1279.

    Google Scholar 

  • Susto, G. A., Maggipinto, M., Zocco, F., & McLoone, S. (2018). A dynamic sampling approach for cost reduction in semiconductor manufacturing. Procedia Manufacturing, 17, 1031–1038.

    Google Scholar 

  • Suthar, K., Shah, D., Wang, J., & He, Q. P. (2018). Feature-based virtual metrology for semiconductor manufacturing. In Computer aided chemical engineering (Vol. 44, pp. 2083–2088). Elsevier.

  • Suthar, K., Shah, D., Wang, J., & He, Q. P. (2019). Next-generation virtual metrology for semiconductor manufacturing: A feature-based framework. Computers & Chemical Engineering, 127, 140–149.

    Google Scholar 

  • Taşkın, G., Kaya, H., & Bruzzone, L. (2017). Feature selection based on high dimensional model representation for hyperspectral images. IEEE Transactions on Image Processing, 26(6), 2918–2928.

    Google Scholar 

  • Tuia, D., Verrelst, J., Alonso, L., Pérez-Cruz, F., & Camps-Valls, G. (2011). Multioutput support vector regression for remote sensing biophysical parameter estimation. IEEE Geoscience and Remote Sensing Letters, 8(4), 804–808.

    Google Scholar 

  • Valizadeh, M., & Wolff, S. J. (2022). Convolutional Neural Network applications in additive manufacturing: A review. Advances in Industrial and Manufacturing Engineering, 4, 100072.

    Google Scholar 

  • Vallejo, M., de la Espriella, C., Gómez-Santamaría, J., Ramírez-Barrera, A. F., & Delgado-Trejos, E. (2019). Soft metrology based on machine learning: A review. Measurement Science and Technology, 31(3), 032001.

    Google Scholar 

  • Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Proceedings of the 31st international conference on neural information processing systems (pp. 6000–6010).

  • Verdier, G., & Ferreira, A. (2010). Adaptive Mahalanobis distance and k-nearest neighbor rule for fault detection in semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 24(1), 59–68.

    Google Scholar 

  • Wen, G., Gao, Z., Cai, Q., Wang, Y., & Mei, S. (2020). A novel method based on deep convolutional neural networks for wafer semiconductor surface defect inspection. IEEE Transactions on Instrumentation and Measurement, 69(12), 9668–9680.

    Google Scholar 

  • Wu, W. M., Cheng, F. T., Lin, T. H., Zeng, D. L., & Chen, J. F. (2010). Selection schemes of dual virtual-metrology outputs for enhancing prediction accuracy. IEEE Transactions on Automation Science and Engineering, 8(2), 311–318.

    Google Scholar 

  • Wu, X., Chen, J., Xie, L., Chan, L. L. T., & Chen, C. I. (2020). Development of convolutional neural network based Gaussian process regression to construct a novel probabilistic virtual metrology in multi-stage semiconductor processes. Control Engineering Practice, 96, 104262.

    Google Scholar 

  • Yang, J. B., Nguyen, M. N., San, P. P., Li, X. L., & Krishnaswamy, S. (2015). Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of the 24th international conference on artificial intelligence (pp. 3995–4001).

  • Yang, W. T., Blue, J., Roussy, A., Pinaton, J., & Reis, M. S. (2019). A structure data-driven framework for virtual metrology modeling. IEEE Transactions on Automation Science and Engineering, 17(3), 1297–1306.

    Google Scholar 

  • Zhang, X., Kano, M., & Li, Y. (2017). Locally weighted kernel partial least squares regression based on sparse nonlinear features for virtual sensing of nonlinear time-varying processes. Computers & Chemical Engineering, 104, 164–171.

    Google Scholar 

  • Zhang, Y., Teng, Y., & Zhang, Y. (2010). Complex process quality prediction using modified kernel partial least squares. Chemical Engineering Science, 65(6), 2153–2158.

    Google Scholar 

  • Zheng, Y., Liu, Q., Chen, E., Ge, Y., & Zhao, J. L. (2014). Time series classification using multi-channels deep convolutional neural networks. In International conference on web-age information management (pp. 298–310).

  • Zhou, J., Wang, D., Chen, J., Feng, Z., Clarson, B., & Baselhuhn, A. (2021). Autonomous nondestructive evaluation of resistance spot welded joints. Robotics and Computer-Integrated Manufacturing, 72, 102183.

    Google Scholar 

  • Zhu, F., Jia, X., Miller, M., Li, X., Li, F., Wang, Y., & Lee, J. (2020). Methodology for important sensor screening for fault detection and classification in semiconductor manufacturing. IEEE Transactions on Semiconductor Manufacturing, 34(1), 65–73.

    Google Scholar 

Download references

Acknowledgements

The computation in this study was supported in part by the Super Computing System (Thorny Flat) at West Virginia University, which is funded in part by the National Science Foundation Major Research Instrumentation Program Award #1726534, and West Virginia University.

Author information

Authors and Affiliations

  1. Department of Management Information Systems, West Virginia University, Morgantown, WV, USA

    Jeongsub Choi

  2. Department of Textile Engineering, Chemistry and Science, North Carolina State University, Raleigh, NC, USA

    Mengmeng Zhu

  3. Operations Research Graduate Program, North Carolina State University, Raleigh, NC, USA

    Mengmeng Zhu

  4. Business Administration, Tech University of Korea, Siheung, South Korea

    Jihoon Kang

  5. Department of Industrial and Systems Engineering, Rutgers, the State University of New Jersey, Piscataway, NJ, USA

    Myong K. Jeong

Authors
  1. Jeongsub Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mengmeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jihoon Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Myong K. Jeong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Myong K. Jeong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, J., Zhu, M., Kang, J. et al. Convolutional neural network based multi-input multi-output model for multi-sensor multivariate virtual metrology in semiconductor manufacturing. Ann Oper Res 339, 185–201 (2024). https://doi.org/10.1007/s10479-024-05902-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-024-05902-z

Keywords