Skip to main content
SpringerLink
Log in

Evaluating the reliability level of virtual metrology results for flexible process control: a novelty detection-based approach

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The purpose of virtual metrology (VM) in semiconductor manufacturing is to support process monitoring and quality control by predicting the metrological values of every wafer without an actual metrology process, based on process sensor data collected during the operation. Most VM-based quality control schemes assume that the VM predictions are always accurate, which in fact may not be true due to some unexpected variations that can occur during the process. In this paper, therefore, we propose a means of evaluating the reliability level of VM prediction results based on novelty detection techniques, which would allow flexible utilization of the VM results. Our models generate a high-reliability score for a wafer’s VM prediction only when its process sensor values are found to be consistent with those of the majority of wafers that are used in model building; otherwise, a low-reliability score is returned. Thus, process engineers can selectively utilize VM results based on their reliability level. Experimental results show that our reliability generation models are effective; the VM results for wafers with a high level of reliability were found to be much more accurate than those with a low level.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Angiulli F, Pizzuti C (2005) Outlier mining in large high-dimensional data sets. IEEE Trans Knowl Data Eng 17(2):203–215

    Article  MathSciNet  Google Scholar 

  2. Ankerst M, Breunig MM, Kriegel HP, Sander J (1999) OPTICS: Ordering points to identify the clustering structure. In: Proceedings of the 1999 ACM SIGMOD International Conference on Management of Data, New York, NY, USA, pp 49–60

  3. Barnett V, Lewis T (1994) Outliers in statistical data. Wiley and Sons, New York

    MATH  Google Scholar 

  4. Besnard J, Toprac A (2006) Wafer-to-wafer virtual metrology applied to run-to-run control. In: Proceedings of the Third ISMI Symposium on Manufacturing Effectiveness, Austin, TX, USA

  5. Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New York

    Google Scholar 

  6. Bode C, Ko B, Edgar T (2004) Run-to-run control and performance monitoring of overlay in semiconductor manufacturing. Control Eng Pract 12(7):893–900

    Article  Google Scholar 

  7. Boutsinas B, Tasoulis DK, Vrahatis MN (2006) Estimating the number of clusters using a windowing technique. Pattern Recogn Image Anal 16(2):143–154

    Article  Google Scholar 

  8. Breunig MM, Kriegel HP, Ng RT, Sander J (2000) LOF: Identifying density-based local outliers. In: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, Dallas, Texas, USA, pp 93–104

  9. Burge P, Shawe-Taylor J (1997) Detecting cellular fraud using adaptive prototypes. In: Proceedings of AAAI-97 Workshop on AI Approaches to Fraud Detection and Risk Management, Rhode Island, USA, pp 9–13

  10. Castillo DE, Yeh JY (1998) An adaptive run-to-run optimizing controller for linear and nonlinear semiconductor processes. IEEE Trans Semicond Manuf 11(2):285–295

    Article  Google Scholar 

  11. Chang YJ, Kang Y, Hsu CL, Chang CT, Chan T (2006) Virtual metrology techniques for semiconductor manufacturing. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2006), Vancouver, BC, Canada, pp 5289–5293

  12. Chemali CE, Freudenberg J, Hankinson M, Bendik JJ (2004) Run-to-run critical dimension and sidewall angle lithography control using the PROLITH simulator. IEEE Trans Semicond Manuf 17(3):388–401

    Article  Google Scholar 

  13. Chen P, Wu S, Lin J, Ko F, Lo H, Wang J (2005) Virtual metrology: a solution for wafer to wafer advanced process control. In: Proceedings of International Symposium on Semiconductor Manufacturing (ISSM 2005), San Jose, CA, USA, pp 155–157

  14. Chen YT, Yang HC, Cheng FT (2006) Multivariate simulation assessment for virtual metrology. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA 2006), Orlando, FL, USA, pp 1048–1053

  15. Cheng CY, Cheng FT (2005) Application development of virtual metrology in semiconductor industry. In: Proceedings of The 31st Annual Conference of IEEE Industrial Electronics Society (IECON 2005), Raleigh, NC, USA, pp 124–129

  16. Cheng FT, Chen YT, Su YC, Zeng DL (2008) Evaluating reliance level of a virtual metrology system. IEEE Trans Semicond Manuf 21(1):92–103

    Article  Google Scholar 

  17. Cheng FT, Chang JC, Huang HC, Kao CA, Chen YL, Peng JL (2011) Benefit model of virtual metrology and integrating AVM into MES. IEEE Trans Semicond Manuf 24(2):261–272

    Article  Google Scholar 

  18. Cheng FT, Huang HC, Kao CA (2012) Developing an automatic virtual metrology system. IEEE Trans Autom Sci Eng 9(1):181–188

    Article  Google Scholar 

  19. Cho SJ, Hermsmeier MA (2002) Genetic algorithm guided selection: variable selection and subset selection. J Chem Inf Model 42(4):927–936

    Article  Google Scholar 

  20. Cyganek B (2012) One-class support vector ensembles for image segmentation and classification. J Math Imaging Vis 42(2–3):103–117

    Article  MathSciNet  MATH  Google Scholar 

  21. Duda RO, Hart PE, Stork DG (2001) Pattern classification. Wiley, New York

    MATH  Google Scholar 

  22. Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. Second international conference on knowledge discovery and data mining. AAAI Press, Menlo Park, pp 226–231

    Google Scholar 

  23. Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139

    Article  MathSciNet  MATH  Google Scholar 

  24. Gergeret F, Gall G (2003) Yield improvement using statistical analysis of process data. IEEE Trans Semicond Manuf 16(3):535–542

    Article  Google Scholar 

  25. Guha S, Rastogi R, Shim K (1998) CURE: An efficient clustering algorithm for large databases. In: Proceedings of ACM SIGMOD Conference on Management of Data, Seattle, WA, USA, pp 73-84

  26. Guha S, Rastogi R, Shim K (2000) ROCK: a robust clustering algorithm for categorical attributes. Inf Syst 25(5):345–366

    Article  Google Scholar 

  27. Harmeling S, Dornhege G, Tax D, Meinecke F, Müller KR (2006) From outliers to prototypes: ordering data. Neurocomputing 69(13–15):1608–1618

    Article  Google Scholar 

  28. Hastie T, Tibshirani R, Friedman J (2002) The element of statistical learning: data mining, inference, and prediction. Springer-Verlag, New York

    Google Scholar 

  29. Hawkins DM (1980) Identification of outliers. Chapman & Hall, London

    Book  MATH  Google Scholar 

  30. Jarvis RM, Goodacre R (2004) Genetic algorithm optimization for pre-processing and variable selection of spectroscopic data. Bioinformatics 21(7):860–868

    Article  Google Scholar 

  31. Jiang H, Liu G, Xiao X, Mei C, Ding Y, Yu S (2012) Monitoring of solid-state fermentation of wheat straw in a pilot scale using FT-NIR spectroscopy and support vector data description. Microchem J 102:68–74

    Article  Google Scholar 

  32. Kang P, Cho S (2008) Locally linear reconstruction for instance-based learning. Pattern Recogn 41(11):3507–3518

    Article  MATH  Google Scholar 

  33. Kang P, Cho S (2009) A hybrid novelty score and its use in keystroke dynamics-based user authentication. Pattern Recogn 42(11):3115–3127

    Article  MATH  Google Scholar 

  34. Karypis G, Han EH, Kumar V (1999) Chameleon: hierarchical clustering using dynamic modeling. IEEE Comput Mag 32(8):68–75

    Article  Google Scholar 

  35. Khan A, Moyne J, Tilbury D (2007) An approach for factory-wide control utilizing virtual metrology. IEEE Trans Semicond Manuf 20(4):364–375

    Article  Google Scholar 

  36. Khazai S, Safari A, Mojaradi B, Homayouni S (2012) Improving the SVDD approach to hyperspectral image classification. IEEE Geosci Remote Sens Lett 9(4):594–598

    Article  Google Scholar 

  37. Knorr EM, Ng RT, Tucakov V (2000) Distance-based outliers: algorithms and applications. VLDB J 8(3–4):237–253

    Article  Google Scholar 

  38. Krawczyk B, Filipczuk P (2013) Cytological image analysis with firely nuclei detection and hybrid one-class classification decomposition. Eng Appl Artif Intell. doi:10.1016/j.engappai.2013.09.017

    Google Scholar 

  39. Krawczyk B, Woźniak M (2014) Diversity measures for one-class classifier ensembles. Neurocomputing 126:36–44

    Article  Google Scholar 

  40. Limanond S, Si J, Tsakalis K (1998) Monitoring and control of semiconductor manufacturing processes. IEEE Control Syst Mag 18(6):46–58

    Article  Google Scholar 

  41. Lin TH, Hung MT, Lin RC, Cheng FT (2006) A virtual metrology scheme for predicting CVD thickness in semiconductor manufacturing. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA 2006), Orlando, FL, USA, pp 1054–1059

  42. Lynn S, Ringwood J, MacGrearailt N (2010) Weighted windowed PLS models for virtual metrology of an industrial plasma etch process. In: Proceedings of IEEE International Conference on Industrial Technology (ICIT 2010), Vina del Mar, Chile, pp 309–314

  43. Mũnoz-Marí J, Bovolo F, Gomez-Chova L, Bruzzone L, Camp-Valls G (2010) Semisupervised one-class support vector machines for classification of remote sensing data. IEEE Trans Geosci Remote Sens 48(8):3188–3197

    Article  Google Scholar 

  44. McLachlan GJ, Peel D (2000) Finite mixture models. Wiley and Sons, New York

    Book  MATH  Google Scholar 

  45. Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Networks 12(2):181–201

    Article  Google Scholar 

  46. Pang TH, Sheng BQ, Wong DSH, Jang SS (2011) A virtual metrology system for predicting end-of-line electrical properties using MANCOVA model with tools clustering. IEEE Trans Ind Inf 7(2):187–195

    Article  Google Scholar 

  47. Park SJ, Lee MS, Shin SY, Cho KH, Lim JT, Cho BS, Jei YH, Kim MK, Park CH (2005) Run-to-run overlay control of steppers in semiconductor manufacturing systems. IEEE Trans Semicond Manuf 18(4):605–613

    Article  Google Scholar 

  48. Qin S, Cherry G, Good R, Wang J, Harrison C (2006) Semiconductor manufacturing process control and monitoring: a fab-wide framework. J Process Control 16(3):179–191

    Article  Google Scholar 

  49. Ramaswamy S, Rastogi R, Shim K (2000) Efficient algorithms for mining outliers from large data sets. In: Proceedings of International Conference on Management of Data (SIGMOD 2000), Dallas, TX, USA

  50. Ross SM (2004) Introduction to probability and statistic for engineers and scientists. Academic Press, San Diego

    Google Scholar 

  51. Sachs E, Hu A, Ingolfsson A (1995) Run by run process control: combining SPC and feedback control. IEEE Trans Semicond Manuf 8(1):26–43

    Article  Google Scholar 

  52. Schölkopf B, Platt JC, Shawe-Taylor J, Smola AJ (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471

    Article  MATH  Google Scholar 

  53. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge

    Book  Google Scholar 

  54. Spanos C, Guo HF, Miller A, Levine-Parril J (1992) Real-time statistical process control using tool data. IEEE Trans Semicond Manuf 5(4):308–318

    Article  Google Scholar 

  55. Su AJ, Jeng JC, Huang HP, Yu CC, Hung SY, Chao CK (2007) Control relevant issues in semiconductor manufacturing: overview with some new results. Control Eng Pract 15(10):1268–1279

    Article  Google Scholar 

  56. Su YC, Lin TH, Cheng FT, Wu WM (2008) Accuracy and real-time considerations for implementing various virtual metrology algorithms. IEEE Trans Semicond Manuf 21(3):426–434

    Article  Google Scholar 

  57. Tax D (2001) One-class classification: Concept-learning in the absence of counterexamples. PhD thesis, Delft University of Technology, URL http://wwwict.ewi.tudelft.nl/davidt

  58. Tax D, Duin R (1999) Support vector domain description. Pattern Recogn Lett 20(11–13):1191–1199

    Article  Google Scholar 

  59. Tax D, Duin R (2005) Characterizing one-class datasets. In: Proceedings of the Sixteenth Annual Symposium of the Pattern Recognition Association of South Africa, Langebaan, South Africa, pp 21–26

  60. Vapnik V (1998) Statistical learning theory. Wiley and Sons, New York

    MATH  Google Scholar 

  61. Wang XA, Mahajan R (1996) Artificial neural network model-based run-to-run process controller. IEEE Trans Compon Packag Manuf Technol part C 19(1):19–26

    Article  Google Scholar 

  62. Wilk T, Wozniak M (2012) Soft computing methods applied to combination of one-class classifiers. Neurocomputing 75(1):185–193

    Article  Google Scholar 

  63. Yeh CY, Lee ZY, Lee SJ (2009) Boosting one-class support vector machines for multi-class classification. Appl Artif Intell 23(4):297–315

    Article  Google Scholar 

  64. Yi J, Sheng Y, Xu C (2003) Neural network based uniformity profile control of linear chemical-mechanical planarization. IEEE Trans Semicond Manuf 16(4):609–620

    Article  Google Scholar 

  65. Ypma A, Duin RPW (1998) Support objects for domain approximation. In: Proceedings of International Conference on Artificial Neural Networks, Skovde, Sweden

  66. Zhang T, Ramakrishnan R, Livny M (1996) BIRCH: an efficient data clustering method for very large databases. In: Proceedings of ACM SIGMOD Conference on Management of Data, Montreal, Canada, pp 103–114

  67. Zhuang L, Dai H (2006) Parameter optimization of kernel-based on-class classifier on imbalance learning. J Comput 1(7):32–40

    Article  Google Scholar 

Download references

Acknowledgments

The first author was supported by the research program funded by the Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2011-0021893) and by the Ministry of Science, ICT, and Future Planning (NRF-2014R1A1A1004648). The dataset used in this paper can be available upon request.

Author information

Authors and Affiliations

  1. School of Industrial Management Engineering, Korea University, Seoul, South Korea

    Pilsung Kang

  2. Department of Industrial Engineering, Seoul National University, 599 Gwanak-ro, Gwanak-gu, Seoul, 151-744, South Korea

    Dongil Kim & Sungzoon Cho

Authors
  1. Pilsung Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongil Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sungzoon Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pilsung Kang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kang, P., Kim, D. & Cho, S. Evaluating the reliability level of virtual metrology results for flexible process control: a novelty detection-based approach. Pattern Anal Applic 17, 863–881 (2014). https://doi.org/10.1007/s10044-014-0386-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-014-0386-6

Keywords

Search

Navigation