Skip to main content
SpringerLink
Log in

The design of error-correcting output codes algorithm for the open-set recognition

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The Open-Set recognition is an important topic in the pattern recognition research field. Different from the close-set recognition task, in the open-set recognition problem, the test data contains unknown classes that do not appear in the training phase. Consequently, the recognition of the open-set data is much more difficult than that of the close-set problem. This study applies the Error-Correcting Output Codes (ECOC) framework to handle the open-set problem by dynamically adding new functions to deal with the unknown classes, named ECOC-OS. Our algorithm includes two steps: (1) the unknown data discovery step based on a rejection strategy; (2) the code matrix expanding step for the separation of the unknown classes from the known classes. Due to the wide and chaotic distribution of the unknown class samples, this paper refines the unknown class into multiple sub-classes, and each sub-class has its own feature distribution. After preliminary row and column expansion and class splitting for the unknown class, the clustering algorithm is used to continuously refine the characteristics of the unknown class, dividing it into several sub-classes. Then the algorithm adds multiple coding rows and multiple "one-to-all" basic classifiers, so as to distinguish each unknown sub-class from multiple known classes. Finally, without re-training the existing learners, the zero symbols in the code matrix are selectively re-encoded according to the basic learners’ preference. The experiments deploy 16 data sets for the test, and the results confirm that ECOC-OS algorithm effectively improves the performance compared with other open-set recognition methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

AUS comparison with rejection algorithm

Fig. 13
Fig. 14
Fig. 15

AUS comparison of various open-set recognition algorithms

Fig. 16
Fig. 17

AUS comparison of each data set. a mfeatpix AUS comparison under each algorithm, b mfeatzer AUS comparison under each algorithm, c Pendigits AUS comparison under each algorithm, d mnist AUS comparison under each algorithm

Fig. 18
Fig. 19

AUS of each data set under different ECOC algorithms

Similar content being viewed by others

References

  1. Scheirer WJ, Rocha ADR, Sapkota A, Transactions on Boult E (2013) Toward open set recognition. IEEE Trans Pattern Anal 35(7):1757–1772

  2. Geng C, Huang SJ, Chen S (2020) Recent advances in open set recognition: a survey. IEEE Trans Pattern Anal. https://doi.org/10.1109/TPAMI.2020.2981604

    Article  Google Scholar 

  3. Júnior PRM, Transactions on Boult E, Wainer J, Rocha A (2021) Open-set support vector machines. IEEE Trans Syst Man Cybern Syst pp 1–14

  4. Zhang H, Patel VM (2017) Sparse representation-based open set recognition. IEEE Trans Pattern Anal 39(8):1690–1696

    Article  Google Scholar 

  5. Bendale A, Transactions on Boult (2015) Towards Open World Recognition. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 1893–1902

  6. Hwang J, Oh SW, Lee J-Y, Han B (2021) Exemplar-based open-set panoptic segmentation network. In: IEEE conference on computer vision and pattern recognition

  7. Jo I, Kim J, Kang H, Kim Y, Choi S (2018) Open set recognition by regularising classifier with fake data generated by generative adversarial networks. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp 2686–2690

  8. Zhang W, Li X, Ma H, Luo Z, Li X (2021) Open set domain adaptation in machinery fault diagnostics using instance-level weighted adversarial learning. IEEE Trans Ind Inform 17:7445–7455

    Article  Google Scholar 

  9. Yu X et al (2021) Deep learning-based open set fault diagnosis by extreme value theory. IEEE Trans Ind Inform. https://doi.org/10.1109/TII.2021.3070324

    Article  Google Scholar 

  10. Zhao Y, Li Y, Wang S (2019) Open-world person re-identification with deep hash feature embedding. IEEE Signal Process Lett 26(12):1758–1762

    Article  Google Scholar 

  11. Liu Y, Stehouwer J, Jourabloo A, Liu X (2019) Deep tree learning for zero-shot face anti-spoofing. In: 2019 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 4675–4684

  12. Yang Y, Hou C, Lang Y, Guan D, Huang D, Xu J (2019) Open-set human activity recognition based on micro-doppler signatures. Pattern Recogn 85:60–69

    Article  Google Scholar 

  13. Shu Y, Shi Y, Wang Y, Zou Y, Yuan Q, Tian Y (2018) ODN: opening the deep network for open-set action recognition. In: 2018 IEEE international conference on multimedia and expo (ICME), pp 1–6

  14. Dong Y, Jiang X, Zhou H, Lin Y, Shi Q (2021) SR2CNN: Zero-Shot Learning for Signal Recognition. IEEE Trans Signal Process 69:2316–2329

    Article  MathSciNet  Google Scholar 

  15. Schlachter P, Liao Y, Yang B (2019) Open-set recognition using intra-class splitting. In: 2019 27th European signal processing conference (EUSIPCO), pp 1–5

  16. Xiong H, Lu H, Liu C, Liu L, Cao Z, Shen C (2019) From open set to closed set: counting objects by spatial divide-and-conquer. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp. 8361–8370

  17. Dang S, Cao Z, Cui Z, Pi Y, Liu N (2019) Open set incremental learning for automatic target recognition. IEEE Trans Geosci Remote Sens 57(7):4445–4456

    Article  Google Scholar 

  18. Cevikalp H, Triggs B, Franc V (2013) Face and landmark detection by using cascade of classifiers. In: 2013 10th IEEE international conference and workshops on automatic face and gesture recognition (FG), pp 1–7

  19. Scherreik MD, Rigling BD (2016) Open set recognition for automatic target classification with rejection. IEEE Trans Aerosp Electron Syst 52(2):632–642

    Article  Google Scholar 

  20. Pavy A, Rigling B (2018) SV-means: A fast SVM-based level set estimator for phase-modulated radar waveform classification. IEEE J Sel Top Signal Process 12(1):191–201

    Article  Google Scholar 

  21. Qin J et al. Zero-shot action recognition with error-correcting output codes.

  22. Mendes Júnior PR et al (2017) Nearest neighbors distance ratio open-set classifier. Mach Learn 106(3):359–386

    Article  MathSciNet  Google Scholar 

  23. Henrydoss J, Cruz S, Rudd EM, Gunther M, Transactions on Boult E (2017) Incremental open set intrusion recognition using extreme value machine. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA), pp 1089–1093

  24. Henrydoss J, Cruz S, Li C, Günther M, Transactions on Boult E (2020) Enhancing open-set recognition using clustering-based extreme value machine (C-EVM). In: 2020 IEEE International Conference on Big Data (Big Data), pp 441–448

  25. Jafarzadeh M, Transactions on Ahmad, Dhamija AR, Li C, Cruz S, Transactions on Boult E (2021) Automatic open-world reliability assessment. In: 2021 IEEE winter conference on applications of computer vision (WACV), pp 1983–1992

  26. Vareto R, Silva S, Costa F, Schwartz WR (2017) Towards open-set face recognition using hashing functions. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp 634–641

  27. Bendale A, Transactions on Boult E (2016) Towards open set deep networks. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 1563–1572

  28. Dhamija AR, Günther M, Transactions on Boult E (2018) Reducing network agnostophobia. In: Presented at the proceedings of the 32nd international conference on neural information processing systems, Montréal, Canada

  29. Yoshihashi R, Shao W, Kawakami R, You S, Iida M, Transactions on Naemura (2018) Classification-reconstruction learning for open-set recognition. In: Presented at the proceedings of the IEEE conference on computer vision and pattern recognition

  30. Yang HM, Zhang XY, Yin F, Yang Q, Liu CL (2020) Convolutional prototype network for open set recognition. IEEE Trans Pattern Anal 1:1

    Google Scholar 

  31. Ge Z, Demyanov S, Garnavi R (2017) Generative OpenMax for multi-class open set classification. In: Presented at the 28th British Machine Vision Conference, London

  32. Neal L, Olson M, Fern X, Wong W-K, Li F (2018) Open Set Learning with Counterfactual Images. In: Presented at the Proceedings of the European Conference on Computer Vision (ECCV)

  33. Shahtalebi S, Mohammadi A (2018) Bayesian optimized spectral filters coupled with ternary ECOC for single-trial EEG classification. IEEE Trans Neural Syst Rehabil Eng 26(12):2249–2259

    Article  Google Scholar 

  34. Nazari S, Moin M, Rashidy Kanan H (2018) Securing templates in a face recognition system using Error-Correcting Output Code and chaos theory. Comput Electr Eng 72:644–659

    Article  Google Scholar 

  35. Zhang M, Yu F, Tang C (2017) Disambiguation-free partial label learning. IEEE Trans Knowl Data Eng 29(10):2155–2167

    Article  Google Scholar 

  36. Zhang X-L (2014) Heuristic ternary error-correcting output codes via weight optimization and layered clustering-based approach. IEEE Trans Cybern 45:289–301

    Article  Google Scholar 

  37. Zhou L, Wang Q, Fujita H (2017) One versus one multi-class classification fusion using optimizing decision directed acyclic graph for predicting listing status of companies. Inf Fusion 36:80–89

    Article  Google Scholar 

  38. Lachaize M, HégaratMascle SL, Aldea E, Maitrot A, Reynaud R (2018) Evidential framework for Error-Correcting Output Code classification. Eng Appl Artif Intell 73:10–21

    Article  Google Scholar 

  39. Sun M, Liu K, Wu Q, Hong Q, Wang B, Zhang H (2019) A novel ECOC algorithm for multiclass microarray data classification based on data complexity analysis. Pattern Recogn 90:346–362

    Article  Google Scholar 

  40. Xue A, Wang X, Song Y, Lei L (2017) Discriminant Error-Correcting output codes based on spectral clustering. Pattern Anal Appl 20(3):653–671

    Article  MathSciNet  Google Scholar 

  41. Sun M, Liu K, Hong Q (2017) An ECOC approach for microarray data classification based on minimizing feature related complexities. In: 2017 10th International Symposium on Computational Intelligence and Design (ISCID) 1:300–303

  42. Feng K-J, S.-Transactions on Liong, Liu K-H (2020) The design of variable-length coding matrix for improving Error-Correcting output codes. Inf Sci 534:192–217

  43. Ma C, Tsang IW, Shen F, Liu C (2019) Error-Correcting Input and Output Hashing. IEEE Trans Cybern 49(3):781–791

    Article  Google Scholar 

  44. Lorena AC, Carvalho AC (2007) Evolutionary design of multiclass support vector machines. J Intell Fuzzy Syst 18(5):445–454

    MATH  Google Scholar 

  45. Bautista MÁ, Escalera S, Baró X, Radeva P, Vitriá J, Pujol O (2012) Minimal design of error-correcting output codes. Pattern Recogn Lett 33:693–702

    Article  Google Scholar 

  46. Zhang Y-P, Ye X-N, Liu K-H, Yao J-F (2020) A novel multi-objective genetic algorithm based Error-Correcting output codes. Swarm Evol Comput 57:100709

    Article  Google Scholar 

  47. X.-N. Ye, K.-H. Liu, S.-Transactions on Liong, "A Ternary Bitwise Calculator Based Genetic Algorithm for Improving Error-Correcting Output Codes," Information Sciences,vol. 537, pp. 485–510, 2020.

  48. Zor C, Yanikoglu B, Merdivan E, Transactions on Windeatt, J. Kittler, and E. Alpaydin (2016) BeamECOC: a local search for the optimization of the ECOC matrix. In: 2016 23rd International Conference on Pattern Recognition (ICPR), pp 198–203

  49. Li K-S, Wang H-R, Liu K-H (2019) A novel Error-Correcting Output Codes algorithm based on genetic programming. Swarm Evol Comput 50:100564

    Article  Google Scholar 

  50. Yi-Fan L et al (2021) A novel error-correcting output codes based on genetic programming and ternary digit operators. Pattern Recogn 110:107642

    Article  Google Scholar 

  51. Cramme K, Singer Y (2002) On the learnability and design of output codes for multiclass problems. Mach Learn 47(2–3):201–233

    Article  Google Scholar 

  52. Escalera S, Pujol O, Radeva P (2010) On the decoding process in ternary error-correcting output codes. IEEE Trans Pattern Anal Mach Intell 32(1):120–134

    Article  Google Scholar 

  53. Liu K-H, Zeng Z-H, V. Transactions on Ng Y (2016) A hierarchical ensemble of ECOC for cancer classification based on multi-class microarray data. Inf Sci 349:102–118

  54. Zou J-Y, Sun M-X, Liu K-H, Wu Q-Q (2021) The design of dynamic ensemble selection strategy for the error-correcting output codes family. Inf Sci 571:1–23

    Article  MathSciNet  Google Scholar 

  55. Zhong G, Liu C (2013) Error-correcting output codes based ensemble feature extraction. Pattern Recogn 46(4):1091–1100

    Article  Google Scholar 

  56. Simeone P, Marrocco C, Tortorella F (2012) Design of rejection rules for ECOC classification systems. Pattern Recogn 45(2):863–875

    Google Scholar 

  57. Lei L, Song Y, Luo X (2020) A new re-coding ECOC using rejection option. Appl Intell 50(10):3090–3100

    Article  Google Scholar 

  58. Escalera S, Pujol O, Radeva P (2010) Re-coding ECOCs without re-training. Pattern Recogn Lett 31(7):555–562

    Article  Google Scholar 

  59. Dua D, Karra Taniskidou E (2010) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61772023) and National Key R&D Program of China (No. 2019QY1803).

Author information

Author notes

  1. Kun-Hong Liu and Yi-Fan Liang are co-first authors.

Authors and Affiliations

  1. School of Informatics, Xiamen University, Xiamen, 361005, Fujian Province, China

    Kun-Hong Liu, Wang-Ping Zhan, Yi-Fan Liang, Ya-Nan Zhang, Hong-Zhou Guo, Jun-Feng Yao, Qing-Qiang Wu & Qing-Qi Hong

Authors
  1. Kun-Hong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wang-Ping Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yi-Fan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ya-Nan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hong-Zhou Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jun-Feng Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Qing-Qiang Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Qing-Qi Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Qing-Qiang Wu or Qing-Qi Hong.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 244 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, KH., Zhan, WP., Liang, YF. et al. The design of error-correcting output codes algorithm for the open-set recognition. Appl Intell 52, 7843–7869 (2022). https://doi.org/10.1007/s10489-021-02854-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02854-w

Keywords

Search

Navigation