research-article

Chinese Character Recognition with Augmented Character Profile Matching

Authors:

Xiangyang XueAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 6094 - 6102

https://doi.org/10.1145/3503161.3547827

Published: 10 October 2022 Publication History

Abstract

Chinese character recognition (CCR) has drawn continuous research interest due to its wide applications. After decades of study, there still exist several challenges,e.g., different characters with similar appearance and the one-to-many problem. There is no unified solution to the above challenges as previous methods tend to address these problems separately. In this paper, we propose a Chinese character recognition method named Augmented Character Profile Matching (ACPM), which utilizes a collection of character knowledge from three decomposition levels to recognize Chinese characters. Specifically, the feature maps of each character image are utilized as the character-level knowledge. In addition, we introduce a radical-stroke counting module (RSC) to help produce augmented character profiles, including the number of radicals, the number of strokes, and the total length of strokes, which characterize the character more comprehensively. The feature maps of the character image and the outputs of the RSC module are collected to constitute a character profile for selecting the closest candidate character through joint matching. The experimental results show that the proposed method outperforms the state-of-the-art methods on both the ICDAR 2013 and CTW datasets by 0.35% and 2.23%, respectively. Moreover, it also clearly outperforms the compared methods in the zero-shot settings. Code is available at https://github.com/FudanVI/FudanOCR/tree/main/character-profile-matching.

Supplementary Material

MP4 File (MM22-fp0357.mp4)

Presentation video of Chinese Character Recognition with Augmented Character Profile Matching

Download
30.64 MB

References

[1]

Zhong Cao, Jiang Lu, Sen Cui, and Changshui Zhang. 2020. Zero-Shot Handwritten Chinese Character Recognition with Hierarchical Decomposition Embedding. Pattern Recognition 107 (2020), 107488.

[2]

Fu Chang. 2006. Techniques for Solving the Large-Scale Classification Problem in chinese handwriting recognition. In SACHR. 161--169.

[3]

Jingye Chen, Bin Li, and Xiangyang Xue. 2021. Zero-Shot Chinese Character Recognition with Stroke-Level Decomposition. In IJCAI. 615--621.

[4]

Jingye Chen, Haiyang Yu, Jianqi Ma, Bin Li, and Xiangyang Xue. 2022. Text Gestalt: Stroke-Aware Scene Text Image Super-Resolution. In AAAI, Vol. 36. 285--293.

[5]

Xiaoxue Chen, Lianwen Jin, Yuanzhi Zhu, Canjie Luo, and Tianwei Wang. 2021. Text Recognition in the Wild: A Survey. Comput. Surveys 54, 2 (2021), 1--35.

[6]

Dan Ciresan and Ueli Meier. 2015. Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification. In IJCNN. 1--6.

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770--778.

[8]

Gao Huang, Zhuang Liu, Van Der Maaten Laurens, and Weinberger Kilian Q. 2017. Densely Connected Convolutional Networks. In CVPR. 4700--4708.

[9]

Lianwen Jin, Junxun Yin, Xue Gao, and Jiangcheng Huang. 2001. Study of Several Directional Feature Extraction Methods with Local Elastic Meshing Technology for HCCR. In the Sixth Int. Conference for Young Computer Scientist. 232--236.

[10]

In Jung Kim, Chenglin Liu, and Jin Hyung Kim. 1999. Stroke-Guided Pixel Matching for Handwritten Chinese Character Recognition. In ICDAR. 665--668.

[11]

Jin Wook Kim, Kwang In Kim, Bong Joon Choi, and Hang Joon Kim. 1999. Decomposition of Chinese Character into Strokes Using Mathematical Morphology. Pattern Recognition Letters 20, 3 (1999), 285--292.

[12]

Chenglin Liu, In Jung Kim, and Jin H. Kim. 2001. Model-Based Stroke Extraction and Matching for Handwritten Chinese Character Recognition. Pattern Recognition 34, 12 (2001), 2339--2352.

Digital Library

[13]

Chenglin Liu, Fei Yin, DahanWang, and QiufengWang. 2013. Online and Offline Handwritten Chinese Character Recognition: Benchmarking on New Databases. Pattern Recognition 46, 1 (2013), 155--162.

Digital Library

[14]

Daming Shi, Steve R. Gunn, and Robert I. Damper. 2003. Handwritten Chinese Radical Recognition Using Nonlinear Active Shape Models. IEEE Transactions on Pattern Analysis and Machine Intelligence 25, 2 (2003), 277--280.

Digital Library

[15]

Yih Ming Su and Jhing Fa Wang. 2003. A Novel Stroke Extraction Method for Chinese Characters Using Gabor Filters. Pattern Recognition 36, 3 (2003), 635--647.

[16]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. NeurIPS 30 (2017).

[17]

AnbangWang, Kuo Chin Fan, andWei HsienWu. 1996. A Recursive Hierarchical Scheme for Radical Extraction of Handwritten Chinese Characters. In ICPR, Vol. 3. 240--244.

[18]

Tianwei Wang, Zecheng Xie, Zhe Li, Lianwen Jin, and Xiangle Chen. 2019. Radical Aggregation Network for Few-Shot Offline Handwritten Chinese Character Recognition. Pattern Recognition Letters 125 (2019), 821--827.

Digital Library

[19]

Wenchao Wang, Jianshu Zhang, Jun Du, Zirui Wang, and Yixing Zhu. 2018. Denseran for Offline Handwritten Chinese Character Recognition. In ICFHR. 104--109.

[20]

Chunpeng Wu, Wei Fan, Yuan He, Jun Sun, and Satoshi Naoi. 2014. Handwritten Character Recognition by Alternately Trained Relaxation Convolutional Neural Network. In ICFHR. 291--296.

[21]

Changjie Wu, Zirui Wang, Jun Du, Jianshu Zhang, and Jiaming Wang. 2019. Joint Spatial and Radical Analysis Network for Distorted Chinese Character Recognition. In ICDARW, Vol. 5. 122--127.

[22]

Yao Xiao, Dan Meng, Cewu Lu, and Chi Keung Tang. 2019. Template-Instance Loss for Offline Handwritten Chinese Character Recognition. In ICDAR. 315--322.

[23]

Yun Yang, Donghai Li, and Zongtao Duan. 2018. Chinese Vehicle License Plate Recognition Using Kernel-Based Extreme Learning Machine with Deep Convolutional Features. IET Intelligent Transport Systems 12, 3 (2018), 213--219.

[24]

Fei Yin, Qiufeng Wang, Xuyao Zhang, and Chenglin Liu. 2013. ICDAR 2013 Chinese Handwriting Recognition Competition. In ICDAR. 1464--1470.

[25]

Tailing Yuan, Zhe Zhu, Kun Xu, Chengjun Li, Taijiang Mu, and ShiMin Hu. 2019. A Large Chinese Text Dataset in the Wild. Journal of Computer Science and Technology 34, 3 (2019), 509--521.

[26]

Matthew D. Zeiler. 2012. Adadelta: an Adaptive Learning Rate Method. arXiv:1212.5701 (2012).

[27]

Chongsheng Zhang, Weiping Ding, Guowen Peng, Feifei Fu, and Wei Wang. 2020. Street View Text Recognition with Deep Learning for Urban Scene Understanding in Intelligent Transportation Systems. IEEE Transactions on Intelligent Transportation Systems 22, 7 (2020), 4727--4743.

Digital Library

[28]

Zhuoyao Zhong, Lianwen Jin, and Zecheng Xie. 2015. High Performance Offline Handwritten Chinese Character Recognition Using Googlenet and Directional Feature Maps. In ICDAR. 846--850.

Cited By

Luo GWang DZhang XLin ZZhu S(2025)Joint radical embedding and detection for zero-shot Chinese character recognitionPattern Recognition10.1016/j.patcog.2024.111286161(111286)Online publication date: May-2025
https://doi.org/10.1016/j.patcog.2024.111286
Hu PMa JZhang ZDu JZhang J(2025)Count, decompose and correct: A new approach to handwritten Chinese character error correctionPattern Recognition10.1016/j.patcog.2024.111110160(111110)Online publication date: Apr-2025
https://doi.org/10.1016/j.patcog.2024.111110
Li ZZhao HNishizaki HLeow CShen X(2025)Chinese Character Recognition based on Swin Transformer-EncoderDigital Signal Processing10.1016/j.dsp.2025.105080(105080)Online publication date: Feb-2025
https://doi.org/10.1016/j.dsp.2025.105080
Show More Cited By

Index Terms

Chinese Character Recognition with Augmented Character Profile Matching
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
      2. Computer vision tasks

Recommendations

Attributed String Matching by Split-and-Merge for On-Line Chinese Character Recognition

Consecutive strokes of Chinese characters tend to be connected in fast writing, and this causes a problem for most stroke-based recognition approaches. A recognition scheme for recognizing cursive Chinese characters under the constraint of correct ...
Character and numeral recognition for non-Indic and Indic scripts: a survey
Abstract
A collection of different scripts is employed in writing languages throughout the world. Character and numeral recognition of a particular script is a key area in the field of pattern recognition. In this paper, we have presented a comprehensive ...
Hippocampus-heuristic character recognition network for zero-shot learning in Chinese character recognition
Highlights
- A novel hippocampus-heuristic character recognition network (HCRN) is proposed for zero/few-shot learning.
Abstract
The recognition of Chinese characters has always been a challenging task due to their huge variety and complex structures. The current radical-based methods fail to recognize Chinese characters without learning all of their radicals in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
298
Total Downloads

Downloads (Last 12 months)98
Downloads (Last 6 weeks)7

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Luo GWang DZhang XLin ZZhu S(2025)Joint radical embedding and detection for zero-shot Chinese character recognitionPattern Recognition10.1016/j.patcog.2024.111286161(111286)Online publication date: May-2025
https://doi.org/10.1016/j.patcog.2024.111286
Hu PMa JZhang ZDu JZhang J(2025)Count, decompose and correct: A new approach to handwritten Chinese character error correctionPattern Recognition10.1016/j.patcog.2024.111110160(111110)Online publication date: Apr-2025
https://doi.org/10.1016/j.patcog.2024.111110
Li ZZhao HNishizaki HLeow CShen X(2025)Chinese Character Recognition based on Swin Transformer-EncoderDigital Signal Processing10.1016/j.dsp.2025.105080(105080)Online publication date: Feb-2025
https://doi.org/10.1016/j.dsp.2025.105080
Cai HZhu A(2024)Cross-Modal Alignment of Local and Global Features for Zero-Shot Chinese Character Recognition2024 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP51287.2024.10647599(2041-2047)Online publication date: 27-Oct-2024
https://doi.org/10.1109/ICIP51287.2024.10647599
Ye-eun K(2024)A Review of the Current Status of AI Research in Handwritten Chinese Character RecognitionSHS Web of Conferences10.1051/shsconf/202418501004185(01004)Online publication date: 14-Mar-2024
https://doi.org/10.1051/shsconf/202418501004
Ishikawa TMiyazaki TOmachi S(2024)Japanese historical character recognition by focusing on character partsPattern Recognition10.1016/j.patcog.2023.110181148:COnline publication date: 17-Apr-2024
https://dl.acm.org/doi/10.1016/j.patcog.2023.110181
Yin XYang CLiu CYin XYang CLiu C(2024)Open-Set Text Recognition Implementations(III): Open-set PredictorOpen-Set Text Recognition10.1007/978-981-97-0361-6_6(79-86)Online publication date: 2-Apr-2024
https://doi.org/10.1007/978-981-97-0361-6_6
Yin XYang CLiu CYin XYang CLiu C(2024)Open-Set Text Recognition Implementations(I): Label-to-Representation MappingOpen-Set Text Recognition10.1007/978-981-97-0361-6_4(53-65)Online publication date: 2-Apr-2024
https://doi.org/10.1007/978-981-97-0361-6_4
Yin XYang CLiu CYin XYang CLiu C(2024)Open-Set Text Recognition: Concept, Dataset, Protocol, and FrameworkOpen-Set Text Recognition10.1007/978-981-97-0361-6_3(27-52)Online publication date: 2-Apr-2024
https://doi.org/10.1007/978-981-97-0361-6_3
Pan SWang DJiang NZhang XZhu S(2024)Learning Explicit Radical Representations for Zero-Shot Chinese Character RecognitionPattern Recognition10.1007/978-3-031-78119-3_21(303-317)Online publication date: 5-Dec-2024
https://doi.org/10.1007/978-3-031-78119-3_21
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten