skip to main content
10.1145/3109761.3158392acmotherconferencesArticle/Chapter ViewAbstractPublication PagesimlConference Proceedingsconference-collections
research-article

Field support vector machines

Published: 17 October 2017 Publication History

Abstract

The identically and independently distributed (i.i.d.) condition required by conventional machine learning approaches may sometimes be violated when patterns occur as groups (where each group shares a homogeneous style, called a field). By breaking it, we extend in this paper the famous Support Vector Machine (SVM) to a novel framework named Field Support Vector Machine (F-SVM), in which the training and predicting a group of patterns (i.e., a field pattern) are performed simultaneously. Specifically, the proposed F-SVM is learned by optimizing simultaneously both the classifier and the Style Normalization Transformation (SNT) for each group of data, even feasible in the high-dimensional kernel space. The SNT transform the original style-discriminative patterns to style-free ones, satisfying the i.i.d. assumption required by the conventional SVM learning and implementation. An efficient optimization algorithm is further developed with the convergence guaranteed theoretically. More importantly, by appropriately exploring the style consistency in each field, the proposed F-SVM model is able to significantly improve the classification accuracy. A series of experiments are conducted to verify the effectiveness and confirmed improvement on the performance of the F-SVM model. Empirical results show that the proposed F-SVM outperforms other relevant baselines in two different benchmark data sets.

References

[1]
Raman Arora, Amitabh Basu, Poorya Mianjy, and Anirbit Mukherjee. 2016. Understanding Deep Neural Networks with Rectified Linear Units. arXiv Preprint arXiv:1611.01491 (2016).
[2]
C. Chang and C. Lin. 2011. LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2 (2011), 27:1--27:27. Issue 3.
[3]
Yinwen Chang, Chojui Hsieh, Kaiwei Chang, Michael Ringgaard, and Chihjen Lin. 2010. Training and Testing Low-degree Polynomial Data Mappings via Linear SVM. Journal of Machine Learning Research 11 (2010), 1471--1490.
[4]
JA Cook and J Ranstam. 2016. Overfitting. British Journal of Surgery 103, 13 (2016), 1814--1814.
[5]
Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine Learning 20, 3 (1995), 273--297.
[6]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: a large-scale hierarchical image database. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 248--255.
[7]
Theodoros Evgeniou and Massimiliano Pontil. 2004. Regularized multi-task learning. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 109--117.
[8]
Nicolas Gourier, Daniela Hall, and James L Crowley. 2004. Estimating face orientation from robust detection of salient facial structures. In FG Net Workshop on Visual Observation of Deictic Gestures. FGnet (IST-2000-26434) Cambridge, UK, 1--9.
[9]
Geoffrey E Hinton, Alexander Krizhevsky, Ilya Sutskever, and Nitish Srivastva. 2016. System and method for addressing overfitting in a neural network. (July 28 2016). US Patent App. 15/222,870.
[10]
Chih-Wei Hsu and Chih-Jen Lin. 2002. A comparison of methods for multiclass support vector machines. Neural Networks, IEEE Transactions on 13,2 (2002), 415--425.
[11]
K. Huang, H. Yang, I. King, and M. R. Lyu. 2008. MaxiÂĂŞmin margin machine: learning large margin classifiers locally and globally. IEEE Transactions on Neural Networks 19, 2 (Feb 2008), 260--272. 2007.905855
[12]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: accelerating deep network training by reducing internal covariate shift. ArXiv Preprint ArXiv:1502.03167 (2015).
[13]
F. Kimura, K. Takashina, S. Tsuruoka, and Y. Miyake. 1987. Modified Quadratic Discriminant Functions and the Application to Chinese Character Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI-9, 1 (Jan 1987), 149--153.
[14]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems (NIPS). 1097--1105.
[15]
C. L. Liu, F. Yin, D. H. Wang, and Q. F. Wang. 2011. CASIA Online and Offline Chinese Handwriting Databases. In Document Analysis and Recognition (ICDAR), 2011 International Conference on. 37--41.
[16]
Cheng-Lin Liu and Xiang-Dong Zhou. 2006. Online Japanese character recognition using trajectory-based normalization and direction feature extraction. In Tenth International Workshop on Frontiers in Handwriting Recognition. Suvisoft.
[17]
Michael J Lyons, Shota Akamatsu, Miyuki Kamachi, and Jiro Gyoba. 1998. Coding facial expressions with Gabor wavelets. (1998), 200--205.
[18]
John Platt and others. 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 (1999), 61--74.
[19]
P. Sarkar and G. Nagy. 2005. Style consistent classification of isogenous patterns. Pattern Analysis and Machine Intelligence, IEEE Transactions on 27, 1 (Jan 2005), 88--98.
[20]
Wenling Shang, Kihyuk Sohn, Diogo Almeida, and Honglak Lee. 2016. Understanding and improving convolutional neural networks via concatenated rectified linear units. In Proceedings of the International Conference on Machine Learning (ICML).
[21]
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929--1958.
[22]
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2818--2826.
[23]
Joshua B Tenenbaum and William T Freeman. 2000. Separating style and content with bilinear models. Neural computation 12, 6 (2000), 1247--1283.
[24]
Singluar. Veeramachaneni and G. Nagy. 2005. Style context with second-order statistics. Pattern Analysis and Machine Intelligence, IEEE Transactions on 27, 1 (Jan 2005), 14--22.
[25]
Michael E Wall, Andreas Rechtsteiner, and Luis M Rocha. 2003. Singular value decomposition and principal component analysis. In A Practical Approach to Microarray Data Analysis. Springer, 91--109.
[26]
Ting-Fan Wu, Chih-Jen Lin, and Ruby C Weng. 2004. Probability estimates for multi-class classification by pairwise coupling. The Journal of Machine Learning Research 5 (2004), 975--1005.
[27]
Tong Xiao, Hongsheng Li, Wanli Ouyang, and Xiaogang Wang. 2016. Learning deep feature representations with domain guided dropout for person re-identification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1249--1258.
[28]
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. 2016. Understanding deep learning requires rethinking generalization. ArXiv Preprint ArXiv:1611.03530 (2016).
[29]
Xu-Yao Zhang, Kaizhu Huang, and Cheng-Lin Liu. 2011. Pattern field classification with style normalized transformation. In International Joint Conference on Artificial Intelligence (IJCAI). AAAI Press, 1621--1626.

Cited By

View all
  • (2019)Style-Neutralized Pattern Classification Based on Adversarially Trained Upgraded U-NetCognitive Computation10.1007/s12559-019-09660-0Online publication date: 7-Sep-2019

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
IML '17: Proceedings of the 1st International Conference on Internet of Things and Machine Learning
October 2017
581 pages
ISBN:9781450352437
DOI:10.1145/3109761
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2017

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Funding Sources

  • Natural science fund for colleges and universities in Jiangsu Province
  • Suzhou Science and Technology Programme
  • National Natural Science Foundation of China (NSFC)

Conference

IML 2017

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Style-Neutralized Pattern Classification Based on Adversarially Trained Upgraded U-NetCognitive Computation10.1007/s12559-019-09660-0Online publication date: 7-Sep-2019

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media