skip to main content
10.1145/3299815.3314429acmconferencesArticle/Chapter ViewAbstractPublication Pagesacm-seConference Proceedingsconference-collections
research-article

Improving Performance of Convolutional Neural Networks via Feature Embedding

Published: 18 April 2019 Publication History

Abstract

Recently convolutional neural networks (CNN) have shown exceptional performance with data where a feature structure is explicitly defined, for example image data. Real world data is often represented as d dimensional vectors and they lack such feature structure. If features could be embedded into a low dimensional space to introduce feature locality, CNNs could take advantage of the newly introduced feature structure and show better performance. In this paper, we present a technique of feature embedding to introduce feature locality so that non-image data exhibit image like feature structure. We achieve this by embedding features into a 1d or 2d space using t-SNE. We show that CNN performs better under the proposed approach.

References

[1]
M. M. Bronstein, J. Bruna, Y. LeCun, A. Szlam, and P. Vandergheynst. 2017. Geometric Deep Learning: Going beyond Euclidean Data. IEEE Signal Processing Magazine 34, 4 (July 2017), pp. 18--42.
[2]
D. C. Ciresan, U. Meier, and J. Schmidhuber. 2012. Multi-column Deep Neural Networks for Image Classification. 2012 IEEE Conference on Computer Vision and Pattern Recognition (2012), pp. 3642--3649.
[3]
D. Dua and C. Graff. 2017. UCI Machine Learning Repository. http://archive.ics.uci.edu/ml
[4]
Y. Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, A meeting of SIGDAT, a Special Interest Group of the ACL. Doha, Qatar, pp. 1746--1751. http://aclweb.org/anthology/D/D14/D14-1181.pdf
[5]
A. Krizhevsky, I. Sutskever, and G. Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (NIPS'12). Curran Associates Inc., Lake Tahoe, Nevada, USA, pp. 1097--1105. http://dl.acm.org/citation.cfm?id=2999134.2999257
[6]
Y. LeCun, Y. Bengio, and G. Hinton. 2015. Deep learning. Nature 521 (27 May 2015), pp. 436--444.
[7]
Y. LeCun, B.Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel. 1989. Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation 1, 4 (Winter 1989), pp. 541--551.
[8]
Y. LeCun and C. Cortes. 2010. MNIST handwritten digit database. http://yann.lecun.com/exdb/mnist/. (2010). http://yann.lecun.com/exdb/mnist/
[9]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26, C.J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., pp. 3111--3119.
[10]
J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global Vectors forWord Representation. In Empirical Methods in Natural Language Processing (EMNLP). Doha, Qatar, pp. 1532--1543. http://www.aclweb.org/anthology/D14-1162
[11]
L.J.P. van der Maaten and G. Hinton. 2008. Visualizing High-Dimensional Data Using t-SNE. Journal of Machine Learning Research 9 (2008), pp. 2579--2605.

Cited By

View all
  • (2021)Detection of local structures in images using local entropy informationProceedings of the 2021 ACM Southeast Conference10.1145/3409334.3452061(114-121)Online publication date: 15-Apr-2021
  • (2020)Statistical machine translation outperforms neural machine translation in software engineering: why and howProceedings of the 1st ACM SIGSOFT International Workshop on Representation Learning for Software Engineering and Program Languages10.1145/3416506.3423576(3-12)Online publication date: 8-Nov-2020

Index Terms

  1. Improving Performance of Convolutional Neural Networks via Feature Embedding

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ACMSE '19: Proceedings of the 2019 ACM Southeast Conference
      April 2019
      295 pages
      ISBN:9781450362511
      DOI:10.1145/3299815
      • Conference Chair:
      • Dan Lo,
      • Program Chair:
      • Donghyun Kim,
      • Publications Chair:
      • Eric Gamess
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 18 April 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Classification
      2. Convolutional Neural Networks
      3. Feature Embedding
      4. Feature Locality
      5. t-SNE

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      ACM SE '19
      Sponsor:
      ACM SE '19: 2019 ACM Southeast Conference
      April 18 - 20, 2019
      GA, Kennesaw, USA

      Acceptance Rates

      Overall Acceptance Rate 502 of 1,023 submissions, 49%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)19
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2021)Detection of local structures in images using local entropy informationProceedings of the 2021 ACM Southeast Conference10.1145/3409334.3452061(114-121)Online publication date: 15-Apr-2021
      • (2020)Statistical machine translation outperforms neural machine translation in software engineering: why and howProceedings of the 1st ACM SIGSOFT International Workshop on Representation Learning for Software Engineering and Program Languages10.1145/3416506.3423576(3-12)Online publication date: 8-Nov-2020

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media