skip to main content
research-article

Field Effect Deep Networks for Image Recognition with Incomplete Data

Published: 03 August 2016 Publication History

Abstract

Image recognition with incomplete data is a well-known hard problem in computer vision and machine learning. This article proposes a novel deep learning technique called Field Effect Bilinear Deep Networks (FEBDN) for this problem. To address the difficulties of recognizing incomplete data, we design a novel second-order deep architecture with the Field Effect Restricted Boltzmann Machine, which models the reliability of the delivered information according to the availability of the features. Based on this new architecture, we propose a new three-stage learning procedure with field effect bilinear initialization, field effect abstraction and estimation, and global fine-tuning with missing features adjustment. By integrating the reliability of features into the new learning procedure, the proposed FEBDN can jointly determine the classification boundary and estimate the missing features. FEBDN has demonstrated impressive performance on recognition and estimation tasks in various standard datasets.

References

[1]
André Aleman, Koen B. E. Böcker, Ron Hijman, Edward H. F. de Haanb, and René S. Kahna. 2003. Cognitive basis of hallucinations in schizophrenia: Role of top-down information processing. Schizophr. Res. 64, 2--3, 178--185.
[2]
Pradeep K. Atrey, M. Anwar Hossain, Abdulmotaleb El Saddik, and Mohan S. Kankanhalli. 2010. Multimodal fusion for multimedia analysis: a survey. Multimedia Syst. 16, 345--379.
[3]
Bernhard E. Boser, Isabelle M. Guyon, and Vladimir N. Vapnik. 1992. A training algorithm for optimal margin classifiers. In COLT. ACM, New York, NY, 144--152.
[4]
Gal Chechik, Geremy Heitz, Gal Elidan, Pieter Abbeel, and Daphne Koller. 2006. Max-margin classification of incomplete data. In NIPS.
[5]
Gal Chechik, Geremy Heitz, Gal Elidan, Pieter Abbeel, and Daphne Koller. 2008. Max-margin classification of data with absent features. J. Mach. Learn. Res. 9, 1--21.
[6]
Hao Chen, Dong Ni, Jing Qin, Shengli Li, Xin Yang, Tianfu Wang, and Pheng Ann Heng. 2015. Standard plane localization in fetal ultrasound via domain transferred deep neural networks. JBHI 19, 5, 1627--1636.
[7]
Yanjiao Chen, Kaishun Wu, and Qian Zhang. 2015. From QoS to QoE: A tutorial on video quality assessment. IEEE Commun. Surv. Tutorials 17, 2, 1126--1165.
[8]
Uwe Dick, Peter Haider, and Tobias Scheffer. 2008. Learning from incomplete data with infinite imputations. In ICML. Citeseerx, Helsinki, Finland, 232--239.
[9]
Huijun Ding, Tan Lee, Ing Yann Soon, Chai Kiat Yeo, Peng Dai, and Guo Dan. 2015. Objective measures for quality assessment of noise-suppressed speech. Speech Commun. 71, 62--73.
[10]
Laura Folguera, Jure Zupan, Daniel Cicerone, and Jorge F. Magallanes. 2015. Self-organizing maps for imputation of missing data in incomplete data matrices. Chemometr. Intell. Lab. 143, 146--151.
[11]
Geoffrey E. Hinton and Roweis R. Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786, 504--507.
[12]
Oliver Jesorsky, Klaus J. Kirchberg, and Robert Frischholz. 2001. Robust face detection using the Hausdorff distance. In AVBPA. Springer-Verlag, London, UK, 90--95.
[13]
Alex Krizhevsky and Geoffrey E. Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report. University of Toronto.
[14]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. 2012. ImageNet classification with deep convolutional neural networks. In NIPS.
[15]
Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y. Ng. 2011. Unsupervised learning of hierarchical representations with convolutional deep belief networks. Commun. ACM. 54, 10, 95--103.
[16]
Xuejun Liao, Hui Li, and Lawrence Carin. 2007. Quadratically gated mixture of experts for incomplete data classification. In ICML. ACM, New York, NY, 553--560.
[17]
Norbert R. Malik. 1995. Electronic Circuits: Analysis, Simulation, and Design. Prentice-Hall, Upper Saddle River, NJ.
[18]
Prabhu Natarajan, Pradeep K. Atrey, and Mohan Kankanhalli. 2015. Multi-camera coordination and control in surveillance systems: a survey. ACM TOMM. 11, 4, Article 57, 30.
[19]
Marc’aurelio Ranzato, Joshua M. Susskind, Volodymyr Mnih, and Geoffrey E. Hinton. 2011. On deep generative models with applications to recognition. In CVPR. 2857--2864.
[20]
Yann LeCun, Léeon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 11, 2278--2324.
[21]
Yann LeCun, Yoshua Bengio, and Geoffrey E. Hinton. 2015. Deep learning. Nature 521, 436--444.
[22]
Kun Li, Jingyu Yang, and Jianmin Jiang. 2015. Nonrigid structure from motion via sparse representation. IEEE Trans. Cybern. 45, 8, 1401--1413.
[23]
Archana Purwar and Sandeep Kumar Singh. 2015. Hybrid prediction model with missing value imputation for media data. ESWA. 42, 5621--5631.
[24]
Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton. 2007. Restricted Boltzmann machines for collaborative filtering. In ICML. ACM, New York, NY, 791--798.
[25]
Ruslan Salakhutdinov and Geoffrey E. Hinton. 2007. Learning a nonlinear embedding by preserving class neighbourhood structure. In AISTATS. Omnipress, San Juan, Puerto Rico, 412--419.
[26]
Jürgen Schmidhuber. 2014. Deep learning in neural networks. Technical Report, 61, 85--117.
[27]
Kihyuk Sohn, Guanyu Zhou, Chansoo Lee, and Honglak Lee. 2013. Learning and selecting features jointly with point-wise gated boltzmann machines. In ICML. Citeseerx, Atlanta, GA, 217--225.
[28]
Charlie Tang and Chris Eliasmith. 2010. Deep networks for robust visual recognition. In ICML. ACM, 1055--1062.
[29]
Neill R. Taylor, Christo Panchev, Matthew Hartley, Stathis Kasderidis, and John G. Taylor. 2006. Occlusion, attention and object representations. In ICANN. Springer-Verlag, Athens, Greece, 592--601.
[30]
Jason Weston, Frédéric Ratle, and Ronan Collobert. 2008. Deep learning via semi-supervised embedding. In ICML. Springer, Berlin, 639--655.
[31]
David Williams, Xuejun Liao, Ya Xue, Lawrence Carin, and Balaji Krishnapuram. 2007. On classification with incomplete data. IEEE TPAMI. 29, 3, 427--436.
[32]
David Williams, Xuejun Liao, Ya Xue, and Lawrence Carin. 2005. Incomplete-data classification using logistic regression. In ICML. ACM, New York, NY, 972--979.
[33]
Hao-tian Wu, Jiwu Huang, and Yun-Qing Shi. 2015. A reversible data hiding method with contrast enhancement for medical images. J. Vis. Commun. Image R. 31, 146--153.
[34]
Wanmin Wu, Ahsan Arefin, Raoul Rivas, Klara Nahrstedt, and Renata M. Sheppard. 2009. Quality of experience in distributed interactive multimedia environments: toward a theoretical framework. In ACM MM. 1--10.
[35]
Xiaoshan Yang, Tianzhu Zhang, and Changsheng Xu. 2015. Boosted multifeature learning for cross-domain transfer. ACM TOMM. Appl. 11, 3, Article 35, 18.
[36]
Quanzeng You, Jiebo Luo, Hailin Jin, and Jianchao Yang. 2015. Robust image sentiment analysis using progressively trained and domain transferred deep networks. In AAAI.
[37]
Sheng-hua Zhong, Yan Liu, and Yang Liu. 2011. Bilinear deep learning for image classification. In ACMMM. ACM, New York, NY, 343--352.
[38]
Sheng-hua Zhong, Yan Liu, Fu-lai Chung, and Gangshan Wu. 2012. Semiconducting bilinear deep learning for incomplete image recognition. In ICMR. ACM, New York, NY, Article 32.
[39]
Sheng-hua Zhong, Yan Liu, Bin Li, and Jing Long. 2015. Query-oriented unsupervised multi-document summarization via deep learning model. ESWA. 42, 21, 8146--8155.
[40]
Mingyuan Zhou, Haojun Chen, John Paisley, Lu Ren, Lingbo Li, Zhengming Xing, David Dunson, Guillermo Sapiro, and Lawrence Carin. 2012. Nonparametric bayesian dictionary learning for analysis of noisy and incomplete images. TIP. 21, 1, 2012.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications
ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 12, Issue 4
August 2016
219 pages
ISSN:1551-6857
EISSN:1551-6865
DOI:10.1145/2983297
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 August 2016
Accepted: 01 May 2016
Revised: 01 May 2016
Received: 01 December 2015
Published in TOMM Volume 12, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Image recognition
  2. deep learning
  3. incomplete data
  4. missing features

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Shenzhen University research funding
  • Special Program for Applied Research on Super Computation of the NSFC-Guangdong Joint Fund
  • National Natural Science Foundation of China
  • Natural Science Foundation of Guangdong Province
  • Science and Technology Innovation Commission of Shenzhen under Grant

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Local tangent space transfer and alignment for incomplete dataKnowledge-Based Systems10.1016/j.knosys.2024.112880(112880)Online publication date: Dec-2024
  • (2022)Privacy Intelligence: A Survey on Image Privacy in Online Social NetworksACM Computing Surveys10.1145/354729955:8(1-35)Online publication date: 13-Jul-2022
  • (2020)Convolutional Neural Network-Based Methods for Eye Gaze Estimation: A SurveyIEEE Access10.1109/ACCESS.2020.30135408(142581-142605)Online publication date: 2020
  • (2020)Image Target Recognition Model of Multichannel Structure Convolutional Neural Network Training Automatic EncoderIEEE Access10.1109/ACCESS.2020.3003059(1-1)Online publication date: 2020
  • (2019)Video summarization via spatio-temporal deep architectureNeurocomputing10.1016/j.neucom.2018.12.040332(224-235)Online publication date: Mar-2019
  • (2019)A role of computer vision in fruits and vegetables among various horticulture products of agriculture fields: A SurveyInformation Processing in Agriculture10.1016/j.inpa.2019.07.003Online publication date: Jul-2019
  • (2019)Fine-art painting classification via two-channel dual path networksInternational Journal of Machine Learning and Cybernetics10.1007/s13042-019-00963-0Online publication date: 18-May-2019
  • (2018)Incomplete-Data Oriented Multiview Dimension Reduction via Sparse Low-Rank RepresentationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2018.282869929:12(6276-6291)Online publication date: Dec-2018
  • (2018)Handling data irregularities in classification: Foundations, trends, and future challengesPattern Recognition10.1016/j.patcog.2018.03.00881(674-693)Online publication date: Sep-2018
  • (2018)Local tangent space alignment via nuclear norm regularization for incomplete dataNeurocomputing10.1016/j.neucom.2017.07.055273(141-151)Online publication date: Jan-2018
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media