research-article

Fine-Grained Visual Computing Based on Deep Learning

Authors:

Amit Kumar Singh,

Qingjun WangAuthors Info & Claims

ACM Transactions on Multimidia Computing Communications and Applications, Volume 17, Issue 1s

Article No.: 16, Pages 1 - 19

https://doi.org/10.1145/3418215

Published: 26 April 2021 Publication History

Abstract

With increasing amounts of information, the image information received by people also increases exponentially. To perform fine-grained categorization and recognition of images and visual calculations, this study combines the Visual Geometry Group Network 16 model of convolutional neural networks and the vision attention mechanism to build a multi-level fine-grained image feature categorization model. Finally, the TensorFlow platform is utilized to simulate the fine-grained image classification model based on the visual attention mechanism. The results show that in terms of accuracy and required training time, the fine-grained image categorization effect of the multi-level feature categorization model constructed by this study is optimal, with an accuracy rate of 85.3% and a minimum training time of 108 s. In the similarity effect analysis, it is found that the chi-square distance between Log Gabor features and the degree of image distortion show a strong positive correlation; in addition, the validity of this measure is verified. Therefore, through the research in this study, it is found that the constructed fine-grained image categorization model has higher accuracy in image recognition categorization, shorter training time, and significantly better performance in similar feature effects, which provides an experimental reference for the visual computing of fine-grained images in the future.

References

[1]

L. Chen, S. Wang, K.-M. Lam, H. Zhou, M. Jian, and J. Dong. 2019. Cascade one-vs-rest detection network for fine-grained recognition without part annotations. Multimedia Tools and Applications 78, 4 (2019), 4381–4395.

Digital Library

[2]

Z. J. Zha, D. Liu, H. Zhang, Y. D. Zhang, and F. Wu. 2019. Context-aware visual policy network for fine-grained image captioning. arXiv:1906.02365

Digital Library

[3]

Y. Q. Huang, J. S. Chen, W. L. Ouyang, W. T. Wan, and Y. Z. Xue. 2020. Image captioning with end-to-end attribute detection and subsequent attributes prediction. IEEE Transactions on Image Processing 29 (2020), 4013–4026.

Digital Library

[4]

T. Soenen, W. Tavernier, M. Peuster, F. Vicens, G. Xilouris, S. Kolometsos, M. A. Kourtis, and D. Colle. 2019. Empowering network service developers: Enhanced NFV DevOps and programmable MANO. IEEE Communications Magazine 57, 5 (2019), 89–95.

[5]

A. Wentzel, P. Hanula, T. Luciani, B. Elgohari, H. Elhalawani, G. Canahuate, D. Vock, C. D. Fuller, and G. E. Marai. 2019. Cohort-based T-SSIM visual computing for radiation therapy prediction and exploration. arXiv:1907.05919

[6]

X. Shu, J. Tang, G. J. Qi, Z. Li, Y. G. Jiang, and S. Yan. 2018. Image classification with tailored fine-grained dictionaries. IEEE Transactions on Circuits and Systems for Video Technology 28, 2 (2018), 454–467.

[7]

L. Wu, Y. Wang, X. Li, and J. Gao. 2018. Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Transactions on Cybernetics 49, 5 (2018), 1791–1802.

[8]

X. Zhou, K. Jin, Y. Shang, and G. Guo. 2020. Visually interpretable representation learning for depression recognition from facial images. IEEE Transactions on Affective Computing 11, 3 (2020), 542–552.

[9]

Z. Q. Wei, M. Xu, L. Geng, H. M. Liu, and H. Yin. 2019. Adversarial similarity metric learning for kinship verification. IEEE Access 7 (2019), 100029–100035.

[10]

X. S. Wei, J. H. Luo, J. Wu, and Z. H. Zhou. 2017. Selective convolutional descriptor aggregation for fine-grained image retrieval. IEEE Transactions on Image Processing 26, 6 (2017), 2868–2881.

Digital Library

[11]

J. Fu, J. Xue, Y. Wang, Z. Liu, and C. Shan. 2018. Malware visualization for fine-grained classification. IEEE Access 6 (2018), 14510–14523.

[12]

I. Baz, E. Yoruk, and M. Cetin. 2019. Context-aware confidence sets for fine-grained product recognition. IEEE Access 7 (2019), 76376–76393.

[13]

S. Min, H. Yao, H. Xie, Z. J. Zha, and Y. Zhang. 2020. Multi-objective matrix normalization for fine-grained visual recognition. IEEE Transactions on Image Processing 29 (2020), 4996–5009.

Digital Library

[14]

W. Abdul, Z. Ali, S. Ghouzali, B. Alfawaz, G. Muhammad, and M. S. Hossain. 2017. Biometric security through visual encryption for fog edge computing. IEEE Access 5 (2017), 5531–5538.

[15]

A. Wentzel, P. Hanula, T. Luciani, B. Elgohari, H. Elhalawani, G. Canahuate, and G. E. Marai. 2019. Cohort-based T-SSIM visual computing for radiation therapy prediction and exploration. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 949–959.

[16]

A. Kaneko, A. Komatsu, T. Itoh, and F. Y. Wang. 2020. Painting image browser applying an associate-rule-aware multidimensional data visualization technique. Visual Computing for Industry, Biomedicine, and Art 3, 1 (2020), 1–13.

[17]

Z. Zhong, J. Li, Z. Luo, and M. Chapman. 2017. Spectral–spatial residual network for hyperspectral image classification: A 3-D deep learning framework. IEEE Transactions on Geoscience and Remote Sensing 56, 2 (2017), 847–858.

[18]

L. Zhu, Y. Chen, P. Ghamisi, and J. A. Benediktsson. 2018. Generative adversarial networks for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 56, 9 (2018), 5046–5063.

[19]

H. Bi, F. Xu, Z. Wei, Y. Xue, and Z. Xu. 2019. An active deep learning approach for minimally supervised PolSAR image classification. IEEE Transactions on Geoscience and Remote Sensing 57, 11 (2019), 9378–9395.

[20]

D. Liu, B. Wen, J. Jiao, X. Liu, Z. Wang, and T. S. Huang. 2020. Connecting image denoising and high-level vision tasks via deep learning. IEEE Transactions on Image Processing 29 (2020), 3695–3706.

Digital Library

[21]

D. J. Hemanth, L. Wang, João Manuel R. S. Tavares, F. Shi, and V. V. Estrela. 2018. IEEE Access special section editorial: Soft computing techniques for image analysis in the medical industry—current trends, challenges and solutions. IEEE Access 6 (2018), 39487–39489.

[22]

D. Park, S. Kim, J. Lee, J. Choo, N. Diakopoulos, and N. Elmqvist. 2018. ConceptVector: Text visual analytics via interactive lexicon building using word embedding. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 361–370.

[23]

H. Strobelt, S. Gehrmann, M. Behrisch, A. Perer, H. Pfister, and A. M. Rush. 2019. Seq2Seq-Vis: A visual debugging tool for sequence-to-sequence models. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2019), 353–363.

Digital Library

[24]

F. Hohman, M. Kahng, R. Pienta, and D. H. Chau. 2019. Visual analytics in deep learning: An interrogative survey for the next frontiers. IEEE Transactions on Visualization and Computer Graphics 25, 8 (2019), 2674–2693.

Digital Library

[25]

M. Khayat, M. Karimzadeh, D. S. Ebert, and A. Ghafoor. 2020. The validity, generalizability and feasibility of summative evaluation methods in visual analytics. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 353–363.

[26]

Z. Zhou, L. Meng, C. Tang, Y. Zhao, and W. Chen. 2018. Visual abstraction of large scale geospatial origin-destination movement data. IEEE Transactions on Visualization and Computer Graphics 25, 1 (2018), 43–53.

Digital Library

[27]

Y. H. Liu, C. Shi, Q. F. Wu, R. M. Zhang, and Z. G. Zhou. 2019. Visual analytics of stratigraphic correlation for multi-attribute well-logging data exploration. IEEE Access 7 (2019), 98122–98135.

[28]

D. Seebacher, M. Miller, T. Polk, J. Fuchs, and D. A. Keim. 2019. Visual analytics of volunteered geographic information: Detection and investigation of urban heat islands. IEEE Computer Graphics and Applications 39, 5 (2019), 83–95.

Digital Library

[29]

C. Shi, H. X. Wang, B. J. Chen, Y. H. Liu, and Z. G. Zhou. 2019. Visual analysis of citation context-based article influence ranking. IEEE Access 7 (2019), 113853–113866.

[30]

H. H. Mei, W. Chen, Y. T. Wei, Y. Z. Hu, S. Y. Zhou, B. R. Lin, Y. Zhao, and J. Z. Xia. 2020. RSATree: Distribution-aware data representation of large-scale tabular datasets for flexible visual query. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 1161–1171.

[31]

S. Hazarika, H. Li, K. C. Wang, H. W. Shen, and C. S. Chou. 2020. NNVA: Neural network assisted visual analysis of yeast cell polarization simulation. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 34–44.

[32]

M. Y. Pan, Y. S. Liu, J. Y. Cao, Y. Li, C. Li, and C. H. Chen. 2020. Visual recognition based on deep learning for navigation mark classification. IEEE Access 8 (2020), 32767–32775.

[33]

L. Wu, Y. Wang, X. Li, and J. Gao. 2018. Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Transactions on Cybernetics 49, 5 (2018), 1791–1802.

[34]

P. Rodríguez, D. Velazquez, G. Cucurull, J. M. Gonfaus, F. X. Roca, and J. Gonzàlez. 2020. Pay attention to the activations: A modular attention mechanism for fine-grained image recognition. IEEE Transactions on Multimedia 22, 2 (2020), 502–514.

Digital Library

[35]

Q. F. Wang, X. H. Zhou, C. Wang, Z. Q. Liu, J. Huang, Y. Zhou, C. L. Li, H. Zhuang, and J. Z. Cheng. 2019. WGAN-based synthetic minority over-sampling technique: Improving semantic fine-grained classification for lung nodules in CT images. IEEE Access 7 (2019), 18450–18463.

[36]

Y. Peng, J. Qi, and Y. Zhuo. 2020. MAVA: Multi-level adaptive visual-textual alignment by cross-media bi-attention mechanism. IEEE Transactions on Image Processing 29 (2020), 2728–2741.

[37]

H. Yao, S. Zhang, C. Yan, Y. Zhang, J. Li, and Q. Tian. 2018. AutoBD: Automated bi-level description for scalable fine-grained visual categorization. IEEE Transactions on Image Processing 27, 1 (2018), 10–23.

[38]

Z. Lv, A. Halawani, S. Feng, H. Li, and S. U. Réhman. 2014. Multimodal hand and foot gesture interaction for handheld devices. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 1s (2014), 1–19.

Digital Library

[39]

Z. Lv, A. Halawani, S. Feng, S. Ur Réhman, and H. Li. 2015. Touch-less interactive augmented reality game on vision-based wearable device. Personal and Ubiquitous Computing 19, 3–4 (2015), 551–567.

Digital Library

Cited By

Dai SZhou H(2024)An electromagnetic signal classification method inspired by the visual characteristics of biological dual-channelWireless Networks10.1007/s11276-023-03518-y30:2(909-921)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s11276-023-03518-y
Miao R(2023)Emotion Analysis and Opinion Monitoring of Social Network Users Under Deep Convolutional Neural NetworkJournal of Global Information Management10.4018/JGIM.31930931:1(1-12)Online publication date: 10-Mar-2023
https://dl.acm.org/doi/10.4018/JGIM.319309
Cui HLi S(2022)Controllable Clustering Algorithm for Associated Real-Time Streaming Big Data Based on Multi-Source Data FusionWireless Communications & Mobile Computing10.1155/2022/52446952022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/5244695
Show More Cited By

Index Terms

Fine-Grained Visual Computing Based on Deep Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Pixel Saliency Based Encoding for Fine-Grained Image Classification
Pattern Recognition and Computer Vision
Abstract
Fine-grained image classification concerns categorization at subordinate levels, where the distinction between inter-class objects is very subtle and highly local. Recently, Convolutional Neural Networks (CNNs) have almost yielded the best results ...
An Attribute-Based Fine-Grained Access Control Mechanism for HBase
Database and Expert Systems Applications
Abstract
In the current age of big data, the access control mechanism of HBase, a kind of NoSQL big data management system, needs to be improved, because there are some limitations of Role-Based Access Control (RBAC) in HBase. The coarse-grained access ...
A Fine Grained RBAC Model Supporting Flexible Administrative Separation of Duty
IIH-MSP '10: Proceedings of the 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing

Role based access control (RBAC) model is widely used in information system for efficient management of complex access control policy. Various extensions to the basic RBAC model are proposed for different purpose. A novel extension to the basic RBAC is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 17, Issue 1s

January 2021

353 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/3453990

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2021 Association for Computing Machinery.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 April 2021

Accepted: 01 July 2020

Revised: 01 May 2020

Received: 01 March 2020

Published in TOMM Volume 17, Issue 1s

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

National Natural Science Foundation of China
Key Research and Development Plan–Major Scientific and Technological Innovation Projects of ShanDong Province

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
409
Total Downloads

Downloads (Last 12 months)36
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dai SZhou H(2024)An electromagnetic signal classification method inspired by the visual characteristics of biological dual-channelWireless Networks10.1007/s11276-023-03518-y30:2(909-921)Online publication date: 1-Feb-2024
https://dl.acm.org/doi/10.1007/s11276-023-03518-y
Miao R(2023)Emotion Analysis and Opinion Monitoring of Social Network Users Under Deep Convolutional Neural NetworkJournal of Global Information Management10.4018/JGIM.31930931:1(1-12)Online publication date: 10-Mar-2023
https://dl.acm.org/doi/10.4018/JGIM.319309
Cui HLi S(2022)Controllable Clustering Algorithm for Associated Real-Time Streaming Big Data Based on Multi-Source Data FusionWireless Communications & Mobile Computing10.1155/2022/52446952022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/5244695
Chen HZhou HZhang JChen DZhang WChen KHua GYu N(2022)Perceptual Hashing of Deep Convolutional Neural Networks for Model Copy DetectionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/357277719:3(1-20)Online publication date: 5-Dec-2022
https://dl.acm.org/doi/10.1145/3572777
Zhao SWang PZhao XTurabieh HMafarja MChen H(2022)Elite dominance scheme ingrained adaptive salp swarm algorithm: a comprehensive studyEngineering with Computers10.1007/s00366-021-01464-x38:Suppl 5(4501-4528)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s00366-021-01464-x
Sun H(2022)A particle swarm optimization and coupled generalized differential quadrature element methods with genetic algorithm for stability analysis of the laminated microsystemsEngineering with Computers10.1007/s00366-021-01455-y38:Suppl 4(3251-3268)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1007/s00366-021-01455-y
Valizadeh AShariatee M(2021)The Progress of Medical Image Semantic Segmentation Methods for Application in COVID-19 DetectionComputational Intelligence and Neuroscience10.1155/2021/72656442021Online publication date: 22-Nov-2021
https://dl.acm.org/doi/10.1155/2021/7265644

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents