skip to main content
10.1145/3364836.3364858acmotherconferencesArticle/Chapter ViewAbstractPublication PagesisicdmConference Proceedingsconference-collections
research-article

An Adaptive Device-Aware Model Optimization Framework

Published: 24 August 2019 Publication History

Abstract

Deep learning technology has been widely developed in all walks of life, especially in the medical research field. Recently, the deep neural network model has become a deeper and better direction, and followed by the problem of computing resources. The feasibility of a large neural network model can be evaluated by its suitability to sophisticated medical devices. With this basis, we propose an adaptive model optimization framework (AMOF). Compared to reported model compression techniques, we focus on the correlation between channels. AMOF cannot only output an accurate compression ratio, but also search for the optimal pruning channel. Specifically, evolutionary algorithms were introduced on the basis of reinforcement learning. Due to the complexity of a neural network, we propose a co-evolutionary algorithm, so as to guarantee the simultaneous evolution of multiple populations and finally output the optimal cutting channel. Notably, AMOF, combining reinforcement learning and evolutionary algorithm, can ensure the accuracy of this model applied under the full compression condition. The effectiveness of AMOF was proved by a large number of experimental tests. For example, on the CIFAR-10, the ResNet56 channel after our frame trimming was reduced by 30%; and the accuracy remained at 89.27%. Compared to the reinforcement learning compression method alone, AMOF can increase by 3.5 percentage points in the ResNet20 model.

References

[1]
Cheng, J., Wang, P. S., Li, G., Hu, Q. H., & Lu, H. Q. (2018). Recent advances in efficient computation of deep convolutional neural networks. Frontiers of Information Technology & Electronic Engineering, 19(1), 64--77.
[2]
Kim, Y. D., Park, E., Yoo, S., Choi, T., Yang, L., & Shin, D. (2015). Compression of deep convolutional neural networks for fast and low power mobile applications. Computer Science,71(2), 576--584.
[3]
Cheng, Y., Wang, D., Zhou, P., & Zhang, T. 2017. A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282.
[4]
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016). Quantized convolutional neural networks for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4820--4828).
[5]
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017, October). Learning Efficient Convolutional Networks through Network Slimming. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 2755--2763). IEEE.
[6]
Luo, J. H., Wu, J., & Lin, W. (2017, October). ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 5068--5076). IEEE.
[7]
Liu, B., Wang, M., Foroosh, H., Tappen, M., & Pensky, M. (2015). Sparse convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 806--814).
[8]
Zhuang, Z., Tan, M., Zhuang, B., Liu, J., Guo, Y., Wu, Q., ... & Zhu, J. (2018). Discrimination-aware channel pruning for deep neural networks. In Advances in Neural Information Processing Systems (pp. 875--886).
[9]
He, Y., Zhang, X., & Sun, J. (2017). Channel pruning for accelerating very deep neural networks. In Proceedings of the IEEE International Conference on Computer Vision (pp. 1389--1397).
[10]
He, Y., Lin, J., Liu, Z., Wang, H., Li, L. J., & Han, S. (2018). Amc: Automl for model compression and acceleration on mobile devices. In Proceedings of the European Conference on Computer Vision (ECCV) (pp. 784--800).
[11]
Wu, J., Leng, C., Wang, Y., Hu, Q., & Cheng, J. (2016). Quantized convolutional neural networks for mobile devices. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4820--4828).
[12]
Courbariaux, M., Bengio, Y., & David, J. P. (2015). Binaryconnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems (pp. 3123--3131).
[13]
Wen, W., Wu, C., Wang, Y., Chen, Y., & Li, H. (2016). Learning structured sparsity in deep neural networks. In Advances in neural information processing systems (pp. 2074--2082).
[14]
Han, S., Mao, H., & Dally, W. J. (2015). Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. Fiber, 56(4), 3--7.
[15]
Guyon, I., Bennett, K., Cawley, G., Escalante, H. J., Escalera, S., Ho, T. K., ... & Viegas, E. (2015, July). Design of the 2015 ChaLearn AutoML challenge. In 2015 International Joint Conference on Neural Networks (IJCNN) (pp. 1--8). IEEE.
[16]
Quanming, Y., Mengshuo, W., Hugo, J. E., Isabelle, G., Yi-Qi, H., Yu-Feng, L., ... & Yang, Y. (2018). Taking human out of learning applications: A survey on automated machine learning. arXiv preprint arXiv:1810.13306.
[17]
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... & Petersen, S. (2015). Human-level control through deep reinforcement learning. Nature, 518(7540), 529--533.
[18]
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., & Tassa, Y., et al. (2015). Continuous control with deep reinforcement learning. Computer Science, 8(6), A187.
[19]
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., ... & Kavukcuoglu, K. (2016, June). Asynchronous methods for deep reinforcement learning. In International conference on machine learning (pp. 1928--1937).
[20]
Rios, L. M., & Sahinidis, N. V. (2013). Derivative-free optimization: a review of algorithms and comparison of software implementations. Journal of Global Optimization, 56(3), 1247--1293.

Index Terms

  1. An Adaptive Device-Aware Model Optimization Framework

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    ISICDM 2019: Proceedings of the Third International Symposium on Image Computing and Digital Medicine
    August 2019
    370 pages
    ISBN:9781450372626
    DOI:10.1145/3364836
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    In-Cooperation

    • Xidian University

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 August 2019

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Deep learning
    2. evolutionary algorithm
    3. model optimization
    4. reinforcement learning
    5. sophisticated medical devices

    Qualifiers

    • Research-article
    • Research
    • Refereed limited

    Conference

    ISICDM 2019

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 57
      Total Downloads
    • Downloads (Last 12 months)2
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 16 Dec 2024

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media