Abstract
Multi-task learning solves a collection of different tasks by sharing a large encoder (common network). However, due to high computational demand for large common network, deploying multi-task learning on resource constrained devices is a challenging task. To guarantee overall accuracy with less computation, we introduce DasNet, which (1) automatically searches the adaptive common network sub-structure for each task; (2) fine-tunes corresponding decoders to adapt to the common network sub-structure. Our method can accelerate the real time inference procedure by dynamically building adaptive structure according to specific task requirements in actual scenarios. Notably, all extra memory and calculations we need for our method can be negligible. Experiments conducted on four public datasets (i.e., ILSCRC-2012, Birds200, CatVsDog, MIT67) demonstrate that our proposed method performs effectively compared with general multi-task network architecture and present related state of the art (SOTA) methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Teichmann, M., Weber, M., Zoellner, M., Cipolla, R., Urtasun, R.: MultiNet: real-time joint semantic reasoning for autonomous driving. In: IEEE Intelligent Vehicles Symposium, pp. 1013–1020 (2018)
Kokkinos, I.: UberNet: training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 5454–5463 (2017)
Misra, I., Shrivastava, A., Gupta, A., Hebert, M.: Cross-stitch networks for multi-task learning. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3994–4003 (2016)
Jou, B., Chang, S.-F.: Deep cross residual learning for multitask visual recognition. In: Proceedings of the 24th ACM International Conference on Multimedia, Amsterdam, pp. 998–1007 (2016)
Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 3366–3375 (2017)
Doersch, C., Zisserman, A.: Multi-task self-supervised visual learning. In: The IEEE International Conference on Computer Vision, pp. 2051–2060 (2017)
Zhong, Y., Li, V., Okada, R., Maki, A.: Target aware network adaptation for efficient representation learning. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11132, pp. 450–467. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11018-5_38
Zamir, A.R., Sax, A., Shen, W., Guibas, L.J., Malik, J., Savarese, S.: Taskonomy: disentangling task transfer learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3712–3722 (2018)
Luo, J.H., Wu, J.: AutoPruner: an end-to-end trainable filter pruning method for efficient deep model inference (2018)
Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_53
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778. IEEE Press, New York (2016)
LeCun, Y., Denker, J.S., Solla, S.A.: Optimal brain damage. In: Advances in Neural Information Processing Systems, pp. 598–605 (1990)
Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: optimal brain surgeon. In: Advances in Neural Information Processing Systems, San Mateo, pp. 164–171 (1993)
Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. In: Advances in Neural Information Processing Systems, vol. 28, pp. 1135–1143 (2015)
Huang, Z., Wang, N.: Data-driven sparse structure selection for deep neural networks. In: The European Conference on Computer Vision, pp. 304–320 (2018)
Manessi, F., Rozza, A., Bianco, S., Napoletano, P., Schettini, R.: Automated pruning for deep neural network compression. In: 24th International Conference on Pattern Recognition, pp. 657–664 (2018)
Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. In: Advances in Neural Information Processing Systems, vol. 29, pp. 2074–2082 (2016)
Luo, J.H., Wu, J., Lin, W.: ThiNet: a filter level pruning method for deep neural network compression. In: IEEE International Conference on Computer Vision, pp. 5068–5076 (2017)
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and \(<\)1MB model size. In: CoRR (2016)
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. In: CoRR (2017)
Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5987–5995. IEEE Press, New York (2017)
Russakovsky, O., et al.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset (2011)
Elson, J., Douceur, J.J., Howell, J., Saul, J.: A CAPTCHA that exploits interest-aligned manual image categorization. In: Proceedings of 14th ACM Conference on Computer and Communications Security (2007)
Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–420 (2009)
Molchanov, P., Tyree, S., Karras, T., Aila, T., Kautz, J.: Pruning convolutional neural networks for resource efficient transfer learning. In: International Conference of Learning Representation (2016)
Acknowledgment
This work was supported in part by the National Nature Science Foundation of China under Grants 61673275, 61873166.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ma, Y., Wu, J., Long, C. (2019). DasNet: Dynamic Adaptive Structure for Accelerating Multi-task Convolutional Neural Network. In: Gedeon, T., Wong, K., Lee, M. (eds) Neural Information Processing. ICONIP 2019. Lecture Notes in Computer Science(), vol 11953. Springer, Cham. https://doi.org/10.1007/978-3-030-36708-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-36708-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36707-7
Online ISBN: 978-3-030-36708-4
eBook Packages: Computer ScienceComputer Science (R0)