Abstract
Transfer learning algorithms typically assume that the training data and the test data come from different distribution. It is better at adapting to learn new tasks and concepts more quickly and accurately by exploiting previously gained knowledge. Deep Transfer Learning (DTL) emerged as a new paradigm in transfer learning in which a deep model offer greater flexibility in extracting high-level features. DTL offers selective layer based transference, and it is problem specific. In this paper, we propose the Ensemble of Deep Transfer Learning (EDTL) methodology to reduce the impact of selective layer based transference and provide optimized framework to work for three major transfer learning cases. Empirical results on character, object and biomedical image recognition tasks achieves that the proposed method indicate statistically significant classification accuracy over the other established transfer learning method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Thrun, S.: Learning to learn: Introduction. In Learning To Learn (1996)
Caruana, R.: Multitask learning. Machine Learning 28(1), 41–75 (1997)
Daumé III, H., Marcu, D.: Domain Adaptation for Statistical Classifiers. J. Artif. Intell. Res. (JAIR) 26, 101–126 (2006)
Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: Proc. of the ACM Conference on (ICML), pp. 759–766 (2007)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. proceedings of the IEEE 86(11), 2278–2324 (1998)
Hinton, G.E., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. The Journal of Neural computation 7, 1527–1554 (2006)
Vincent, P., Larochelle, H., Lajoie, I., Bengio, Y., Manzagol, P.-A.: Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 11, 3371–3408 (2010)
Bengio, Y., et al.: Towards Biologically Plausible Deep Learning. arXiv preprint arXiv:1502.04156 (2015)
Kandaswamy, C., Silva, L., Alexandre, L., Sousa, R., Santos, J.M., Marques de Sá, J.: Improving transfer learning accuracy by reusing Stacked Denoising Autoencoders. In: IEEE Conference on Systems Man and Cybernetics. IEEE (2014)
Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M., de Sá, J.M.: Improving deep neural network performance by reusing features trained with transductive transference. In: Wermter, S., Weber, C., Duch, W., Honkela, T., Koprinkova-Hristova, P., Magg, S., Palm, G., Villa, A.E.P. (eds.) ICANN 2014. LNCS, vol. 8681, pp. 265–272. Springer, Heidelberg (2014)
Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: Advances in Neural Information Processing Systems, pp. 3320–3328 (2014)
Kandaswamy, C., Silva, L., Cardoso, J.S.: Source-target-source classification using Stacked Denoising Autoencoders. In: Proc. of the 7th Iberian Conference on Pattern Recognition and Image Analysis, Santiago de Compostela, Spain, June 2015
Razavian, A.S., Azizpour, H., Sullivan, J., Carlsson, S.: CNN features off-the-shelf: an astounding baseline for recognition. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 512–519. IEEE (2014)
Deng, L., Platt, J.C.: Ensemble deep learning for speech recognition. In: Proceedings of the Annual Conference of International Speech Communication Association (INTERSPEECH) (2014)
Abdullah, A., Veltkamp, R.C., Wiering, M.A.: An ensemble of deep support vector machines for image categorization. In: International Conference of Soft Computing and Pattern Recognition, SOCPAR 2009, pp. 301–306. IEEE (2009)
Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2012)
Lin, J.: Divergence measures based on the Shannon entropy. IEEE Transactions on Information Theory 37, 145–151 (1991)
Ljosa, V.: Katherine L. Sokolnicki, and Anne E. Carpenter.: Annotated high-throughput microscopy image sets for validation. Nat Methods 9(7), 637 (2012)
Ljosa, V., et al.: Comparison of methods for image-based profiling of cellular morphological responses to small-molecule treatment. Journal of biomolecular screening (2013)
Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley Press (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kandaswamy, C., Silva, L.M., Alexandre, L.A., Santos, J.M. (2015). Deep Transfer Learning Ensemble for Classification. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2015. Lecture Notes in Computer Science(), vol 9094. Springer, Cham. https://doi.org/10.1007/978-3-319-19258-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-19258-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19257-4
Online ISBN: 978-3-319-19258-1
eBook Packages: Computer ScienceComputer Science (R0)