Abstract
Transfer learning has been widely applied in Artificial Intelligence of Things (AIoT) to support intelligent services. Typically, collection and collaboration are two mainstreaming methods to improve transfer learning performance, whose efficiency has been evaluated by real-data experimental results but lacks validation of theoretical analysis. In order to provide guidance of implementing transfer learning in real applications, a theoretical analysis is in desired need to help us fully understand how to efficiently improve transfer learning performance. To this end, in this paper, we conduct comprehensive analysis on the methods of enhancing transfer learning performance. More specifically, we prove the answers to three critical questions for transfer learning: i) by comparing collecting instances and collecting attributes, which collection approach is more efficient? ii) is collaborative transfer learning efficient? and iii) by comparing collection with collaboration, which one is more efficient? Our answers and findings can work as fundamental guidance for developing transfer learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
AL-Sammarraie, O.A., Bashir, M.A.: Generalization of Newton’s forward interpolation formula. Int. J. Sci. Res. Publ. (2015)
Argyriou, A., Evgeniou, T., Pontil, M.: Convex multi-task feature learning. Mach. Learn. 73, 243–272 (2008). https://doi.org/10.1007/s10994-007-5040-8
Blanke, U., Schiele, B.: Remember and transfer what you have learned-recognizing composite activities based on activity spotting. In: International Symposium on Wearable Computers, pp. 1–8. IEEE (2010)
Chen, Y., Qin, X., Wang, J., Yu, C., Gao, W.: FedHealth: a federated transfer learning framework for wearable healthcare. IEEE Intell. Syst. 35(4), 83–93 (2020)
Daga, H., Nicholson, P.K., Gavrilovska, A., Lugones, D.: Cartel: a system for collaborative transfer learning at the edge. In: Proceedings of the ACM Symposium on Cloud Computing, pp. 25–37. ACM (2019)
Davis, J., Domingos, P.: Deep transfer via second-order Markov logic. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 217–224. ACM (2009)
Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 109–117. ACM (2004)
Farhadi, A., Forsyth, D., White, R.: Transfer learning in sign language. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2007)
Global Forecast: Cisco visual networking index: global mobile data traffic forecast. Update 2017–2022 (2019)
Hsieh, K., et al.: Gaia: geo-distributed machine learning approaching LAN speeds. In: 14th USENIX Symposium on Networked Systems Design and Implementation, pp. 629–647. USENIX (2017)
Jebara, T.: Multi-task feature and kernel selection for SVMs. In: Proceedings of the 21st International Conference on Machine Learning, p. 55. ACM (2004)
Jiang, J., Zhai, C.: Instance weighting for domain adaptation in NLP. In: Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pp. 264–271. ACL (2007)
Ju, C., Gao, D., Mane, R., Tan, B., Liu, Y., Guan, C.: Federated transfer learning for EEG signal classification. In: 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society, EMBC 2020, Montreal, QC, Canada, 20–24 July 2020, pp. 3040–3045. IEEE (2020). https://doi.org/10.1109/EMBC44109.2020.9175344
Kang, Y., et al.: Neurosurgeon: collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Comput. Archit. News 45, 615–629 (2017)
Kearns, M.J., Vazirani, U.V., Vazirani, U.: An Introduction to Computational Learning Theory. MIT Press, Cambridge (1994)
Lawrence, N.D., Platt, J.C.: Learning to learn with the informative vector machine. In: Proceedings of the 21st International Conference on Machine Learning, p. 65. ACM (2004)
Lee, S.I., Chatalbashev, V., Vickrey, D., Koller, D.: Learning a meta-level prior for feature relevance from multiple related tasks. In: Proceedings of the 24th International Conference on Machine Learning, pp. 489–496. ACM (2007)
Liao, X., Xue, Y., Carin, L.: Logistic regression with an auxiliary data source. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 505–512. ACM (2005)
Luo, C., et al.: AIoT bench: towards comprehensive benchmarking mobile and embedded device intelligence. In: Zheng, C., Zhan, J. (eds.) Bench 2018. LNCS, vol. 11459, pp. 31–35. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32813-9_4
Ma, Z., et al.: PMKT: privacy-preserving multi-party knowledge transfer for financial market forecasting. Future Gener. Comput. Syst. 106, 545–558 (2020)
Ma, Z., et al.: PMKT: privacy-preserving multi-party knowledge transfer for financial market forecasting. Future Gener. Comput. Syst. 106, 545–558 (2020). https://doi.org/10.1016/j.future.2020.01.007
Mihalkova, L., Huynh, T., Mooney, R.J.: Mapping and revising Markov logic networks for transfer learning. In: AAAI, vol. 7, pp. 608–614. AAAI (2007)
Ogoe, H.A., Visweswaran, S., Lu, X., Gopalakrishnan, V.: Knowledge transfer via classification rules using functional mapping for integrative modeling of gene expression data. BMC Bioinform. 16, 1–15 (2015). https://doi.org/10.1186/s12859-015-0643-8
Olmedilla, D.: Applying machine learning to ads integrity at Facebook. In: Proceedings of the 8th ACM Conference on Web Science, p. 4. ACM (2016)
Palatucci, M., Pomerleau, D., Hinton, G.E., Mitchell, T.M.: Zero-shot learning with semantic output codes. In: Advances in Neural Information Processing Systems, pp. 1410–1418. MIT Press (2009)
Perlich, C., Dalessandro, B., Raeder, T., Stitelman, O., Provost, F.: Machine learning for targeted display advertising: transfer learning in action. Mach. Learn. 95, 103–127 (2014). https://doi.org/10.1007/s10994-013-5375-2
Rababah, A.: Taylor theorem for planar curves. Proc. Am. Math. Soc. 119, 803–810 (1993)
Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: transfer learning from unlabeled data. In: Proceedings of the 24th International Conference on Machine Learning, pp. 759–766. ACM (2007)
Schwaighofer, A., Tresp, V., Yu, K.: Learning gaussian process kernels via hierarchical Bayes. In: Advances in Neural Information Processing Systems, pp. 1209–1216. MIT Press (2005)
Sharma, S., Xing, C., Liu, Y., Kang, Y.: Secure and efficient federated transfer learning. In: 2019 IEEE International Conference on Big Data, pp. 2569–2576. IEEE (2019)
Torralba, A., Murphy, K.P., Freeman, W.T.: Sharing visual features for multiclass and multiview object detection. IEEE Trans. Pattern Anal. Mach. Intell. 29, 854–869 (2007)
Wang, C., Mahadevan, S.: Manifold alignment using procrustes analysis. In: Proceedings of the 25th International Conference on Machine Learning, pp. 1120–1127. ACM (2008)
Wang, W., Zheng, V.W., Yu, H., Miao, C.: A survey of zero-shot learning: settings, methods, and applications. ACM Trans. Intell. Syst. Technol. 10, 1–37 (2019)
Wu, P., Dietterich, T.G.: Improving SVM accuracy by training on auxiliary data sources. In: Proceedings of the 21st International Conference on Machine Learning, p. 110. ACM (2004)
Yang, H., He, H., Zhang, W., Cao, X.: FedSteg: a federated transfer learning framework for secure image steganalysis. IEEE Trans. Netw. Sci. Eng. 14, 78–88 (2018)
Acknowledgment
This work was partly supported by the National Science Foundation of U.S. (2118083, 1912753, 1704287, 1741277, 1829674).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, H., Cai, Z., Li, W. (2021). Which Option Is a Better Way to Improve Transfer Learning Performance?. In: Du, DZ., Du, D., Wu, C., Xu, D. (eds) Combinatorial Optimization and Applications. COCOA 2021. Lecture Notes in Computer Science(), vol 13135. Springer, Cham. https://doi.org/10.1007/978-3-030-92681-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-92681-6_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92680-9
Online ISBN: 978-3-030-92681-6
eBook Packages: Computer ScienceComputer Science (R0)