Abstract
The learnware paradigm has been proposed as a new manner for reusing models from a market of various well-trained models, which can relieve users’ burden of training a new model from scratch. A learnware consists of a well-trained model and a specification which explains the purpose or specialty of the model without revealing data. By specification matching, the market can identify the most useful learnwares for users’ tasks. Prior art attempted to generate the specification by a reduced kernel mean embedding approach. However, such kind of specification is defined by some pre-designed kernel function, which lacks flexibility. In this paper, we advance a methodology for direct specification learning from data, introducing a novel neural network named SpecNet for this purpose. Our approach accepts unordered datasets as input and subsequently produces specification vectors in a latent space. Notably, the flexibility and efficiency of our learned specifications are underscored by their derivation from diverse tasks, rendering them particularly adept for learnware identification. Empirical studies provide validation for the efficacy of our proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Zhou Z H. Learnware: on the future of machine learning. Frontiers of Computer Science, 2016, 10(4): 589–590
Tan P, Tan Z H, Jiang Y, Zhou Z H. Towards enabling learnware to handle heterogeneous feature spaces. Machine Learning, 2024, 113(4): 1839–1860
Zhang Y J, Yan Y H, Zhao P, Zhou Z H. Towards enabling learnware to handle unseen jobs. In: Proceedings of the 35th AAAI Conference on Artificial Intelligence. 2021, 10964–10972
Guo L Z, Zhou Z, Li Y F, Zhou Z H. Identifying useful learnwares for heterogeneous label spaces. In: Proceedings of the 40th International Conference on Machine Learning. 2023, 12122–12131
Hoffman J, Tzeng E, Park T, Zhu J Y, Isola P, Saenko K, Efros A, Darrell T. CyCADA: cycle-consistent adversarial domain adaptation. In: Proceedings of the 35th International Conference on Machine Learning. 2018, 1994–2003
Ben-David S, Blitzer J, Crammer K, Kulesza A, Pereira F, Vaughan J W. A theory of learning from different domains. Machine Learning, 2010, 79(1–2): 151–175
Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation. In: Proceedings of the 32nd International Conference on Machine Learning. 2015, 1180–1189
Kumar A, Ma T, Liang P. Understanding self-training for gradual domain adaptation. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 507
Luo Y, Wang Z, Huang Z, Baktashmotlagh M. Progressive graph learning for open-set domain adaptation. In: Proceedings of the 37th International Conference on Machine Learning. 2020, 600
Wu X Z, Xu W, Liu S, Zhou Z H. Model reuse with reduced kernel mean embedding specification. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(1): 699–710
Smola A, Gretton A, Song L, Schölkopf B. A hilbert space embedding for distributions. In: Proceedings of the 18th International Conference on Algorithmic Learning Theory. 2007, 13–31
Ding Y X, Zhou Z H. Boosting-based reliable model reuse. In: Proceedings of the 12th Asian Conference on Machine Learning. 2020, 145–160
Ben-David S, Blitzer J, Crammer K, Pereira F. Analysis of representations for domain adaptation. In: Proceedings of the 19th International Conference on Neural Information Processing Systems. 2006, 137–144
Germain P, Habrard A, Laviolette F, Morvant E. A new PAC-Bayesian perspective on domain adaptation. In: Proceedings of the 33rd International Conference on Machine Learning. 2016, 859–868
Liu M Y, Tuzel O. Coupled generative adversarial networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems. 2016, 469–477
Chang W, Shi Y, Tuan H D, Wang J. Unified optimal transport framework for universal domain adaptation. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 2140
Zhu Y, Wu X, Qiang J, Yuan Y, Li Y. Representation learning via an integrated autoencoder for unsupervised domain adaptation. Frontiers of Computer Science, 2023, 17(5): 175334
Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345–1359
Ding Y X, Wu X Z, Zhou K, Zhou Z H. Pre-trained model reusability evaluation for small-data transfer learning. In: Proceedings of the 36th International Conference on Neural Information Processing Systems. 2022, 2710
Wang B, Mendez J A, Cai M B, Eaton E. Transfer learning via minimizing the performance gap between domains. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. 2019, 955
Houlsby N, Giurgiu A, Jastrzebski S, Morrone B, De Laroussilhe Q, Gesmundo A, Attariyan M, Gelly S. Parameter-efficient transfer learning for NLP. In: Proceedings of the 36th International Conference on Machine Learning. 2019, 2790–2799
Yang Y, Guo J, Li G, Li L, Li W, Yang J. Alignment efficient image-sentence retrieval considering transferable cross-modal representation learning. Frontiers of Computer Science, 2024, 18(1): 181335
Wang Z, Dai Z, Póczos B, Carbonell J. Characterizing and avoiding negative transfer. In: Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019, 11293–11302
Maturana D, Scherer S. VoxNet: a 3D convolutional neural network for real-time object recognition. In: Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. 2015, 922–928
Qi C R, Su H, Nießner M, Dai A, Yan M, Guibas L J. Volumetric and multi-view CNNs for object classification on 3D data. In: Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. 2016, 5648–5656
Wu Z, Song S, Khosla A, Yu F, Zhang L, Tang X, Xiao J. 3D ShapeNets: a deep representation for volumetric shapes. In: Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. 2015, 1912–1920
Vinyals O, Bengio S, Kudlur M. Order matters: sequence to sequence for sets. In: Proceedings of the 4th International Conference on Learning Representations. 2016
Qi Charles R, Su H, Kaichun M, Guibas L J. PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. 2017, 77–85
Le Y, Yang X. Tiny ImageNet visual recognition challenge. In: Proceedings of the Stanford CS231N Convolutional Neural Networks for Visual Recognition. 2015
Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report TR-2009. Toronto: University of Toronto, 2009
Acknowledgements
This research was supported by the National Natural Science Foundation of China (Grant Nos. 62076121, 61921006) and the Major Program (JD) of Hubei Province (2023BAA024).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Competing interests Ming Li is an Editorial Board member of the journal and a co-author of this article. To minimize bias, they were excluded from all editorial decision-making related to the acceptance of this article for publication. The remaining authors declare no conflict of interest.
Additional information
Zhi-Yu Shen received the BEng degree in computer science and technology from Nanjing University, China in 2014. Currently he is a PhD candidate in the Department of Computer Science and Technology, Nanjing University, and is a member of the LAMDA Group. His research interests mainly include machine learning and data mining, especially in domain adaptation and transfer learning.
Ming Li is currently a professor with the LAMDA group, the National Key Laboratory for Novel Software Technology, Nanjing University, China. His major research interests include machine learning and data mining, especially on software mining. He has served as the area chair of IJCAI, IEEE ICDM, etc., senior PC member of the premium conferences in artificial intelligence such as AAAI, and PC members for other premium conferences such as KDD, NeurIPS, and ICML. He is the founding chair of the International Workshop on Software Mining. He has been granted various awards including the PAKDD Early Career Award, the NSFC Excellent Youth Award, the New Century Excellent Talents program of the Education Ministry of China, etc.
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Shen, ZY., Li, M. Identifying useful learnwares via learnable specification. Front. Comput. Sci. 19, 199344 (2025). https://doi.org/10.1007/s11704-024-40135-0
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11704-024-40135-0