Abstract
Molecular property prediction is becoming increasingly important in drug and material discovery, and many research works have demonstrated the great potential of machine learning techniques, especially deep learning. This paper presents our proposed solution for CCKS-2022 task 8, a chemical domain knowledge-aware framework for multi-view molecular property prediction. As a generative self-supervised approach to molecular graph representation learning, the framework is based on Knowledge-guided Pre-training of Graph Transformer (KPGT), which adopts a graph transformer guided by molecular fingerprint and descriptor knowledge. In the fine-tuning stage, combined with practical prediction problems, we fuse functional group information and chemical element knowledge graphs to predict molecular properties. From the perspective of chemical structure, KPGT provides structural information of molecular graphs (especially highlighting chemical bonds), and we further integrate chemical domain knowledge, using functional groups and chemical element knowledge graph, which is the information on physicochemical properties of atoms. From molecular graphs to functional groups, and to atoms, the molecular representation is jointly enhanced by multiple views from coarse to fine. When introducing functional group information and chemical element knowledge graph, we propose a novel BiLSTM-based recurrent module to accumulate domain knowledge. Our framework is able to simultaneously consider molecular graph, functional groups, and atomic physicochemical properties in practical predictions to better predict molecular properties. Finally, without using other external knowledge, the AUC-ROC of the test data reaches 0.88587, ranking second among 140 teams, which validates the performance of our approach.
R. Hua, X. Wang and C. Cheng—These authors contributed equally to the work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kontogeorgis, G.M., Gani, R.: Computer Aided Property Estimation for Process and Product Design: Computers Aided Chemical Engineering. Elsevier (2004)
Xu, Z., Wang, S., Zhu, F., et al.: Seq2seq fingerprint: an unsupervised deep molecular embedding for drug discovery. In: Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pp. 285–294. Association for Computing Machinery, New York (2017)
Gilmer, J., Schoenholz, S.S., Riley, P.F., et al.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning, pp. 1263–1272. PMLR, Sydney (2017)
Brown, N., Fiscato, M., Segler, M.H.S., et al.: GuacaMol: benchmarking models for de novo molecular design. J. Chem. Inf. Model. 59(3), 1096–1108 (2019)
Jing, L., Tian, Y.: Self-supervised visual feature learning with deep neural networks: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 43(11), 4037–4058 (2020)
Devlin, J., Chang, M.W., Lee, K., et al.: Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Guo, Z., Nan, B., Tian, Y., et al.: Graph-based Molecular Representation Learning. arXiv preprint arXiv:2207.04869 (2022)
Chang, D.T.: Embodied-Symbolic Contrastive Graph Self-Supervised Learning for Molecular Graphs. arXiv preprint arXiv:2205.06783 (2022)
Li, H., Zhao, D., Zeng, J.: KPGT: Knowledge-Guided Pre-training of Graph Transformer for Molecular Property Prediction. arXiv preprint arXiv:2206.03364 (2022)
Fang, Y., Zhang, Q., Yang, H., et al.: Molecular contrastive learning with chemical element knowledge graph. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, pp. 3968–3976 (2022)
Duvenaud, D.K., Maclaurin, D., Iparraguirre, J., et al.: Convolutional networks on graphs for learning molecular fingerprints. Adv. Neural Inf. Process. Syst. 28 (2015)
Yang, K., Swanson, K., Jin, W., et al.: Analyzing learned molecular representations for property prediction. J. Chem. Inf. Model. 59(8), 3370–3388 (2019)
Song, Y., Zheng, S., Niu, Z., et al.: Communicative representation learning on attributed molecular graphs. In: Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pp. 2831–2838 (2020)
Chen, J., Zheng, S., Song, Y., et al.: Learning attributed graph representations with communicative message passing transformer. arXiv preprint arXiv:2107.08773 (2021)
Chithrananda, S., Grand, G., Ramsundar, B.: ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. arXiv preprint arXiv:2010.09885 (2020)
Wang, Y., Wang, J., Cao, Z., et al.: Molecular contrastive learning of representations via graph neural networks. Nat. Mach. Intell. 4(3), 279–287 (2022)
Grave, K.D., Costa, F.: Molecular graph augmentation with rings and functional groups. J. Chem. Inf. Model. 50(9), 1660–1668 (2010)
Al-shammari, A.K., Al-Bermany, E.: Polymer functional group impact on the thermo-mechanical properties of polyacrylic acid, polyacrylic amide-poly (vinyl alcohol) nanocomposites reinforced by graphene oxide nanosheets. J Polym Res 29, 351 (2022)
Belkin, M., Hsu, D., Ma, S., et al.: Reconciling modern machine-learning practice and the classical bias–variance trade-off. In: Proceedings of the National Academy of Sciences, pp. 15849–15854 (2019)
Zhang, N., et al.: OntoProtein: protein pretraining with gene ontology embedding. In International Conference on Learning Representations (2022)
Nakkiran, P., Kaplun, G., Bansal, Y., et al.: Deep double descent: where bigger models and more data hurt. J. Stat. Mecha. Theory Exper. 12, 124003 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Hua, R., Wang, X., Cheng, C., Zhu, Q., Zhou, X. (2022). A Chemical Domain Knowledge-Aware Framework for Multi-view Molecular Property Prediction. In: Zhang, N., Wang, M., Wu, T., Hu, W., Deng, S. (eds) CCKS 2022 - Evaluation Track. CCKS 2022. Communications in Computer and Information Science, vol 1711. Springer, Singapore. https://doi.org/10.1007/978-981-19-8300-9_1
Download citation
DOI: https://doi.org/10.1007/978-981-19-8300-9_1
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-8299-6
Online ISBN: 978-981-19-8300-9
eBook Packages: Computer ScienceComputer Science (R0)