End-to-end Deep Reinforcement Learning for Targeted Drug Generation
Authors: 
Tiago Oliveira Pereira, Maryam Abbasi, Bernardete Ribeiro, Joel P. ArraisAuthors Info & Claims
Tiago Oliveira Pereira, Maryam Abbasi, Bernardete Ribeiro, Joel P. ArraisAuthors Info & ClaimsPages 7 - 13
Abstract
The long period of time and the enormous financial costs required to bring a new drug to the market are a clear impediment to the development of new drugs. Deep Learning techniques at early stages of drug discovery can help to select candidate drugs with biological properties of interest, reduce the enormous research space of drug-like compounds and minimize these issues. This study aims to perform generation of targeted molecules by training the recurrent neural network to learn the building rules of production of valid molecules in the form of SMILES strings and optimize it to produce molecules with bespoke properties through Reinforcement Learning. The fitness of the newly generated molecules is obtained by a second neural network model. To demonstrate the effectiveness of the method, we trained the proposed model to design molecules with high inhibitory power for the k-opioid receptor (KOR). The optimized model was able to generate molecules with a stronger affinity for KOR, maintaining the percentage of valid molecules and, with satisfactory internal and external diversities based on Tanimoto similarity over 95%.
References
[1]
Getting Started with the RDKit in Python. 2019. https://www.rdkit.org/docs/GettingStartedInPython.html Accessed: 2019-01-31.
[2]
RDKit: Open-Source Cheminformatics Software. 2017. https://www.rdkit.org/
[3]
Softmax activation function.2019.https://towardsdatascience.com/softmax-function-simplified-714068bf8156 Accessed: 2019-01-31
[4]
Chemaxon. 2017. Marvin Sketch. www.chemaxon.com/products/marvin/
[5]
Andrew L. Beam, Benjamin Kompa, Allen Schmaltz, Inbar Fried, Griffin Weber, Nathan P. Palmer, Xu Shi, Tianxi Cai, and Isaac S. Kohane. 2018. Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data.arXiv:cs.CL/1804.01486
[6]
Tyler C. Beck, Matthew A. Hapstack, Kyle R. Beck, and Thomas A. Dix. 2019. Therapeutic potential of kappa opioid agonists. Pharmaceuticals (2019). https://doi.org/10.3390/ph12020095
[7]
Mostapha Benhenda. 2017. ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity? arXiv preprint arXiv:1708.08227 (2017).
[8]
G. Richard Bickerton, Gaia V. Paolini, Jérémy Besnard, Sorel Muresan, and Andrew L. Hopkins. 2012. Quantifying the chemical beauty of drugs. Nature Chemistry (2012). https://doi.org/10.1038/nchem.1243
[9]
Walter Cedeño and Dimitris K. Agrafiotis. 2003. Using particle swarms for the development of QSAR models based on Knearest neighbor and kernel regression. Journal of Computer-Aided Molecular Design (2003). https://doi.org/10.1023/A:1025338411016
[10]
Suman K. Chakravarti and Sai Radha Mani Alla. 2019. Descriptor free QSAR modeling using deep learning with long short-term memory neural networks. Frontiers in Artificial Intelligence (2019). https://doi.org/10.3389/frai.2019.00017
[11]
Daniel C. Elton, Zois Boukouvalas, Mark D. Fuge, and Peter W. Chung. 2019. Deep learning for molecular design – A review of the state of the art. https://doi.org/10.1039/c9me00039a arXiv:1903.04388
[12]
Peter Ertl and Ansgar Schuffenhauer. 2009. Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. Journal of Cheminformatics (2009). https://doi.org/10.1186/1758-2946-1-8
[13]
Yuan Feng, Xiaozhou He, Yilin Yang, Dongman Chao, Lawrence H. Lazarus, and Ying Xia. 2012. Current Research on Opioid Receptor Function. Current Drug Targets (2012). https://doi.org/10.2174/138945012799201612
[14]
Vincent François-Lavet, Peter Henderson, Riashat Islam, Marc G. Bellemare, and Joelle Pineau. 2018. An introduction to deep reinforcement learning. Foundations and Trends in Machine Learning (2018). https://doi.org/10.1561/2200000071 arXiv:1811.12560
[15]
Garrett B Goh, Nathan O Hodas, Charles Siegel, and Abhinav Vishnu. 2017. Smiles2vec: An interpretable general purpose deep neural network for predicting chemical properties. arXiv preprint arXiv:1712.02034 (2017).
[16]
Anvita Gupta, Alex T Müller, Berend JH Huisman, Jens A Fuchs, Petra Schneider, and Gisbert Schneider. 2018. Generative recurrent networks for de novo drug design. Molecular informatics 37, 1-2 (2018), 1700111.
[17]
Christos A. Nicolaou, Joannis Apostolakis, and Costas S. Pattichis. 2009. De novo drug design using multiobjective evolutionary graphs. Journal of Chemical Information and Modeling (2009). https://doi.org/10.1021/ci800308h
[18]
Marcus Olivecrona, Thomas Blaschke, Ola Engkvist, and Hongming Chen. 2017. Molecular de-novo design through deep reinforcement learning. Journal of Cheminformatics (2017). https://doi.org/10.1186/s13321-017-0235-x arXiv:1704.07555
[19]
F. Pedregosa and Varoquaux 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.
[20]
Mariya Popova, Olexandr Isayev, and Alexander Tropsha. 2018. Deep reinforcement learning for de novo drug design. Science Advances (2018). https://doi.org/10.1126/sciadv.aap7885
[21]
Jean Louis Reymond, Lars Ruddigkeit, Lorenz Blum, and Ruud van Deursen. 2012. The enumeration of chemical space. https://doi.org/10.1002/wcms.1104
[22]
David Rogers and Mathew Hahn. 2010. Extended-connectivity fingerprints. Journal of Chemical Information and Modeling (2010). https://doi.org/10.1021/ci100050t
[23]
Timon Sebastian Schroeter, Anton Schwaighofer, Sebastian Mika, Antonius Ter Laak, Detlev Suelzle, Ursula Ganzer, Nikolaus Heinrich, and Klaus Robert Müller. 2007. Estimating the domain of applicability for machine learning QSAR models: A study on aqueous solubility of drug discovery molecules. Journal of Computer-Aided Molecular Design (2007). https://doi.org/10.1007/s10822-007-9160-9
[24]
Marwin H.S. Segler, Thierry Kogej, Christian Tyrchan, and Mark P.Waller. 2018. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Science (2018). https://doi.org/10.1021/acscentsci.7b00512 arXiv:1701.01329
[25]
Yi Shang and Marta Filizola. 2015. Opioid receptors: Structural and mechanistic insights into pharmacology and signaling. European Journal of Pharmacology (2015). https://doi.org/10.1016/j.ejphar.2015.05.012
[26]
Dagmar Stumpfe and Jürgen Bajorath. 2011. Similarity searching. https://doi.org/10.1002/wcms.23
[27]
Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction (second ed.). The MIT Press. http://incompleteideas.net/book/the-book-2nd.html
[28]
Ulrike von Luxburg and Bernhard Schölkopf. 2011. Statistical Learning Theory: Models, Concepts, and Results. Elsevier. https://doi.org/10.1016/B978-0-444-52936-7.50016-1 arXiv:0810.4752
[29]
Yu Hua Wang, Jian Feng Sun, Yi Min Tao, Zhi Qiang Chi, and Jing Gen Liu. 2010. The role of K-opioid receptor activation in mediating antinociception and addiction. https://doi.org/10.1038/aps.2010.138
Recommendations
Structure based drug design studies on urokinase plasminogen activator inhibitors using AutoDock
CCSEIT '12: Proceedings of the Second International Conference on Computational Science, Engineering and Information TechnologyThe urokinase plasminogen activator receptor (uPAR) is a glycosylphosphatidylinositol (GPI) membrane-anchored receptor that binds the serine protease urokinase plasminogen activator (uPA). That uPAR plays an important role in determining malignancy of ...
FSM-DDTR: End-to-end feedback strategy for multi-objective De Novo drug design using transformers
AbstractThe design of compounds that target specific biological functions with relevant selectivity is critical in the context of drug discovery, especially due to the polypharmacological nature of most existing drug molecules. In recent years, in silico-...
Graphical abstractDisplay Omitted
Highlights- Novel multi-objective Transformer-based architecture to generate drug candidates.
- Transformer-based predictor and generator, and a multi-objective feedback loop.
- Unbiased generator outperforms state-of-the-art baselines in the ...
Coarse-Grained Modeling of the HIV---1 Protease Binding Mechanisms: II. Folding Inhibition
Computational Intelligence Methods for Bioinformatics and BiostatisticsEvolutionary and structurally conserved fragments 24---34 and 83---93 from each of the HIV---1 protease (HIV---1 PR) monomers constitute the critical components of the HIV---1 PR folding nucleus. It has been recently discovered that the peptide with the ...
Comments
Information & Contributors
Information
Published In
December 2020
80 pages
ISBN:9781450388443
DOI:10.1145/3449258
Copyright © 2020 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 22 June 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
- FCT - Fundação para a Ciência e a Tecnologia
Conference
ICCBB '20
ICCBB '20: 2020 4th International Conference on Computational Biology and Bioinformatics
December 27 - 29, 2020
Bali Island, Indonesia
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 470Total Downloads
- Downloads (Last 12 months)271
- Downloads (Last 6 weeks)29
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in