Brian Leonard
ABSTRACT
We describe hyper-parameter optimization and training for a novel convolutional neural network that identifies neural axons in 3D volume electron microscope data for a larval zebrafish. Training a convolutional neural network to detect image components in three dimensional space has a variety of applications. There are many parts of a network that can be tuned to work most effectively in a supercomputing environment. We optimized run parameters including batch sizes, thread counts, and epoch sizes as well as hyper-parameters such as drop-out rates. Once we had improved the speed and accuracy of the network we created a large three dimensional volume to visualize the errors of the network. We added new samples to our training set based on the larger volume and improved the accuracy of the network using those samples. Adapting the network to our supercomputing environment and adjusting the dropout layers increased the accuracy and efficiency of the network. Being able to accurately detect neurons in three dimensional space shows potential for other deep learning applications.
- Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensorflow.org/ Software available from tensorflow.org.Google Scholar
- Hank Childs, Eric Brugger, Brad Whitlock, Jeremy Meredith, Sean Ahern, David Pugmire, Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rübel, Marc Durant, Jean M. Favre, and Paul Navrátil. 2012. VisIt: An End-User Tool For Visualizing and Analyzing Very Large Data. In High Performance Visualization–Enabling Extreme-Scale Scientific Insight. 357–372.Google Scholar
- Taco S. Cohen, Mario Geiger, Jonas Koehler, and Max Welling. 2018. Spherical CNNs. arxiv:1801.10130 [cs.LG]Google Scholar
- Richard Courant and David Hilbert. 1962. Methods of Mathematical Physics. Vol. 1. Wiley Interscience.Google Scholar
- David Grant Colburn Hildebrand, Marcelo Cicconet, Russel Miguel Torres, Woohyuk Choi, Tran Minh Quan, Jungmin Moon, Arthur Willis Wetzel, Andrew Scott Champion, Brett Jesse Graham, Owen Randlett, George Scott Plummer, Ruben Portugues, Isaac Henry Bianco, Stephan Saalfeld, Alex Baden, Kunal Lillaney, Randal Burns, Joshua Tzvi Vogelstein, Alexander Franz Schier, Wei-Chung Allen Lee, Won-Ki Jeong, Jeff William Lichtman, and Florian Engert. 2017. Whole-brain serial-section electron microscopy in larval zebrafish. Nature 545(2017), 345–349. https://doi.org/10.1038/nature22356Google ScholarCross Ref
- Stephan Saalfeld, Albert Cardona, Volker Hartenstein, and Pavel Tomančák. 2009. CATMAID: collaborative annotation toolkit for massive amounts of image data. Bioinformatics 25, 15 (04 2009), 1984–1986. https://doi.org/10.1093/bioinformatics/btp266 arXiv:https://academic.oup.com/bioinformatics/article-pdf/25/15/1984/555362/btp266.pdfGoogle Scholar
- Casey M. Schneider-Mizell, Stephan Gerhard, Mark Longair, Tom Kazimiers, Feng Li, Maarten F. Zwart, Andrew Champion, Frank M. Midgley, Richard D. Fetter, Stephan Saalfeld, and Albert Cardona. 2016. Quantitative neuroanatomy for connectomics in Drosophila. eLife (2016). https://doi.org/10.7554/eLife.12059Google Scholar
- Jean Serra. 1983. Image Analysis and Mathematical Morphology. Academic Press, Inc., USA.Google ScholarDigital Library
- Arthur W. Wetzel, Jennifer Bakal, Markus Dittrich, David G.C. Hildebrand, Josh L. Morgan, and Jeff W. Lichtman. 2016. Registering large volume serial-section electron microscopy image sets for neural circuit reconstruction using FFT signal whitening. 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR) (Oct 2016). https://doi.org/10.1109/aipr.2016.8010595Google ScholarCross Ref
- Mark A. Wieczorek and Matthias Meschede. 2018. SHTools: Tools for Working with Spherical Harmonics. Geochemistry, Geophysics, Geosystems 19, 8 (2018), 2574–2592. https://doi.org/10.1029/2018GC007529 arXiv:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2018GC007529Google Scholar
Index Terms
- Optimizing a Specialized Convolutional Neural Network in a Supercomputing Environment Using Tensorflow and VisIt
Recommendations
Using VisIt to Evaluate the Accuracy of a Convolutional Neural Network
PEARC '20: Practice and Experience in Advanced Research ComputingUsing VisIt, we visualized a 3D volume of neural axons for a larval zebrafish from electron microscope data and locations for those neurons estimated by a deep neural network. Errant points were recorded and used to further train the network. These ...
Scene text recognition using residual convolutional recurrent neural network
Text is a significant tool for human communication, and text recognition in scene images becomes more and more important. In this paper, we propose a residual convolutional recurrent neural network for solving the task of scene text recognition. The ...
Edge-preserving image denoising using a deep convolutional neural network
Highlights- This paper makes use of a deep CNN for image denoising.
- The network is trained ...
AbstractThis paper introduces a novel denoising approach making use of a deep convolutional neural network to preserve image edges. The network is trained by using the edge map obtained from the well-known Canny algorithm and aims at ...
Comments