Davin Lin
ABSTRACT
This paper presents some experiments to validate the design of an Automated Computer Vision (AutoCV) library for applications in scientific image understanding. AutoCV attempts to define a search space of algorithms used in common image analysis workflows and then uses a fitness function to automatically select individual algorithmic workflows for a given problem. The final goal is a semi-automated system that can assist researchers in finding specific computer vision algorithms that work for their specific research questions. As an example of this method the researchers have built the SEE-Insight tool which uses genetic algorithms to search for image analysis workflows. This tool has been used to implement an image segmentation workflow (SEE-Segment) and is being updated and modified to work with other image analysis workflows such as anchor point detection and counting. This work is motivated by analogous work being done in Automated Machine Learning (AutoML). As a way to validate the approach, this paper uses the SEE-Insight tool to recreate an AutoML solution (called SEE-Classify) and compares results to an existing AutoML solution (TPOT). As expected the existing AutoML tool worked better than the prototype SEE-Classify tool. However, the goal of this work was to learn from these well-established tools and possibly identify one of them that could be modified as a mature replacement for the core SEE-Insight search algorithm. Although this drop-in replacement was not found, reproducing the AutoML experiments in the SEE-Insight framework provided quite a few insights into best practices for moving forward with this research.
- [n. d.]. SEE-Segment. https://github.com/see-insight/see-segment Accessed: 2021-09-11.Google Scholar
- Matthias Feurer and Frank Hutter. 2019. Hyperparameter Optimization. Springer International Publishing, Cham, 3–33. https://doi.org/10.1007/978-3-030-05318-5_1Google Scholar
- Matthias Feurer, Aaron Klein, Jost Eggensperger, Katharina Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In Advances in Neural Information Processing Systems 28 (2015). 2962–2970.Google ScholarDigital Library
- P. Gijsbers, E. LeDell, S. Poirier, J. Thomas, B. Bischl, and J. Vanschoren. 2019. An Open Source AutoML Benchmark. arXiv preprint arXiv:1907.00909 [cs.LG](2019). https://arxiv.org/abs/1907.00909 Accepted at AutoML Workshop at ICML 2019.Google Scholar
- Trang T Le, Weixuan Fu, and Jason H Moore. 2020. Scaling tree-based automated machine learning to biomedical big data with a feature set selector. Bioinformatics 36, 1 (2020), 250–256.Google ScholarCross Ref
- J. Sayyad Shirabad and T.J. Menzies. 2005. The PROMISE Repository of Software Engineering Databases.School of Information Technology and Engineering, University of Ottawa, Canada. http://promise.site.uottawa.ca/SERepositoryGoogle Scholar
- William Wolberg, W. Street, and Olvi Mangasarian. 1995. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository.Google Scholar
- I-Cheng Yeh. 2008. Blood Transfusion Service Center. UCI Machine Learning Repository.Google Scholar
Recommendations
Autonomicity Levels and Requirements for Automated Machine Learning
RACS '17: Proceedings of the International Conference on Research in Adaptive and Convergent SystemsMachine learning algorithms have various factors to be tuned for successful application. There have been strong demands on automating the tuning process in machine learning practices. This paper characterizes the autonomicity levels at which developers ...
From Zero to AI Hero with Automated Machine Learning
KDD '20: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data MiningAutomated ML is an emerging field in Machine Learning that helps developers and new data scientists with little data science knowledge build Machine Learning models and solutions without understanding the complexity of Learning Algorithm selection, and ...
Comments