Abstract
Segmenting materials’ images is a laborious and time-consuming process, and automatic image segmentation algorithms usually contain imperfections and errors. Interactive segmentation is a growing topic in the areas of image processing and computer vision, which seeks to find a balance between fully automatic methods and fully-manual segmentation processes. By allowing minimal and simplistic interaction from the user in an otherwise automatic algorithm, interactive segmentation is able to simultaneously reduce the time taken to segment an image while achieving better segmentation results. Given the specialized structure of materials’ images and level of segmentation quality required, we show an interactive segmentation framework for materials’ images that has three key contributions: (1) a multi-labeling approach that can handle a large number of structures while still quickly and conveniently allowing manual addition and removal of segments in real-time, (2) multiple extensions to the interactive tools which increase the simplicity of the interaction, and (3) a web interface for using the interactive tools in a client/server architecture. We show a full formulation of each of these contributions and example results from their application.
Similar content being viewed by others
Notes
For clarity, we inverted the image intensity in this figure, as well as several other figures in the later sections. The original is similar in appearance to Fig. 11.
References
Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: iCoseg: Interactive co-segmentation with intelligent scribble guidance. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3169–3176 (2010)
Birkbeck, N., Cobzas, D., Jagersand, M., Murtha, A., Kesztyues, T.: An interactive graph cut method for brain tumor segmentation. In: Workshop on Applications of Computer Vision (WACV), pp. 1–7 (2009)
Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: IEEE International Conference on Computer Vision, vol. 1, pp. 105–112. IEEE Press, New York (2001)
Boykov, Y., Jolly, M.P.: Interactive organ segmentation using graph cuts. In: Delp, S., DiGoia, A., Jaramaz, B. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2000. Lecture Notes in Computer Science, vol. 1935, pp. 147–175. Springer, Berlin (2000)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)
Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)
Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)
Cates, J.E., Lefohn, A., Whitaker, R.T.: Gist: An interactive, gpu-based level-set segmentation. Med. Image Anal. 8(3), 217–231 (2004). http://graphics.cs.ucdavis.edu/lefohn/work/rls/tumorSeg/
Chuang, H., Huffman, L., Comer, M., Simmons, J., Pollak, I.: An automated segmentation for nickel-based superalloy. In: IEEE International Conference on Image Processing, pp. 2280–2283 (2008)
Comer, M., Bouman, C., De Graef, M., Simmons, J.: Bayesian methods for image segmentation. J. Miner. Metals Mater. Soc. 63, 55–57 (2011)
Comer, M., Delp, E.: Parameter estimation and segmentation of noisy or textured images using the EM algorithm and MPM estimation. In: IEEE International Conference on Image Processing, vol. 2, pp. 650–654. IEEE Press, New York (1994)
Comer, M., Delp, E.: The EM/MPM algorithm for segmentation of textured images: analysis and further experimental results. IEEE Trans. Image Process. 9(10), 1731–1744 (2000)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)
Django Software Foundation: Django (version 1.5). http://djangoproject.com
Fragkiadaki, K., Zhang, W., Shi, J., Bernardis, E.: Structural-flow trajectories for unravelling tubular structure bundles. In: Medical Image Computing and Computer-Assisted Intervention, vol. 3, pp. 631–638 (2012)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice Hall, New York (2008)
Heckel, F., Konrad, O., Hahn, H.K., Peitgen, H.O.: Interactive 3D medical image segmentation with energy-minimizing implicit functions. Comput. Graph. 35(2), 275–287 (2011)
Huffman, L., Simmons, J., Pollak, I.: Segmentation of digital microscopy data for the analysis of defect structures in materials using nonlinear diffusion. In: C. Bouman, E. Miller, I. Pollak (eds.) Computational Imaging VI, Proceedings of SPIE (2008)
Huffman, L.M., Simmons, J.P., De Graef, M., Pollak, I.: Shape priors for map segmentation of alloy micrographs using graph cuts. In: IEEE Workshop on Statistical, Signal Processing, pp. 28–30 (2011)
Ibrahim, I.A., Mohamed, F.A., Lavernia, E.J.: Particulate reinforced metal matrix composites: a review. J. Mater. Sci. 26, 1137–1156 (1991)
Jacinto, H., Kchichan, R., Desvignes, M., Prost, R., Valette, S.: A web interface for 3D visualization and interactive segmentation of medical images. In: 17th International Conference on 3D Web Technology (Web 3D 2012), pp. 51–58 (2012)
Jackson, M., Groeber, M.: DREAM3D (2012). http://dream3d.bluequartz.net
Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: Open source scientific tools for Python (2001). http://www.scipy.org/
Kang, Y., Engelke, K., Kalender, W.A.: Interactive 3D editing tools for image segmentation. Med. Image Anal. 8(1), 35–46 (2004)
Kiefer, W.: Intelligent scissoring for interactive segmentation of 3D meshes. Master’s thesis, Princeton University (2004)
Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)
Kuang, Z., Schnieders, D., Zhou, H., Wong, K.Y., Yu, Y., Peng, B.: Learning image-specific parameters for interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 590–597 (2012)
Li, Q., Ni, X., Liu, G.: Ceramic image processing using the second curvelet transform and watershed algorithm. In: IEEE International Conference on Robotics and Biomimetics, pp. 2037–2042 (2007)
Marroquin, J., Mitter, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. J. Am. Stat. Assoc. 76–89 (1987)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)
Mortensen, E.N., Barrett, W.A.: Intelligent scissors for image composition. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’95, pp. 191–198. ACM, New York (1995)
Moschidis, E., Graham, J.: Propagating interactive segmentation of a single 3D example to similar images: an evaluation study using MR images of the prostate. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1472–1475 (2011)
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Pfister, S.S., Betizeau, M., Dehay, C., Douglas, R.J.: INTERSEG: Interactive 3D segmentation (2012). http://n.ethz.ch/student/sabpfist/interseg.htm
Python Software Foundation: Python language reference. http://www.python.org
Reed, R.: The Superalloys: Fundamentals and Applications. Cambridge University Press, Cambridge (2006)
Rollett, A., Gottstein, G., Shvindlerman, L., Molodov, D.: Grain boundary mobility: a brief review. Z. Metallkunde 95, 226–229 (2004)
Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 23(3), 309–314 (2004)
Rowenhorst, D., Lewis, A., Spanos, G.: Three-dimensional analysis of grain topology and interface curvature in a \(\beta \)-titanium alloy. Acta. Mater. 58, 5511–5519 (2010)
Santner, J., Pock, T., Bischof, H.: Interactive multi-label segmentation. In: Asian Conference on Computer Vision, pp. 397–410 (2011)
Schneider, C., Rasband, W., Eliceiri, K.: NIH image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012)
Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)
Shalev-Shwartz, S.: Online Learning: Theory, Algorithms, and Applications. Ph.D. thesis, The Hebrew University of Jerusalem (2007)
Shapiro, L.G., Stockman, G.C.: Computer Vision. Prentice Hall, Upper Saddle River (2001)
Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)
Simmons, J., Bartha, B., De Graef, M., Comer, M.: Development of bayesian segmentation techniques for automated segmentation of titanium alloy images. Microsc. Microanal. 14(S2), 602–603 (2008)
Simmons, J.P., Chuang, P., Comer, M., Spowart, J.E., Uchic, M.D., De Graef, M.: Application and further development of advanced image processing algorithms for automated analysis of serial section image data. Model. Simul. Mater. Sci. Eng. 17(2), 025,002 (2009)
Straehle, C., Koethe, U., Knott, G., Briggman, K., Denk, W., Hamprecht, F.: Seeded watershed cut uncertainty estimators for guided interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 765–772 (2012)
Straehle, C.N., Köthe, U., Knott, G., Hamprecht, F.A.: Carving: scalable interactive segmentation of neural volume electron microscopy images. In: Medical Image Computing and Computer-Assisted Intervention, pp. 653–660 (2011)
Swiler, T.P., Holm, E.A.: Diffusion in polycrystalline microstructures. In: Annual Meeting of the American Ceramic Society (1995)
Tan, J., Saltzman, W.: Biomaterials with hierarchically defined micro and nanoscale structure. Biomaterials 25(17), 3593–3601 (2004)
Top, A., Hamarneh, G., Abugharbieh, R.: Active learning for interactive 3D image segmentation. In: Medical Image Computing and Computer-Assisted Intervention vol. 6893, pp. 603–610 (2011)
Unger, M., Pock, T., Trobin, W., Cremers, D., Bischof, H.: TVSeg–interactive total variation based image segmentation. In: British Machine Vision Conference 2008, pp. 40.1–40.10 (2008)
Veksler, O.: Efficient graph-based energy minimization methods in computer vision. Ph.D. thesis, Cornell University, Ithaca (1999)
Veksler, O., Delong, A.: GCO (2011). http://vision.csd.uwo.ca/code/
Vezhnevets, V., Konouchine, V.: Grow-Cut–interactive multi-label N-D image segmentation. In: Graphicon, pp. 150–156 (2005)
Waggoner, J., Zhou, Y., Simmons, J., De Graef, M., Wang, S.: 3D materials image segmentation by 2D propagation: a graph-cut approach considering homomorphism. IEEE Trans. Image Process. 22, 5282–5293 (2013)
Waggoner, J., Zhou, Y., Simmons, J., Salem, A., De Graef, M., Wang, S.: Interactive grain image segmentation using graph cut algorithms. In: Proceedings of SPIE (Computational Imaging XI), vol. 8657. Burlingame (2013)
Acknowledgments
This work was supported in part by AFOSR FA9550-11-1-0327 and NSF-1017199. A preliminary version of this work has been published in a conference proceedings [59].
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (avi 9465 KB)
Rights and permissions
About this article
Cite this article
Waggoner, J., Zhou, Y., Simmons, J. et al. Graph-cut based interactive segmentation of 3D materials-science images. Machine Vision and Applications 25, 1615–1629 (2014). https://doi.org/10.1007/s00138-014-0616-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-014-0616-3