Skip to main content
SpringerLink
Log in

Graph-cut based interactive segmentation of 3D materials-science images

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Segmenting materials’ images is a laborious and time-consuming process, and automatic image segmentation algorithms usually contain imperfections and errors. Interactive segmentation is a growing topic in the areas of image processing and computer vision, which seeks to find a balance between fully automatic methods and fully-manual segmentation processes. By allowing minimal and simplistic interaction from the user in an otherwise automatic algorithm, interactive segmentation is able to simultaneously reduce the time taken to segment an image while achieving better segmentation results. Given the specialized structure of materials’ images and level of segmentation quality required, we show an interactive segmentation framework for materials’ images that has three key contributions: (1) a multi-labeling approach that can handle a large number of structures while still quickly and conveniently allowing manual addition and removal of segments in real-time, (2) multiple extensions to the interactive tools which increase the simplicity of the interaction, and (3) a web interface for using the interactive tools in a client/server architecture. We show a full formulation of each of these contributions and example results from their application.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. For clarity, we inverted the image intensity in this figure, as well as several other figures in the later sections. The original is similar in appearance to Fig. 11.

    Fig. 1
    figure 1

    Two adjacent slices of a titanium image volume [40]. Image intensity inverted for clarity; the original is similar in appearance to Fig. 11

    Full size image

References

  1. Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: iCoseg: Interactive co-segmentation with intelligent scribble guidance. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3169–3176 (2010)

  2. Birkbeck, N., Cobzas, D., Jagersand, M., Murtha, A., Kesztyues, T.: An interactive graph cut method for brain tumor segmentation. In: Workshop on Applications of Computer Vision (WACV), pp. 1–7 (2009)

  3. Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: IEEE International Conference on Computer Vision, vol. 1, pp. 105–112. IEEE Press, New York (2001)

  4. Boykov, Y., Jolly, M.P.: Interactive organ segmentation using graph cuts. In: Delp, S., DiGoia, A., Jaramaz, B. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2000. Lecture Notes in Computer Science, vol. 1935, pp. 147–175. Springer, Berlin (2000)

  5. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)

    Article  Google Scholar 

  6. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  7. Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)

  8. Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)

    Article  Google Scholar 

  9. Cates, J.E., Lefohn, A., Whitaker, R.T.: Gist: An interactive, gpu-based level-set segmentation. Med. Image Anal. 8(3), 217–231 (2004). http://graphics.cs.ucdavis.edu/lefohn/work/rls/tumorSeg/

  10. Chuang, H., Huffman, L., Comer, M., Simmons, J., Pollak, I.: An automated segmentation for nickel-based superalloy. In: IEEE International Conference on Image Processing, pp. 2280–2283 (2008)

  11. Comer, M., Bouman, C., De Graef, M., Simmons, J.: Bayesian methods for image segmentation. J. Miner. Metals Mater. Soc. 63, 55–57 (2011)

    Article  Google Scholar 

  12. Comer, M., Delp, E.: Parameter estimation and segmentation of noisy or textured images using the EM algorithm and MPM estimation. In: IEEE International Conference on Image Processing, vol. 2, pp. 650–654. IEEE Press, New York (1994)

  13. Comer, M., Delp, E.: The EM/MPM algorithm for segmentation of textured images: analysis and further experimental results. IEEE Trans. Image Process. 9(10), 1731–1744 (2000)

    Article  MATH  Google Scholar 

  14. Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)

    MATH  Google Scholar 

  15. Django Software Foundation: Django (version 1.5). http://djangoproject.com

  16. Fragkiadaki, K., Zhang, W., Shi, J., Bernardis, E.: Structural-flow trajectories for unravelling tubular structure bundles. In: Medical Image Computing and Computer-Assisted Intervention, vol. 3, pp. 631–638 (2012)

  17. Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice Hall, New York (2008)

    Google Scholar 

  18. Heckel, F., Konrad, O., Hahn, H.K., Peitgen, H.O.: Interactive 3D medical image segmentation with energy-minimizing implicit functions. Comput. Graph. 35(2), 275–287 (2011)

    Article  Google Scholar 

  19. Huffman, L., Simmons, J., Pollak, I.: Segmentation of digital microscopy data for the analysis of defect structures in materials using nonlinear diffusion. In: C. Bouman, E. Miller, I. Pollak (eds.) Computational Imaging VI, Proceedings of SPIE (2008)

  20. Huffman, L.M., Simmons, J.P., De Graef, M., Pollak, I.: Shape priors for map segmentation of alloy micrographs using graph cuts. In: IEEE Workshop on Statistical, Signal Processing, pp. 28–30 (2011)

  21. Ibrahim, I.A., Mohamed, F.A., Lavernia, E.J.: Particulate reinforced metal matrix composites: a review. J. Mater. Sci. 26, 1137–1156 (1991)

    Article  Google Scholar 

  22. Jacinto, H., Kchichan, R., Desvignes, M., Prost, R., Valette, S.: A web interface for 3D visualization and interactive segmentation of medical images. In: 17th International Conference on 3D Web Technology (Web 3D 2012), pp. 51–58 (2012)

  23. Jackson, M., Groeber, M.: DREAM3D (2012). http://dream3d.bluequartz.net

  24. Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: Open source scientific tools for Python (2001). http://www.scipy.org/

  25. Kang, Y., Engelke, K., Kalender, W.A.: Interactive 3D editing tools for image segmentation. Med. Image Anal. 8(1), 35–46 (2004)

    Article  Google Scholar 

  26. Kiefer, W.: Intelligent scissoring for interactive segmentation of 3D meshes. Master’s thesis, Princeton University (2004)

  27. Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)

    Article  Google Scholar 

  28. Kuang, Z., Schnieders, D., Zhou, H., Wong, K.Y., Yu, Y., Peng, B.: Learning image-specific parameters for interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 590–597 (2012)

  29. Li, Q., Ni, X., Liu, G.: Ceramic image processing using the second curvelet transform and watershed algorithm. In: IEEE International Conference on Robotics and Biomimetics, pp. 2037–2042 (2007)

  30. Marroquin, J., Mitter, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. J. Am. Stat. Assoc. 76–89 (1987)

  31. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)

  32. Mortensen, E.N., Barrett, W.A.: Intelligent scissors for image composition. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’95, pp. 191–198. ACM, New York (1995)

  33. Moschidis, E., Graham, J.: Propagating interactive segmentation of a single 3D example to similar images: an evaluation study using MR images of the prostate. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1472–1475 (2011)

  34. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MATH  MathSciNet  Google Scholar 

  35. Pfister, S.S., Betizeau, M., Dehay, C., Douglas, R.J.: INTERSEG: Interactive 3D segmentation (2012). http://n.ethz.ch/student/sabpfist/interseg.htm

  36. Python Software Foundation: Python language reference. http://www.python.org

  37. Reed, R.: The Superalloys: Fundamentals and Applications. Cambridge University Press, Cambridge (2006)

    Book  Google Scholar 

  38. Rollett, A., Gottstein, G., Shvindlerman, L., Molodov, D.: Grain boundary mobility: a brief review. Z. Metallkunde 95, 226–229 (2004)

    Article  Google Scholar 

  39. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 23(3), 309–314 (2004)

  40. Rowenhorst, D., Lewis, A., Spanos, G.: Three-dimensional analysis of grain topology and interface curvature in a \(\beta \)-titanium alloy. Acta. Mater. 58, 5511–5519 (2010)

    Article  Google Scholar 

  41. Santner, J., Pock, T., Bischof, H.: Interactive multi-label segmentation. In: Asian Conference on Computer Vision, pp. 397–410 (2011)

  42. Schneider, C., Rasband, W., Eliceiri, K.: NIH image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012)

  43. Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)

  44. Shalev-Shwartz, S.: Online Learning: Theory, Algorithms, and Applications. Ph.D. thesis, The Hebrew University of Jerusalem (2007)

  45. Shapiro, L.G., Stockman, G.C.: Computer Vision. Prentice Hall, Upper Saddle River (2001)

    Google Scholar 

  46. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)

    Article  Google Scholar 

  47. Simmons, J., Bartha, B., De Graef, M., Comer, M.: Development of bayesian segmentation techniques for automated segmentation of titanium alloy images. Microsc. Microanal. 14(S2), 602–603 (2008)

    Article  Google Scholar 

  48. Simmons, J.P., Chuang, P., Comer, M., Spowart, J.E., Uchic, M.D., De Graef, M.: Application and further development of advanced image processing algorithms for automated analysis of serial section image data. Model. Simul. Mater. Sci. Eng. 17(2), 025,002 (2009)

    Article  Google Scholar 

  49. Straehle, C., Koethe, U., Knott, G., Briggman, K., Denk, W., Hamprecht, F.: Seeded watershed cut uncertainty estimators for guided interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 765–772 (2012)

  50. Straehle, C.N., Köthe, U., Knott, G., Hamprecht, F.A.: Carving: scalable interactive segmentation of neural volume electron microscopy images. In: Medical Image Computing and Computer-Assisted Intervention, pp. 653–660 (2011)

  51. Swiler, T.P., Holm, E.A.: Diffusion in polycrystalline microstructures. In: Annual Meeting of the American Ceramic Society (1995)

  52. Tan, J., Saltzman, W.: Biomaterials with hierarchically defined micro and nanoscale structure. Biomaterials 25(17), 3593–3601 (2004)

    Article  Google Scholar 

  53. Top, A., Hamarneh, G., Abugharbieh, R.: Active learning for interactive 3D image segmentation. In: Medical Image Computing and Computer-Assisted Intervention vol. 6893, pp. 603–610 (2011)

  54. Unger, M., Pock, T., Trobin, W., Cremers, D., Bischof, H.: TVSeg–interactive total variation based image segmentation. In: British Machine Vision Conference 2008, pp. 40.1–40.10 (2008)

  55. Veksler, O.: Efficient graph-based energy minimization methods in computer vision. Ph.D. thesis, Cornell University, Ithaca (1999)

  56. Veksler, O., Delong, A.: GCO (2011). http://vision.csd.uwo.ca/code/

  57. Vezhnevets, V., Konouchine, V.: Grow-Cut–interactive multi-label N-D image segmentation. In: Graphicon, pp. 150–156 (2005)

  58. Waggoner, J., Zhou, Y., Simmons, J., De Graef, M., Wang, S.: 3D materials image segmentation by 2D propagation: a graph-cut approach considering homomorphism. IEEE Trans. Image Process. 22, 5282–5293 (2013)

    Article  Google Scholar 

  59. Waggoner, J., Zhou, Y., Simmons, J., Salem, A., De Graef, M., Wang, S.: Interactive grain image segmentation using graph cut algorithms. In: Proceedings of SPIE (Computational Imaging XI), vol. 8657. Burlingame (2013)

Download references

Acknowledgments

This work was supported in part by AFOSR FA9550-11-1-0327 and NSF-1017199. A preliminary version of this work has been published in a conference proceedings [59].

Author information

Authors and Affiliations

  1. University of South Carolina, Columbia, USA

    Jarrell Waggoner, Youjie Zhou & Song Wang

  2. Materials and Manufacturing Directorate, Air Force Research Labs, Dayton, USA

    Jeff Simmons

  3. Department of Materials Science and Engineering, Carnegie Mellon University, Pittsburgh, USA

    Marc De Graef

Authors
  1. Jarrell Waggoner
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Youjie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jeff Simmons
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marc De Graef
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Song Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Song Wang.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (avi 9465 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Waggoner, J., Zhou, Y., Simmons, J. et al. Graph-cut based interactive segmentation of 3D materials-science images. Machine Vision and Applications 25, 1615–1629 (2014). https://doi.org/10.1007/s00138-014-0616-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0616-3

Keywords

Search

Navigation