Skip to main content
SpringerLink
Log in

Interactive image segmentation with a regression based ensemble learning paradigm

  • Published:
Frontiers of Information Technology & Electronic Engineering Aims and scope Submit manuscript

Abstract

To achieve fine segmentation of complex natural images, people often resort to an interactive segmentation paradigm, since fully automatic methods often fail to obtain a result consistent with the ground truth. However, when the foreground and background share some similar areas in color, the fine segmentation result of conventional interactive methods usually relies on the increase of manual labels. This paper presents a novel interactive image segmentation method via a regression-based ensemble model with semi-supervised learning. The task is formulated as a non-linear problem integrating two complementary spline regressors and strengthening the robustness of each regressor via semi-supervised learning. First, two spline regressors with a complementary nature are constructed based on multivariate adaptive regression splines (MARS) and smooth thin plate spline regression (TPSR). Then, a regressor boosting method based on a clustering hypothesis and semi-supervised learning is proposed to assist the training of MARS and TPSR by using the region segmentation information contained in unlabeled pixels. Next, a support vector regression (SVR) based decision fusion model is adopted to integrate the results of MARS and TPSR. Finally, the GraphCut is introduced and combined with the SVR ensemble results to achieve image segmentation. Extensive experimental results on benchmark datasets of BSDS500 and Pascal VOC have demonstrated the effectiveness of our method, and the comparison with experiment results has validated that the proposed method is comparable with the state-of-the-art methods for interactive natural image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adamowski, J., Chan, H.F., Prasher, S.O., et al., 2012. Com-parison of multivariate adaptive regression splines with coupled wavelet transform artificial neural networks for runoff forecasting in Himalayan micro-watersheds with limited data. J. Hydroinform., 14(3): 731–744. https://doi.org/10.2166/hydro.2011.044

    Article  Google Scholar 

  • Balcan, M.F., Blum, A., Yang, K., 2004. Co-training and expansion: towards bridging theory and practice. 17th Int. Conf. on Neural Information Processing Systems, p.89–96.

    Google Scholar 

  • Blum, A., Mitchell, T., 1998. Combining labeled and unla-beled data with co-training. 11th Annual Conf. on Com-putational Learning Theory, p.92–100. https://doi.org/10.1145/279943.279962

    Google Scholar 

  • Boykov, Y.Y., Jolly, M.P., 2001. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. 8th IEEE Int. Conf. on Computer Vision, p.105–112. https://doi.org/10.1109/ICCV.2001.937505

    Google Scholar 

  • Boykov, Y.Y., Veksler, O., Zabih, R., 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Patt. Anal. Mach. Intell., 23(11): 1222–1239. https://doi.org/10.1109/ICCV.1999.791245

    Article  Google Scholar 

  • Ding, J.J., Lin, C.J., Lu, I.F., et al., 2015. Real-time interactive image segmentation using improved superpixels. IEEE Int. Conf. on Digital Signal Processing, p.740–744. https://doi.org/10.1109/ICDSP.2015.7251974

    Google Scholar 

  • Everingham, M., van Gool, L., Williams, C.K., et al., 2010. The Pascal Visual Object Classes (VOC) challenge. Int. J. Comput. Vis., 88(2): 303–338. https://doi.org/10.1007/s11263-009-0275-4

    Article  Google Scholar 

  • Friedman, J.H., 1991. Multivariate adaptive regression splines. Ann. Statist., 19(1): 1–67.

    Article  MathSciNet  MATH  Google Scholar 

  • Fu, Z., Wang, L., Zhang, D., 2014. An improved multi-label classification ensemble learning algorithm. In: Li, S., Liu, C., Wang, Y. (Eds.), Pattern Recognition. Springer Berlin Heidelberg, p.243–252. https://doi.org/10.1007/978-3-662-45646-0_25

    Google Scholar 

  • Galar, M., Fernandez, A., Barrenechea, E., et al., 2012. A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. C, 42(4): 463–484. https://doi.org/10.1109/TSMCC.2011.2161285

    Article  Google Scholar 

  • Ge, L., Ju, R., Ren, T., et al., 2015. Interactive RGB-D image segmentation using hierarchical graph cut and geodesic distance. In: Ho, Y.S., Sang, J., Ro, Y.M., et al. (Eds.), Advances in Multimedia Information Processing. Springer International Publishing, p.114–124. https://doi.org/10.1007/978-3-319-24075-6_12

    Google Scholar 

  • Gulshan, V., Rother, C., Criminisi, A., et al., 2010. Geodesic star convexity for interactive image segmentation. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, p.3129–3136. https://doi.org/10.1109/CVPR.2010.5540073

    Google Scholar 

  • Jian, M., Jung, C., 2016. Interactive image segmentation using adaptive constraint propagation. IEEE Trans. Image Process., 25(3): 1301–1311. https://doi.org/10.1109/TIP.2016.2518480

    MathSciNet  Google Scholar 

  • Jobst, A.M., Kingston, D.G., Cullen, N.J., et al., 2016. Com-bining thin-plate spline interpolation with a lapse rate model to produce daily air temperature estimates in a data-sparse alpine catchment. Int. J. Climatol., 37(1): 214–229. https://doi.org/10.1002/joc.4699

    Article  Google Scholar 

  • Jung, C., Jian, M., Liu, J., et al., 2014. Interactive image segmentation via kernel propagation. Patt. Recogn., 47(8): 2745–2755. https://doi.org/10.1016/j.patcog.2014.02.010

    Article  Google Scholar 

  • Kolmogorov, V., Zabih, R., 2004. What energy functions can be minimized via graph cuts? IEEE Trans. Patt. Anal. Mach. Intell., 26(2): 147–159. https://doi.org/10.1109/TPAMI.2004.1262177

    Article  MATH  Google Scholar 

  • Lazaridis, A., Mporas, I., Ganchev, T., et al., 2011. Support vector regression fusion scheme in phone duration mod-eling. IEEE Int. Conf. on Acoustics, Speech and Signal Processing, p.4732–4735. https://doi.org/10.1109/ICASSP.2011.5947412

    Google Scholar 

  • Lee, Y.S., Cho, S.B., 2014. Activity recognition with Android phone using mixture-of-experts co-trained with labeled and unlabeled data. Neurocomputing, 126: 106–115. https://doi.org/10.1016/j.neucom.2013.05.044

    Article  Google Scholar 

  • Li, Y., Sun, J., Tang, C.K., et al., 2004. Lazy snapping. ACM Trans. Graph., 23(3): 303–308. https://doi.org/10.1145/1015706.1015719

    Article  Google Scholar 

  • Liu, Y., Yu, Y., 2012. Interactive image segmentation based on level sets of probabilities. IEEE Trans. Visual. Comput. Graph., 18(2): 202–213. https://doi.org/10.1109/TVCG.2011.77

    Article  Google Scholar 

  • Martin, D., Fowlkes, C., Tal, D., et al., 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring eco-logical statistics. 8th IEEE Int. Conf. on Computer Vision, p.416–423. https://doi.org/10.1109/ICCV.2001.937655

    Google Scholar 

  • Menon, R., Bhat, G., Saade, G.R., et al., 2014. Multivariate adaptive regression splines analysis to predict biomarkers of spontaneous preterm birth. Acta Obstetr. Gynecol. Scandinav., 93(4): 382–391. https://doi.org/10.1111/aogs.12344

    Article  Google Scholar 

  • Nguyen, T.N.A., Cai, J., Zhang, J., et al., 2012. Robust inter-active image segmentation using convex active contours. IEEE Trans. Image Process., 21(8): 3734–3743. https://doi.org/10.1109/TIP.2012.2191566

    Article  MathSciNet  Google Scholar 

  • Ning, J., Zhang, L., Zhang, D., et al., 2010. Interactive image segmentation by maximal similarity based region merg-ing. Patt. Recogn., 43(2): 445–456. https://doi.org/10.1016/j.patcog.2009.03.004

    Article  MATH  Google Scholar 

  • Opitz, D., Maclin, R., 1999. Popular ensemble methods: an empirical study. J. Artif. Intell. Res., 11: 169–198. https://doi.org/10.1613/jair.614

    MATH  Google Scholar 

  • Pauchard, Y., Fitze, T., Browarnik, D., et al., 2016. Interactive graph-cut segmentation for fast creation of finite element models from clinical CT data for hip fracture prediction. Comput. Methods Biomech. Biomed. Eng., 19(16): 1693–1703. https://doi.org/10.1080/10255842.2016.1181173

    Article  Google Scholar 

  • Peng, B., Zhang, L., Zhang, D., 2013. A survey of graph the-oretical approaches to image segmentation. Patt. Recogn., 46(3): 1020–1038. https://doi.org/10.1016/j.patcog.2012.09.015

    Article  Google Scholar 

  • Qin, C., Zhang, G., Zhou, Y., et al., 2014. Integration of the saliency-based seed extraction and random walks for image segmentation. Neurocomputing, 129: 378–391. https://doi.org/10.1016/j.neucom.2013.09.021

    Article  Google Scholar 

  • Rother, C., Kolmogorov, V., Blake, A., 2004. GrabCut: in-teractive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3): 309–314. http://dx.doi.org/10.1145/1015706.1015720

    Article  Google Scholar 

  • Shahshahani, B.M., Landgrebe, D.A., 1994. The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon. IEEE Trans. Geosci. Remote Sens., 32(5): 1087–1095. https://doi.org/10.1109/36.312897

    Article  Google Scholar 

  • Tang, M., Gorelick, L., Veksler, O., et al., 2013. GrabCut in one cut. IEEE Int. Conf. on Computer Vision, p.1769–1776. https://doi.org/10.1109/ICCV.2013.222

    Google Scholar 

  • Wang, T., Sun, Q., Ji, Z., et al., 2016. Multi-layer graph con-straints for interactive image segmentation via game theory. Patt. Recogn., 55: 28–44. https://doi.org/10.1016/j.patcog.2016.01.018

    Article  Google Scholar 

  • Wang, X.Y., Wang, Q.Y., Yang, H.Y., et al., 2011a. Color image segmentation using automatic pixel classification with support vector machine. Neurocomputing, 74(18): 3898–3911. https://doi.org/10.1016/j.neucom.2011.08.004

    Article  MATH  Google Scholar 

  • Wang, X.Y., Wang, T., Bu, J., 2011b. Color image segmenta-tion using pixel wise support vector machine classifica-tion. Patt. Recogn., 44(4): 777–787. https://doi.org/10.1016/j.patcog.2010.08.008

    Article  MATH  Google Scholar 

  • Wu, J., Zhao, Y., Zhu, J.Y., et al., 2014. MILCut: a sweeping line multiple instance learning paradigm for interactive image segmentation. IEEE Conf. on Computer Vision and Pattern Recognition, p.256–263. https://doi.org/10.1109/CVPR.2014.40

    Google Scholar 

  • Xiang, S., Nie, F., Zhang, C., et al., 2009. Interactive natural image segmentation via spline regression. IEEE Trans. Image Process., 18(7): 1623–1632. https://doi.org/10.1109/TIP.2009.2018570

    Article  MathSciNet  Google Scholar 

  • Xiang, S., Nie, F., Zhang, C., 2010. Semi-supervised classifi-cation via local spline regression. IEEE Trans. Patt. Anal. Mach. Intell., 32(11): 2039–2053. https://doi.org/10.1109/TPAMI.2010.35

    Article  Google Scholar 

  • Yang, W., Cai, J., Zheng, J., et al., 2010. User-friendly inter-active image segmentation through unified combinatorial user inputs. IEEE Trans. Image Process., 19(9): 2470–2479. https://doi.org/10.1109/TIP.2010.2048611

    Article  MathSciNet  Google Scholar 

  • Zhang, J., Tang, Z., Liu, J., et al., 2016. Recognition of flota-tion working conditions through froth image statistical modeling for performance monitoring. Miner. Eng., 86: 116–129. https://doi.org/10.1016/j.mineng.2015.12.008

    Article  Google Scholar 

  • Zhang, W., Goh, A.T., 2016. Evaluating seismic liquefaction potential using multivariate adaptive regression splines and logistic regression. Geomech. Eng., 10(3): 269–284. https://doi.org/10.12989/gae.2016.10.3.269

    Article  Google Scholar 

  • Zhang, Y., Song, H., Gu, J., et al., 2010. Interactive object extraction using hierarchical graph cuts. Int. Conf. on Audio Language and Image Processing, p.851–858. https://doi.org/10.1109/ICALIP.2010.5685212

    Google Scholar 

  • Zhang, Y., Wen, J., Wang, X., et al., 2014. Semi-supervised learning combining co-training with active learning. Ex-pert Syst. Appl., 41(5): 2372–2378. https://doi.org/10.1016/j.eswa.2013.09.035

    Article  Google Scholar 

  • Zhou, W., Garcia, E.V., 2016. Nuclear image-guided ap-proaches for cardiac resynchronization therapy (CRT). Curr. Cardiol. Rep., 18(1): 1–11. https://doi.org/10.1007/s11886-015-0687-4

    Article  Google Scholar 

  • Zhou, W., Hou, X., Piccinelli, M., et al., 2014. 3D fusion of LV venous anatomy on fluoroscopy venograms with epi-cardial surface on SPECT myocardial perfusion images for guiding CRT LV lead placement. JACC Cardiov. Imag., 7(12): 1239–1248. https://doi.org/10.1016/j.jcmg.2014.09.002

    Article  Google Scholar 

  • Zhou, Z.H., 2011. When semi-supervised learning meets en-semble learning. Front. Electr. Electron. Eng. China, 6(1): 6–16. https://doi.org/10.1007/978-3-642-02326-2_53

    Article  Google Scholar 

  • Zhou, Z.H., Li, M., 2005. Semi-supervised regression with co-training. 19th Int. Joint Conf. on Artificial Intelligence, p.908–913.

    Google Scholar 

  • Zhou, Z.H., Li, M., 2007. Semisupervised regression with cotraining-style algorithms. IEEE Trans. Knowl. Data Eng., 19(11): 1479–1493. https://doi.org/10.1109/TKDE.2007.190644

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Central South University, Changsha, 410083, China

    Jin Zhang, Zhao-hui Tang & Wei-hua Gui

  2. College of Computer and Communication, Hunan University of Technology, Zhuzhou, 412007, China

    Qing Chen

  3. College of Mathematics and Computer Science, Hunan Normal University, Changsha, 410083, China

    Jin-ping Liu

Authors
  1. Jin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhao-hui Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei-hua Gui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jin-ping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhao-hui Tang.

Additional information

Project supported by the National Natural Science Foundation of China (Nos. 61071176, 61171192, and 61272337) and the Doctoral Fund of the Ministry of Education of China (No. 20130162110013)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Tang, Zh., Gui, Wh. et al. Interactive image segmentation with a regression based ensemble learning paradigm. Frontiers Inf Technol Electronic Eng 18, 1002–1020 (2017). https://doi.org/10.1631/FITEE.1601401

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1631/FITEE.1601401

Key words

CLC number

Search

Navigation