Skip to main content
Log in

Semantic segmentation based on fusion of features and classifiers

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a feed forward architecture algorithm using fusion of features and classifiers for semantic segmentation. The algorithm consists of three phases: Firstly, the features from hierarchical convolutional neural network (CNN) and the features based on region are extracted and fused on super pixel level; secondly, multiple classifiers of Softmax, XGBoost and Random Forest are ensemble to compute the per-pixel class probabilities; at last, a fully connected conditional random field is employed to enhance the final performance. The hierarchical features contain more global evidence and the region features contain more local evidence. So the fusion of these two features is expected to enhance the feature representation ability. In classification phase, integrating multiple classifiers aims to improve the generalization ability of classification algorithms. Experiments are conducted on Sift-Flow datasets by our proposed methods with competitive labeling accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127

    Article  MATH  Google Scholar 

  3. Bertasius G, Shi J, Torresani L (2015) High-for-low and low for-high: Efficient boundary detection from deep object features and its applications to high-level vision. In: Proceedings of the IEEE International Conference on Computer Vision, p 504–512

  4. Bu S, Han P, Liu Z, Han J (2016) Scene parsing using inference embedded deep networks. Pattern Recongnition 59:188–198

    Article  Google Scholar 

  5. Byeon W, Breuel TM, Raue F, Liwicki M (2015) Scene labeling with LSTM recurrent neural network. In: CVPR

  6. Caesar H, Jasper U, Ferrari V. Region-based semantic segmentation with end-to-end training. arXiv1607.07671

  7. Carreira J, Caseiro R, Batista J, Sminchisescu C (2012) Semantic segmentation with second-order pooling. In: ECCV

  8. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2015) Semantic image segmentation with deep convolutional nets and fully connected crfs. In: ICLRW

  9. Dai J, He K, Sun J (2015) Convolutional feature masking for joint object and stuff segmentation. In: CVPR

  10. Eigen D, Puhrsch C, Fergus R (2014) Depth map prediction from a single imae using a multi-scale deep network. In: NIPS

  11. Farabet C, Couprie C, Najman L, LeCun Y (2013) Learning hierarchial features for scene labeling. IEEE TPAMI

  12. Gao Z, Zhang L-f, Chen M-y, Hauptmann A, Zhang H, Cai A (2014) Enhanced and hierarchical structure algorithm for data imbalance problem in semantic extraction under massive video dataset. Multimed Tools Appl 68(3):641–657

    Article  Google Scholar 

  13. Gatta C, Romero A, van de Veijer J (2014) Unrolling loopy top-down semantic feedback in convolutional deep networks. In: Workshop at CVPR

  14. Girshick R (2015) Fast R-CNN. In: ICCV

  15. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: CVPR

  16. Hariharan B, Arbelaez P, Girshick R, Malik J (2014) Simultaneous detection and segmentation. In: ECCV

  17. He K, Zhang X, Ren S, Sun J (2016) Deep Residual Learning for Image Recognition, CVPR

  18. Janai J, Güney F, Behl A, Geiger A (2017) Computer vision for autonomous vehicles: problems, datasets and state-of-the-art. arXiv:1704.05519v1

  19. Kokkinos I (2015) Pushing the boundaries of boundary detection using deep learning. arXiv preprint arXiv:1511.07386

  20. Krahenbuhl P, Koltun V (2011) Efficient inference in fully connected crfs with Gaussian edge potentials. In: NI-PS

  21. Li F, Carreira J, Lebanon G, Sminchisescu C (2013) Composite statistical inference for semantic segmentation. In: CVPR

  22. Liu A-A, Su Y-T, Nie W-Z, Kankanhalli M (2017) Hierarchical clustering multi-task learning for joint human action grouping and recognition. IEEE Trans Pattern Anal Mach Intell 39(1):102–114

    Article  Google Scholar 

  23. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: IEEE CVPR

  24. Nie W-Z, Liu A-A, Gao Z, Su Y-T (2015) Clique-graph matching by preserving global & local structure. In: CVPR 2015. IEEE, Boston

  25. Pinheiro P, Collobert R (2014) Recurrent convolutional neural networks for scene labeling. In: ICML

  26. Plath N, Toussaint M, Nakajima S (2009) Multi-class image segmentation using conditional random fields and global classification. In: ICML

  27. Wu Z, Shen C, Hengel AVD (2016) High-performance semantic segmentation using very deep fully convolutional networks. arXiv preprint arXiv:1604.04339

  28. Yang Y, Song J, Huang Z, Ma Z, Sebe N, Hauptmann AG (2013) Multi-feature fusion via hierarchical regression for multimedia analysis. IEEE Trans Multimedia 15(3):572–581

    Article  Google Scholar 

  29. Zavaschi THH, Britto AS, Oliveira LES, Koerich AL (2013) Fusion of feature sets and classifiers for facial expression recognition. Expert Syst Appl 40(2):646–655

    Article  Google Scholar 

  30. Zhang H, Shang X, Luan H, Wang M, Chua T-S (2016) Learning from collective intelligence: feature learning using social images and tags. ACM Trans Multimed Comput Commun Appl 13:1529–1537

  31. Zheng S, Jayasuamana S, Romera-Paredes B, Vineet V, Su Z, Du D, Huang C, Torr P. Conditional random fields as recurrent neural networks. arXiv preprint ar-Xiv: 1502.03240

Download references

Acknowledgements

This research has been supported by National Natural Science Foundation of China (U1509207, 61472278, 61403281 and 61572357).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yanbing Xue.

Additional information

Yanbing Xue and Huiqiang Geng should be considered as joint first authors.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, Y., Geng, H., Zhang, H. et al. Semantic segmentation based on fusion of features and classifiers. Multimed Tools Appl 77, 22199–22211 (2018). https://doi.org/10.1007/s11042-018-5858-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5858-z

Keywords

Navigation