Abstract
In natural scene images, rare class objects have low occurrence frequencies and limited spatial coverage, and they may be easily ignored during scene labeling. However, rare class objects are often more important to semantic labeling and image understanding compared to background areas. In this work, we present a rare class-oriented scene labeling framework (RCSL) that involves two new techniques pertaining to rare classes. First, scene assisted rare class retrieval is introduced in label transfer that is intended to enrich the retrieval set with scene-relevant rare classes. Second, a complementary rare class balanced CNN is incorporated to address the unbalanced training data issues, where rare classes are usually dominated by common ones in natural scene images. Furthermore, a superpixels-based re-segmentation was implemented to produce perceptually meaningful object boundaries. Experimental results demonstrate promising scene labeling performance of the proposed framework on the SIFTflow dataset both qualitatively and quantitatively, especially for rare class objects.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Caesar, H., Uijlings, J., Ferrari, V.: Joint calibration for semantic segmentation. In: Proceedings of BMVC, pp. 29.1–29.13. BMVA Press (2015)
Yang, J., Price, B., Cohen, S., Yang, M.H.: Context driven scene parsing with attention to rare classes. In: Proceedings of CVPR, pp. 3294–3301 (2014)
Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: Sun database: large-scale scene recognition from abbey to zoo. In: Proceedings of CVPR, pp. 3485–3492 (2010)
Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. PAMI 25, 1075–1088 (2003)
Li, L.J., Socher, R., Fei-Fei, L.: Towards total scene understanding: classification, annotation and segmentation in an automatic framework. In: Proceedings of CVPR, pp. 2036–2043 (2009)
George, M.: Image parsing with a wide range of classes and scene-level context. In: Proceedings of CVPR, pp. 3622–3630 (2015)
LeCun, Y., Boser, B., Denker, J.S., Henderson, D., Howard, R.E., Hubbard, W., Jackel, L.D.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989)
Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. PAMI 35, 1915–1929 (2013)
Pinheiro, P.H.O., Collobert, R.: Recurrent convolutional neural networks for scene labeling. In: Proceedings of ICML, pp. 82–90 (2014)
Liang, M., Hu, X., Zhang, B.: Convolutional neural networks with intra-layer recurrent connections for scene labeling. In: Advances in Neural Information Processing Systems 28. Curran Associates, Inc., pp. 937–945 (2015)
Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing: label transfer via dense scene alignment. In: Proceedings of CVPR, pp. 1972–1979 (2009)
Liu, C., Yuen, J., Torralba, A.: Nonparametric scene parsing via label transfer. IEEE Trans. PAMI 33, 2368–2382 (2011)
Gould, S., Zhang, Y.: PatchMatchGraph: building a graph of dense patch correspondences for label transfer. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 439–452. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33715-4_32
Tung, F., Little, J.J.: CollageParsing: nonparametric scene parsing by adaptive overlapping windows. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 511–525. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10599-4_33
Eigen, D., Fergus, R.: Nonparametric image parsing using adaptive neighbor sets. In: Proceedings of CVPR, pp. 2799–2806 (2012)
Singh, G., Kosecka, J.: Nonparametric scene parsing with adaptive feature relevance and semantic context. In: Proceedings of CVPR, pp. 3151–3157 (2013)
Tighe, J., Lazebnik, S.: SuperParsing: scalable nonparametric image parsing with superpixels. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6315, pp. 352–365. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15555-0_26
Tighe, J., Lazebnik, S.: Finding things: image parsing with regions and per-exemplar detectors. In: Proceedings of CVPR, pp. 3001–3008 (2013)
Shuai, B., Wang, G., Zuo, Z., Wang, B., Zhao, L.: Integrating parametric and non-parametric models for scene labeling. In: Proceedings of CVPR, pp. 4249–4258 (2015)
Shuai, B., Zuo, Z., Wang, G., Wang, B.: Scene parsing with integration of parametric and non-parametric models. IEEE Trans. Image Process. 25, 2379–2391 (2016)
Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient graph-based image segmentation. IJCV 59, 167–181 (2004)
Gould, S., Zhao, J., He, X., Zhang, Y.: Superpixel graph label transfer with learned distance metric. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 632–647. Springer, Heidelberg (2014). doi:10.1007/978-3-319-10590-1_41
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Yu, L., Fan, G. (2016). Rare Class Oriented Scene Labeling Using CNN Incorporated Label Transfer. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2016. Lecture Notes in Computer Science(), vol 10072. Springer, Cham. https://doi.org/10.1007/978-3-319-50835-1_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-50835-1_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-50834-4
Online ISBN: 978-3-319-50835-1
eBook Packages: Computer ScienceComputer Science (R0)