Skip to main content
SpringerLink
Log in

Feature Selection with Positive Region Constraint for Test-Cost-Sensitive Data

  • Chapter
Transactions on Rough Sets XVIII

Part of the book series: Lecture Notes in Computer Science ((TRS,volume 8449))

  • 509 Accesses

Abstract

In many data mining and machine learning applications, data are not free, and there is a test cost for each data item. Due to economic, technological and legal reasons, it is neither possible nor necessary to obtain a classifier with 100 % accuracy. In this paper, we consider such a situation and propose a new constraint satisfaction problem to address it. With this in mind, one has to minimize the test cost to keep the accuracy of the classification under a budget. The constraint is expressed by the positive region, whereas the object is to minimizing the total test cost. The new problem is essentially a dual of the test cost constraint attribute reduction problem, which has been addressed recently. We propose a heuristic algorithm based on the information gain, the test cost, and a user specified parameter \(\lambda \) to deal with the new problem. The algorithm is tested on four University of California - Irvine datasets with various test cost settings. Experimental results indicate that the algorithm finds optimal feature subset in most cases, the rational setting of \(\lambda \) is different among datasets, and the algorithm is especially stable when the test cost is subject to the Pareto distribution.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 34.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chen, X.: An improved branch and bound algorithm for feature selection. Pattern Recogn. Lett. 24(12), 1925–1933 (2003)

    Article  Google Scholar 

  2. Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. 1, 131–156 (1997)

    Article  Google Scholar 

  3. Fayyad, U., Piatetsky-Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Mag. 17, 37–54 (1996)

    Google Scholar 

  4. Greco, S., Matarazzo, B., Slowinski, R., Stefanowski, J.: Variable consistency model of dominance-based rough sets approach. In: Ziarko, W.P., Yao, Y. (eds.) RSCTC 2000. LNCS (LNAI), vol. 2005, pp. 170–181. Springer, Heidelberg (2001)

    Chapter  MATH  Google Scholar 

  5. Blake, C.L., Merz, C.J.: UCI repository of machine learning databases (1998). http://www.ics.uci.edu/~mlearn/mlrepository.html

  6. He, H.P., Min, F.: Accumulated cost based test-cost-sensitive attribute reduction. In: Kuznetsov, S.O., Ślȩzak, D., Hepting, D.H., Mirkin, B.G. (eds.) RSFDGrC 2011. LNCS, vol. 6743, pp. 244–247. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  7. He, H.P., Min, F., Zhu, W.: Attribute reduction in test-cost-sensitive decision systems with common-test-costs. In: Proceedings of the 3rd International Conference on Machine Learning and Computing, vol. 1, pp. 432–436 (2011)

    Google Scholar 

  8. Hu, Q.H., Yu, D.R., Liu, J.F., Wu, C.: Neighborhood rough set based heterogeneous feature subset selection. Inf. Sci. 178(18), 3577–3594 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  9. Hunt, E.B., Marin, J., Stone, P.J. (eds.): Experiments in Induction. Academic Press, New York (1966)

    Google Scholar 

  10. Lanzi, P.: Fast feature selection with genetic algorithms: a filter approach. In: IEEE International Conference on Evolutionary Computation 1997. IEEE (1997)

    Google Scholar 

  11. Lin, T.Y.: Granular computing on binary relations - analysis of conflict and Chinese wall security policy. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 296–299. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  12. Lin, T.Y.: Granular computing - structures, representations, and applications. In: Wang, G., Liu, Q., Yao, Y., Skowron, A. (eds.) RSFDGrC 2003. LNCS (LNAI), vol. 2639, pp. 16–24. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  13. Liu, Q.H., Li, F., Min, F., Ye, M., Yang, W.G.: An efficient reduction algorithm based on new conditional information entropy. Control Decis. (in Chinese) 20(8), 878–882 (2005)

    MATH  Google Scholar 

  14. Liu, J.B., Min, F., Liao, S.J., Zhu, W.: A genetic algorithm to attribute reduction with test cost constraint. In: Proceedings of 6th International Conference on Computer Sciences and Convergence Information Technology, pp. 751–754 (2011)

    Google Scholar 

  15. Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. The Springer International Series in Engineering and Computer Science, vol. 454. Kluwer Academic Publishers, Boston (1998)

    Book  MATH  Google Scholar 

  16. Ma, L.W.: On some types of neighborhood-related covering rough sets. Int. J. Approx. Reason. 53(6), 901–911 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  17. Min, F., He, H.P., Qian, Y.H., Zhu, W.: Test-cost-sensitive attribute reduction. Inf. Sci. 181, 4928–4942 (2011)

    Article  Google Scholar 

  18. Min, F., Hu, Q.H., Zhu, W.: Feature selection with test cost constraint. Int. J. Approximate Reasoning (2013, to appear). doi:10.1016/j.ijar.2013.04.003

    Article  MathSciNet  MATH  Google Scholar 

  19. Min, F., Liu, Q.H.: A hierarchical model for test-cost-sensitive decision systems. Inf. Sci. 179, 2442–2452 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  20. Min, F., Zhu, W.: Attribute reduction with test cost constraint. J. Electr. Sci. Technol. China 9(2), 97–102 (2011)

    Google Scholar 

  21. Min, F., Zhu, W.: Minimal cost attribute reduction through backtracking. In: Kim, T., et al. (eds.) DTA/BSBT 2011. CCIS, vol. 258, pp. 100–107. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  22. Min, F., Zhu, W.: Optimal sub-reducts in the dynamic environment. In: Proceedings of IEEE International Conference on Granular Computing, pp. 457–462 (2011)

    Google Scholar 

  23. Min, F., Zhu, W.: Optimal sub-reducts with test cost constraint. In: Yao, J.T., Ramanna, S., Wang, G., Suraj, Z. (eds.) RSKT 2011. LNCS, vol. 6954, pp. 57–62. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  24. Min, F., Zhu, W., Zhao, H., Pan, G.Y., Liu, J.B., Xu, Z.L.: Coser: cost-sensitive rough sets (2012). http://grc.fjzs.edu.cn/~fmin/coser/

  25. Pan, G.Y., Min, F., Zhu, W.: A genetic algorithm to the minimal test cost reduct problem. In: Proceedings of IEEE International Conference on Granular Computing. pp. 539–544 (2011)

    Google Scholar 

  26. Pawlak, Z.: Rough set approach to knowledge-based decision support. Eur. J. Oper. Res. 99, 48–57 (1997)

    Article  MATH  Google Scholar 

  27. Pawlak, Z.: Rough sets. Int. J. Comput. Inf. Sci. 11, 341–356 (1982)

    Article  MATH  Google Scholar 

  28. Pawlak, Z.: Rough sets and intelligent data analysis. Inf. Sci. 147(12), 1–12 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  29. Qian, Y.H., Liang, J.Y., Pedrycz, W., Dang, C.Y.: Positive approximation: an accelerator for attribute reduction in rough set theory. Artif. Intell. 174(9–10), 597–618 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  30. Skowron, A., Rauszer, C.: The discernibility matrices and functions in information systems. In: Intelligent Decision Support (1992)

    Chapter  Google Scholar 

  31. Swiniarski, R.W., Skowron, A.: Rough set methods in feature selection and recognition. Pattern Recogn. Lett. 24(6), 833–849 (2003)

    Article  MATH  Google Scholar 

  32. Tseng, T.L.B., Huang, C.-C.: Rough set-based approach to feature selection in customer relationship management. Omega 35(4), 365–383 (2007)

    Article  Google Scholar 

  33. Wang, G.Y.: Attribute core of decision table. In: Alpigini, J.J., Peters, J.F., Skowron, A., Zhong, N. (eds.) RSCTC 2002. LNCS (LNAI), vol. 2475, pp. 213–217. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  34. Wang, X., Yang, J., Teng, X., Xia, W., Jensen, R.: Feature selection based on rough sets and particle swarm optimization. Pattern Recogn. Lett. 28(4), 459–471 (2007)

    Article  Google Scholar 

  35. Xu, Z.L., Min, F., Liu, J.B., Zhu, W.: Ant colony optimization to minimal test cost reduction. In: Proceedings of the 2011 IEEE International Conference on Granular Computing. pp. 688–693 (2012)

    Google Scholar 

  36. Yao, J.T., Zhang, M.: Feature selection with adjustable criteria. In: Ślȩzak, D., Wang, G., Szczuka, M.S., Düntsch, I., Yao, Y. (eds.) RSFDGrC 2005. LNCS (LNAI), vol. 3641, pp. 204–213. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  37. Yao, Y.Y., Zhao, Y.: Attribute reduction in decision-theoretic rough set models. Inf. Sci. 178(17), 3356–3373 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhang, W.X., Mi, J., Wu, W.: Knowledge reductions in inconsistent information systems. Chin. J. Comput. 26(1), 12–18 (2003)

    MathSciNet  Google Scholar 

  39. Zhao, H., Min, F., Zhu, W.: A backtracking approach to minimal cost feature selection of numerical data. J. Inf. Comput. Sci. 10(13), 4105–4115 (2013)

    Article  Google Scholar 

  40. Zhao, H., Min, F., Zhu, W.: Test-cost-sensitive attribute reduction based on neighborhood rough set. In: Proceedings of the 2011 IEEE International Conference on Granular Computing, pp. 802–806 (2011)

    Google Scholar 

  41. Zhao, H., Min, F., Zhu, W.: Test-cost-sensitive attribute reduction of data with normal distribution measurement errors. Math. Prob. Eng. 2013, 12 pp (2013)

    Google Scholar 

  42. Zhong, N., Dong, Z.J., Ohsuga, S.: Using rough sets with heuristics to feature selection. J. Intell. Inf. Syst. 16(3), 199–214 (2001)

    Article  MATH  Google Scholar 

  43. Zhu, W.: Generalized rough sets based on relations. Inf. Sci. 177(22), 4997–5011 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  44. Zhu, W.: Topological approaches to covering rough sets. Inf. Sci. 177(6), 1499–1508 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  45. Zhu, W.: Relationship between generalized rough sets based on binary relation and covering. Inf. Sci. 179(3), 210–225 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  46. Zhu, W., Wang, F.: Reduction and axiomization of covering generalized rough sets. Inf. Sci. 152(1), 217–230 (2003)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the Natural Science Foundation of Department of Education of Sichuan Province under Grant No. 13ZA0136, and National Science Foundation of China under Grant Nos. 61379089, 61379049.

Author information

Authors and Affiliations

  1. Department of Computer Science, Sichuan University for Nationalities, Kangding, 626001, China

    Jiabin Liu

  2. Lab of Granular Computing, Zhangzhou Normal University, Zhangzhou, 363000, China

    Jiabin Liu, Fan Min, Hong Zhao & William Zhu

Authors
  1. Jiabin Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fan Min .

Editor information

Editors and Affiliations

  1. ECE Department Computational Intelligence Laboratory, University of Manitoba, Winnipeg, MB, Canada

    James F. Peters

  2. University of Warsaw, Warsaw, Poland

    Andrzej Skowron  & Hung Son Nguyen  & 

  3. Southwest Jiaotong University, Sichuan, China

    Tianrui Li  & Yan Yang  & 

  4. University of Regina, Regina, SK, Canada

    JingTao Yao

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Liu, J., Min, F., Zhao, H., Zhu, W. (2014). Feature Selection with Positive Region Constraint for Test-Cost-Sensitive Data. In: Peters, J.F., Skowron, A., Li, T., Yang, Y., Yao, J., Nguyen, H.S. (eds) Transactions on Rough Sets XVIII. Lecture Notes in Computer Science(), vol 8449. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44680-5_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44680-5_2

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44679-9

  • Online ISBN: 978-3-662-44680-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation