Skip to main content
SpringerLink
Log in

A Hybrid Distributed Frequent Itemset Mining Method with Its Application in Medical Diagnosis

  • Conference paper
  • First Online:
Machine Learning for Cyber Security (ML4CS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNSC,volume 12488))

Included in the following conference series:

  • 885 Accesses

Abstract

The world has entered an era of globalization, which represents the explosion of information. The application and development of big data technology make the scale of medical data geometric growth. People cannot intuitively see the correlation and the implicit relationship between complex medical data, which leads to the situation of more data and less knowledge. By using Spark and a variety of data pre-processing techniques and machine learning related algorithms, we implemented a platform which could help patients recommend more accurate treatment plans, help doctors analyze the relationship between diseases, and provide more natural results through a visual interface. Besides, we proposed a distributed frequent itemset mining algorithm (DSDFIM) based on the adjacency list and Spark. After evaluating, the proposed algorithm could reduce data transportation once between main memory and secondary storage, and improved the speed of data processing through distributed computing, compared with the classic algorithm. Meanwhile, it could solve the problem of merging frequent itemset of the same item under different independent paths.

This research is supported by the National Natural Science Foundation of China (No. 62003280), Research Project of Education and Teaching Reform in Southwest University (No. 2019JY053), Fundamental Research Funds for the Central Universities (No. XDJK2019C085), and Chongqing Overseas Scholars Innovation Program (No. cx2018077).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ding, Z., Liu, Y., Jing, S., Zhang, X.: A review of medical data mining. Smart Health 2(10), 54–56 (2016)

    Google Scholar 

  2. Han, J., Pei, J., Yin, Y.: Mining frequent patterns without candidate generation. In: ACM SIGMOD International Conference on Management of Data, pp. 1–12. ACM Press, New York (2000)

    Google Scholar 

  3. Qiu, H., Gu, R., Yuan, C., Huang, Y.: YAFIM: a parallel frequent itemset mining algorithm with spark. In: 2014 IEEE International Parallel & Distributed Processing Symposium Workshops, Phoenix, AZ, USA, pp. 1664–1671. IEEE Computer Society (2014)

    Google Scholar 

  4. Shi, X., Chen, S., Yang, H.: DFPS: distributed FP-growth algorithm based on Spark. In: 2017 IEEE 2nd Advanced Information Technology, Electronic and Automation Control Conference (IAEAC), Chongqing, China, pp. 1725–1731. IEEE (2017)

    Google Scholar 

  5. Miao, Y., Lin, J., Xu, N.: An improved parallel FP-growth algorithm based on Spark and its application. In: 2019 Chinese Control Conference (CCC), Guangzhou, China, pp. 3793–3797. IEEE (2019)

    Google Scholar 

  6. Zhang, W., Luo, K.: A parallel FP-growth mining algorithm based on Spark framework. Comput. Eng. Sci. 39(08), 1403–1409 (2017)

    Google Scholar 

  7. Li, Y., Yin, S.: Mining algorithm for weighted FP-growth frequent item sets based on ordered FP-tree. Int. J. Eng. Manag. Res. 9(5), 154–158 (2019)

    Article  Google Scholar 

  8. Yin, M., Wang, W., Liu, Y., Jiang, D.: An improvement of FP-Growth association rule mining algorithm based on adjacency table. MATEC Web Conf. 189, 10012 (2018)

    Article  Google Scholar 

  9. Li, Y., Yin, S.: Mining algorithm for weighted FP-tree frequent item sets based on two-dimensional table. J. Phys. Conf. Ser. 1453, 012002 (2020)

    Article  Google Scholar 

  10. Xiao, F.: EFMCDM: evidential fuzzy multicriteria decision making based on belief entropy. IEEE Trans. Fuzzy Syst. 28, 1477–1491 (2020)

    Article  Google Scholar 

  11. Xiao, F.: GIQ: a generalized intelligent quality-based approach for fusing multi-source information. IEEE Trans. Fuzzy Syst. (2020). https://doi.org/10.1109/TFUZZ.2020.2991296

    Article  Google Scholar 

  12. Gao, X., Liu, F., Pan, L., Deng, Y., Tsai, S.-B.: Uncertainty measure based on Tsallis entropy in evidence theory. Int. J. Intell. Syst. 34(11), 3105–3120 (2019)

    Article  Google Scholar 

  13. Fan, L., Deng, Y.: Determine the number of unknown targets in Open World based on Elbow method. IEEE Trans. Fuzzy Syst. (2020). https://doi.org/10.1109/TFUZZ.2020.2966182

    Article  Google Scholar 

  14. Fei, L., Deng, Y.: Multi-criteria decision making in Pythagorean fuzzy environment. Appl. Intell. 50, 537–561 (2019). https://doi.org/10.1007/s10489-019-01532-2

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer and Information Science, Southwest University, No. 2 Tiansheng Road, Beibei District, Chongqing, 400715, China

    Mingxue Zhang & Fuyuan Xiao

Authors
  1. Mingxue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fuyuan Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fuyuan Xiao .

Editor information

Editors and Affiliations

  1. Xidian University, Xi'an, China

    Xiaofeng Chen

  2. Guangzhou University, Guangzhou, China

    Hongyang Yan

  3. Michigan State University, East Lansing, MI, USA

    Qiben Yan

  4. Division of Computer, Electrical and Mathematical Sciences and Engineering, King Abdullah University of Science, Thuwal, Saudi Arabia

    Xiangliang Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, M., Xiao, F. (2020). A Hybrid Distributed Frequent Itemset Mining Method with Its Application in Medical Diagnosis. In: Chen, X., Yan, H., Yan, Q., Zhang, X. (eds) Machine Learning for Cyber Security. ML4CS 2020. Lecture Notes in Computer Science(), vol 12488. Springer, Cham. https://doi.org/10.1007/978-3-030-62463-7_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-62463-7_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-62462-0

  • Online ISBN: 978-3-030-62463-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation