Skip to main content
SpringerLink
Log in

Federated deep active learning for attention-based transaction classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Cyber-physical transactions can be clustered and classified using an effective vector representation. Traditionally, methods for finding patterns have relied on heuristics and pruning. Despite the high dimensionality of transactional data, approaches that use frequent item sets as features can be affected by dimensionality, sparsity, and privacy issues. As a result, FIs are not uniformly distributed over the data. In this paper, we propose an embedded architecture for transaction classification based on distributed learning. The model transforms transaction data into frequent sets and implements attention mechanisms based on encoder-decoder structures. Consequently, the model can learn continuous vectors in low dimensions while maintaining context and colocation. We analyzed a high-dimensional transaction dataset to test attention-based methods and federated learning. To improve decision limits while maintaining privacy and security, the proposed model lowers the global loss function. In the experiment, four datasets are used for comparison. The data are randomly selected and distributed to different clients for each dataset. We run each experiment with five different random partitions of the dataset to evaluate the active learning. The training set is the same size in each round, and the test set is not evaluated in each round. Our methods are compared to the best performing baseline method using the F1 score and the percentage of the dataset used. Compared to the baseline model, the proposed model performed better in terms of percentage increases and output classes, namely retail (1.7%, 2), cancer (17.38%, 3), food (0.74%, 2), and snippet (3.47%, 8).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Algorithm 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Liu Y, Yu J, Xu L, Wang L, Yang J (2022) Sissos: intervention of tabular data and its applications. Appl Intell 52(1):1044–1058

    Article  Google Scholar 

  2. Ahmed U, Srivastava G, Lin JC-W (2021) A federated learning approach to frequent itemset mining in cyber-physical systems. J Netw Syst Manag 29(4):1–17

    Article  Google Scholar 

  3. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules. In: Proceeding of the 20th international conference on very large databases, vol 1215, pp 487–499

  4. Fournier-Viger P, Lin JC-W, Kiran RU, Koh YS, Thomas R (2017) A survey of sequential pattern mining. Data Sci Pattern Recognit 1(1):54–77

    Google Scholar 

  5. Mannila H, Toivonen H, Verkamo AI (1997) Discovery of frequent episodes in event sequences. Data Min Knowl Disc 1(3):259–289

    Article  Google Scholar 

  6. Lin CW, Hong TP, Hsu HC (2014) Reducing side effects of hiding sensitive itemsets in privacy preserving data mining. Sci World J 2014:235837

    Article  Google Scholar 

  7. Lin JCW, et al. (2016) Fast algorithms for hiding sensitive high-utility itemsets in privacy-preserving utility mining. Eng Appl Artif Intel 55:269–284

    Article  Google Scholar 

  8. Cheng P, Roddick JF, Chu S-C, Lin C-W (2016) Privacy preservation through a greedy, distortion-based rule-hiding method. Appl Intell 44:295–306

    Article  Google Scholar 

  9. Lin JCW, Srivastava G, Zhang Y, Djenouri Y, Aloqaily M (2021) Privacy-preserving multiobjective sanitization model in 6G IoT environments. IEEE Internet Things J 8(7):5340–5349

    Article  Google Scholar 

  10. Yang Q, Liu Y, Chen T, Tong Y (2019) Federated machine learning: concept and applications. ACM Trans Intell Syst Technol 10(2):12:1–12:19

    Article  Google Scholar 

  11. Ahmed U, Lin J, C-W, Srivastava G (2022) Hyper-graph attention based federated learning method for mental health detection. IEEE Journal of Biomedical and Health Informatics

  12. Vaswani A, et al. (2017) Attention is all you need. Adv Neural Inf Process Syst, 5998–6008

  13. Bi M, Zhang Q, Zuo M, Xu Z, Jin Q (2020) Bi-directional LSTM model with symptoms-frequency position attention for question answering system in medical domain. Neural Process Lett 51(2):1185–1199

    Article  Google Scholar 

  14. Lu J, Yang J, Batra D, Parikh D (2016) Hierarchical question-image co-attention for visual question answering. Adv Neural Inf Process Syst, 289–297

  15. Luong M-T, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. arXiv:1508.04025

  16. Xu K, et al. (2015) Show, attend and tell: Neural image caption generation with visual attention. In: Proceedings of the 32nd international conference on machine learning, vol 37, pp 2048–2057

  17. Cheng H, Yan X, Han J, Hsu C (2007) Discriminative frequent pattern analysis for effective classification. In: Proceedings of the international conference on data engineering, pp 716– 725

  18. Chang K, et al. (2018) Distributed deep learning networks among institutions for medical imaging. J Am Med Inform Assoc 25(8):945–954

    Article  Google Scholar 

  19. He Z, et al. (2017) Conditional discriminative pattern mining: concepts and algorithms. Inform Sci 375:1–15

    Article  Google Scholar 

  20. Nguyen D, Nguyen TD, Luo W, Venkatesh S (2018) Trans2vec: learning transaction embedding via items and frequent itemsets. Adv Knowl Discov Data Min 10939:361–372

    Article  Google Scholar 

  21. Fournier-Viger P, Chun-Wei Lin J, Truong-Chi T, Nkambou R (2019) A survey of high utility itemset mining. In: High-utility pattern mining. Springer, pp 1–45

  22. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Adv Neural Inf Process Syst, 3111–3119

  23. Grover A, Leskovec J (2016) node2vec: scalable feature learning for networks. In: ACM SIGKDD international conference on knowledge discovery and data mining, pp 855–864

  24. Ahmed U, Lin JC-W, Srivastava G (2021) Fuzzy explainable attention-based deep active learning on mental-health data. In: IEEE international conference on fuzzy systems, pp 1–6

  25. Nguyen G, et al. (2019) Machine learning and deep learning frameworks and libraries for large-scale data mining: a survey. Artif Intell Rev 52(1):77–124

    Article  Google Scholar 

  26. Cho K, et al. (2014) Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: Proceedings of the conference on empirical methods in natural language processing, pp 1724–1734

  27. Mothukuri V, et al. (2020) A survey on security and privacy of federated learning. Future Generation Computer Systems

  28. Shokri R, Shmatikov V (2015) Privacy-preserving deep learning. In: Proceedings of the ACM SIGSAC conference on computer and communications security, pp 1310–1321

  29. Hayes J, Ohrimenko O (2019) Contamination attacks and mitigation in multi-party machine learning. arXiv:1901.02402

  30. Fredrikson M, Jha S, Ristenpart T (2015) Model inversion attacks that exploit confidence information and basic countermeasures. In: Proceedings of the 22nd ACM SIGSAC conference on computer and communications security, pp 1322–1333

  31. Mohassel P, Rindal P (2018) Aby3: a mixed protocol framework for machine learning. In: ACM SIGSAC conference on computer and communications security, pp 35–52

  32. Fournier-Viger P, et al. (2016) The SPMF open-source data mining library version 2. In: Joint European conference on machine learning and knowledge discovery in databases, pp 36–40

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Electrical Engineering and Mathematical, Western Norway University of Applied Sciences, Bergen, Norway

    Usman Ahmed & Jerry Chun-Wei Lin

  2. College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China

    Philippe Fournier-Viger

Authors
  1. Usman Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jerry Chun-Wei Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philippe Fournier-Viger
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jerry Chun-Wei Lin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Emerging Topics in Artificial Intelligence Selected from IEA/AIE2021

Guest Editors: Ali Selamat and Jerry Chun-Wei Lin

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, U., Lin, J.CW. & Fournier-Viger, P. Federated deep active learning for attention-based transaction classification. Appl Intell 53, 8631–8643 (2023). https://doi.org/10.1007/s10489-022-04388-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04388-1

Keywords

Search

Navigation