Skip to main content
SpringerLink
Log in

A fundamental approach to discover closed periodic-frequent patterns in very large temporal databases

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Periodic frequent-pattern mining (PFPM) is a vital knowledge discovery technique that identifies periodically occurring patterns in a temporal database. Although traditional PFPM algorithms have many applications, they often produce a large set of periodic-frequent patterns (PFPs) in a database. As a result, analyzing PFPs can be very time-consuming for users. Moreover, a large set of PFPs makes PFPM algorithms less efficient regarding runtime and memory consumption. This paper handles this problem by proposing a novel model of closed periodic-frequent patterns (CPFPs) found in databases. CPFPs are less expensive to mine because they represent a concise and lossless subset uniquely describing the entire set of PFPs. We also present an efficient depth-first search algorithm, called Closed Periodic-Frequent Pattern-Miner (CPFP-Miner), to discover the patterns. The proposed algorithm utilizes the weighted ordering of the patterns concept to reduce the patterns’ search space. On the other hand, the current periodicity concept is also applied to prune aperiodic patterns from the search space. Extensive experiments on both real-world and synthetic databases demonstrate that the CPFP-Miner algorithm is efficient. It outperforms the state-of-the-art algorithms regarding runtime requirements, memory consumption, and energy consumption on several real-world and synthetic databases. Additionally, the scalability of the CPFP-Miner algorithm is demonstrated to be more effective and productive than the state-of-the-art algorithms. Finally, we present two case studies to show the functionality of the proposed patterns.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Algorithm 1
Algorithm 2
Algorithm 3
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Data availability

We have downloaded the above mentioned databases from the well known open-source data mining library named sequence pattern mining framework found at [59]. We have also used one more real-world database named Drought found at [60].

Code availability

To ensure the repeatability of our experiments, we made the complete evaluation results, as well as the databases and algorithms, available on GitHub [62].

References

  1. Agrawal R, TImieliński T, Swami A (1993) Mining association rules between sets of items in large databases. In SIGMOD, p 207–216

  2. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Proceedings of the 20th International Conference on Very Large Data Bases, VLDB ‘94 . Morgan Kaufmann Publishers Inc., San Francisco, CA, pp 487–499

  3. Han J, Pei J, Yin Y, Mao R (2004) Mining frequent patterns without candidate generation: A frequent-pattern tree approach. Data Min Knowl Discov 8(1):53–87

    Article  MathSciNet  Google Scholar 

  4. Han J, Cheng H, Xin D, Yan X (2007) Frequent pattern mining: Current status and future directions. Data Min Knowl Disc, 14(1)

  5. Aggarwal CC (2014) Applications of Frequent Pattern Mining, pages 443–467. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-07821-2_18

  6. Fournier-Viger P, Lin JC-W, Kiran RU, Koh YS (2017) A survey of sequential pattern mining. Data Sci Pattern Recog 1(1):54–77

    Google Scholar 

  7. Luna JM, Fournier-Viger P, Ventura S (2019) Frequent itemset mining: A 25 years review. Wiley Interdiscip. Rev Data Min Knowl Discov. 9(6)

  8. Tanbeer SK, Ahmed CF, Jeong B-S, Lee Y-K (2009) Discovering periodic-frequent patterns in transactional databases. In Advances in Knowledge Discovery and Data Mining, p 242–253

  9. Kiran RU, Kitsuregawa M (2014) Novel techniques to reduce search space in periodic-frequent pattern mining. In Database Systems for Advanced Applications, p 377–391, Cham, 2014. Springer International Publishing

  10. Tanbeer SK, Hassan MM, Almogren A, Zuair M, Jeong B (2017) Scalable regular pattern mining in evolving body sensor data. Future Gener Comp Syst 75:172–186

    Article  Google Scholar 

  11. Amphawan K, Lenca P, Surarerks A (2009) Mining top-k periodic-frequent pattern from transactional databases without support threshold. Adv Inf Technol, p18–29

  12. Fournier-Viger P, Yang P, Kiran RU, Ventura S, Luna JM (2021) Mining local periodic patterns in a discrete sequence. Inf Sci 544:519–548

    Article  MathSciNet  MATH  Google Scholar 

  13. Kiran RU, Shang H, Toyoda M, Kitsuregawa M (2015) Discovering recurring patterns in time series. In Proceedings of the 18th International Conference on Extending Database Technology, p 97–108

  14. Kiran RU, Venkatesh JN, Toyoda M, Kitsuregawa M, Reddy PK (2017) Discovering partial periodic-frequent patterns in a transactional database. J Syst Softw 125:170–182

    Article  Google Scholar 

  15. Kiran RU, Veena P, Ravikumar P, Saideep C, Zettsu K, Shang H, Toyoda M, Kitsuregawa M, Reddy PK (2022) Efficient discovery of partial periodic patterns in large temporal databases. Electronics, 11(10). https://doi.org/10.3390/electronics11101523

  16. Nakamura S, Kiran RU, Likhitha P, Ravikumar P, Watanobe Y, Dao MS, Zettsu K, Toyoda M (2021) Efficient discovery of partial periodic-frequent patterns in temporal databases. In Christine Strauss, Gabriele Kotsis, A. Min Tjoa, and Ismail Khalil, editors, Database and Expert Systems Applications, p 221–227, Cham. Springer International Publishing

  17. Kiran RU, Watanobe Y, Chaudhury B, Zettsu K, Toyoda M, Kitsuregawa M (2020) Discovering maximal periodic-frequent patterns in very large temporal databases. In 2020 IEEE 7th International Conference on Data Science and Advanced Analytics (DSAA), p 11–20

  18. Fournier-Viger P, Yang P, Lin JC-W, Kiran RU (2019) Discovering stable periodic-frequent patterns in transactional data. In: Wotawa F, Friedrich G, Pill I, Koitz-Hristov R, Ali M (eds.), Advances and Trends in Artificial Intelligence. From Theory to Practice, pages 230–244, Cham. Springer International Publishing

  19. Dao HN, Ravikumar P, Likhitha P, Rage UK, Watanobe Y, Paik I (2023) Finding stable periodic-frequent itemsets in big columnar databases. IEEE Access 11:12504–12524. https://doi.org/10.1109/ACCESS.2023.3241313

    Article  Google Scholar 

  20. Fournier-Viger P, Wang Y, Yang P, Lin JC-W, Yun U, Kiran RU (2022) Tspin: mining top-k stable periodic patterns. Appl Intell 52(6):6917–6938. https://doi.org/10.1007/s10489-020-02181-6

    Article  Google Scholar 

  21. Kiran RU, Saideep C, Ravikumar P, Zettsu K, Toyoda M, Kitsuregawa M, Reddy PK (2020) Discovering fuzzy periodic-frequent patterns in quantitative temporal databases. In 2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pages 1–8

  22. Dinh D-T, Le B, Fournier-Viger P, Huynh V-N (2018) An efficient algorithm for mining periodic high-utility sequential patterns. Appl Intell 48(12):4694–4714. https://doi.org/10.1007/s10489-018-1227-x

    Article  MATH  Google Scholar 

  23. Fournier-Viger P, Li Z, Lin JC, Kiran RU, Fujita H (2019) Efficient algorithms to identify periodic patterns in multiple sequences. Inf Sci 489:205–226

    Article  MathSciNet  MATH  Google Scholar 

  24. Pasquier N, Bastide Y, Taouil R, Lakhal L (1999) Discovering frequent closed itemsets for association rules. In ICDT, p 398–416

  25. Pei J, Han J, Mao R (2000) Closet: An efficient algorithm for mining frequent closed itemsets. In SIGMOD Int’l Workshop on Data Mining and Knowledge Discovery, p 21–30

  26. Burdick D, Calimlim M, Flannick J, Gehrke J, Yiu T (2005) Mafia: A maximal frequent itemset algorithm. IEEE Trans Knowl Data Eng 17(11):1490–1504

    Article  Google Scholar 

  27. Bastide Y, Taouil R, Pasquier N, Stumme G, Lakhal L (2000) Mining frequent patterns with counting inference. SIGKDD Explor Spec Issue Scalable Algorithm 2(2):71–80

    MATH  Google Scholar 

  28. Zaki MJ, Hsiao C (2002) CHARM: an efficient algorithm for closed itemset mining. In SIAM SDM, p 457–473

  29. Grahne G, Zhu J (2005) Fast algorithms for frequent itemset mining using fp-trees. IEEE Trans Knowl Data Eng 17:1347–1362

    Article  Google Scholar 

  30. Likhitha P, Ravikumar P, Uday Kiran R, Hayamizu Y, Goda K,Toyoda M, Zettsu K, Shrivastava S (2020) Discovering closed periodic-frequent patterns in very large temporal databases. In 2020 IEEE International Conference on Big Data (Big Data), p 4700–4709. https://doi.org/10.1109/BigData50022.2020.9378215

  31. Anirudh A, Uday Kirany R, Krishna Reddy P, Kitsuregaway M (2016) Memory efficient mining of periodic-frequent patterns in transactional databases. In 2016 IEEE Symposium Series on Computational Intelligence (SSCI), p 1–8. https://doi.org/10.1109/SSCI.2016.7849926

  32. Ravikumar P, Likhitha P, Venus Vikranth Raj B, Uday Kiran R, Watanobe Y, Zettsu K (2021) Efficient discovery of periodic-frequent patterns in columnar temporal databases. Electronics, 10(12). https://doi.org/10.3390/electronics10121478

  33. Cheng-Wei W, Huang J, Lin Y-W, Chuang C-Y, Tseng Y-C (2022) Efficient algorithms for deriving complete frequent itemsets from frequent closed itemsets. Appl Intell 52(6):7002–7023. https://doi.org/10.1007/s10489-020-02172-7

    Article  Google Scholar 

  34. Bayardo RJ (1998) Efficiently mining long patterns from databases. SIGMOD Rec 27(2):85–93

    Article  Google Scholar 

  35. Burdick D, Calimlim M, Gehrke J (2001) Mafia: a maximal frequent itemset algorithm for transactional databases. In Proceedings 17th International Conference on Data Engineering, p 443–452

  36. Gouda K, Zaki MJ (2001) Efficiently mining maximal frequent itemsets. In ICDM, p 163–170

  37. Grahne G, Zhu J (2003) High performance mining of maximal frequent itemsets. In 6th International Workshop on High Performance Data Mining

  38. Jiang N, Gruenwald L (2006) Cfi-stream: Mining closed frequent itemsets in data streams. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’06, p 592–597, New York, NY, USA. Association for Computing Machinery. https://doi.org/10.1145/1150402.1150473

  39. Lin D-I, Kedem ZM (1998) Pincer-search: A new algorithm for discovering the maximum frequent set. In Advances in Database Technology — EDBT’98, p 103–119, Berlin, Heidelberg. Springer Berlin Heidelberg

  40. Zaki MJ (2000) Scalable algorithms for association mining. IEEE Trans Knowl Data Eng 12(3):372–390. https://doi.org/10.1109/69.846291

    Article  Google Scholar 

  41. Karim MR, Cochez M, Beyan OD, Ahmed CF, Decker S (2018) Mining maximal frequent patterns in transactional databases and dynamic data streams: A spark-based approach. Inf Sci 432:278–300

    Article  MathSciNet  MATH  Google Scholar 

  42. Zaki MJ, Gouda K (2003) Fast vertical mining using diffsets. In Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’03, page 326–335, New York, NY, USA. Association for Computing Machinery

  43. Yen S-J, Lee Y-S, Wang C-K (2014) An efficient algorithm for incrementally mining frequent closed itemsets. Appl Intell 40(4):649–668

    Article  Google Scholar 

  44. Chanda AK, Saha S, Nishi MA, Samiullah M, Ahmed CF (2015) An efficient approach to mine flexible periodic patterns in time series databases. Eng Appl Artif Intell 44:46–63

    Article  Google Scholar 

  45. Han J, Gong W, Yin Y (1998) Mining segment-wise periodic patterns in time-related databases. KDD 98:214–218

    Google Scholar 

  46. Han J, Dong G, Yin Y (1999) Efficient mining of partial periodic patterns in time series database. In Proceedings 15th International Conference on Data Engineering (Cat. No. 99CB36337), pages 106–115. IEEE

  47. Kim H, Yun U, Vo B, Lin JC-W, Pedrycz W (2021) Periodicity-oriented data analytics on time-series data for intelligence system. IEEE Syst J 15(4):4958–4969

    Article  Google Scholar 

  48. Nishi MA, Ahmed CF, Samiullah M, Jeong B-S (2013) Effective periodic pattern mining in time series databases. Expert Syst Appl 40(8):3015–3027

    Article  Google Scholar 

  49. Rasheed F, Alhajj R (2010) Stnr: A suffix tree based noise resilient algorithm for periodicity detection in time series databases. Appl Intell 32(3):267–278

    Article  Google Scholar 

  50. Yang R, Wang W, Yu PS (2002) Infominer+: mining partial periodic patterns with gap penalties. In 2002 IEEE International Conference on Data Mining, 2002. Proceedings., pages 725–728. IEEE

  51. Ozden B, Ramaswamy S, Silberschatz A: Cyclic association rules (1998) proceedings of the fourteenth international conference on data engineering. Orlando, FL, USA, IEEE Computer Society, Washington, p 412–421

  52. Kiran RU, Reddy PK (2011) An alternative interestingness measure for mining periodic-frequent patterns. In DASFAA (1), p 183–192

  53. Kiran RU, Kitsuregawa M, Reddy PK (2016) Efficient discovery of periodic-frequent patterns in very large databases. J Syst Softw 112:110–121. https://doi.org/10.1016/j.jss.2015.10.035

    Article  Google Scholar 

  54. Venkatesh JN, Kiran RU, Reddy PK, Kitsuregawa M (2016) Discovering periodic-frequent patterns in transactional databases using all-confidence and periodic-all-confidence. In Database and Expert Systems Applications - 27th International Conference, DEXA 2016, Porto, Portugal, September 5–8, 2016, Proceedings, Part I, pages 55–70

  55. Surana A, Kiran RU, Reddy PK (2011) An efficient approach to mine periodic-frequent patterns in transactional databases. In PAKDD Workshops, p 254–266

  56. Rashid MM, Karim MR, Jeong BS, Choi HJ (2012) Efficient mining regularly frequent patterns in transactional databases. In International Conference on Database Systems for Advanced Applications (1), p 258–271

  57. Dao HN, Ravikumar P, Likitha P, Raj BVV, Kiran RU, Watanobe Y, Paik I (2022) Towards efficient discovery of stable periodic patterns in big columnar temporal databases. In: Fujita H, Fournier-Viger P, Ali M,Wang Y (eds.) Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence, pages 831–843, Cham. Springer International Publishing

  58. Yokogawa. WT1800 Precision Power Analyzer. https://tmi.yokogawa.com/eu/library/resources/white-papers/wt1800-precision-power-analyzer, 2011. [Online; accessed 31 Jan 2021]

  59. Fournier-Viger P (2022) Spmf: A java open-source data mining library. http://www.philippe-fournier-viger.com/spmf/index.php?link=datasets.php. [Online; accessed 4 Apr 2022]

  60. National Center for Atmospheric Research, University Corporation for Atmospheric Research. Standardized precipitation index (spi) for global land surface (1949–2012) (2013) Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, Boulder CO

  61. Japan The Ministry of Environment. Soramame. http://soramame.taiki.go.jp/. [Online; accessed 12 Dec 2020]

  62. Kiran RU (2023) PAMI: PAttern MIning. https://github.com/udayRage/PAMI/tree/main/PAMI/periodicFrequentPattern. [Online; accessed 12 Apr 2023]

  63. Rostami M, Berahmand K, Nasiri E, Forouzandeh S (2021) Review of swarm intelligence-based feature selection methods. Eng Appl Artif Intell 100:104210

    Article  Google Scholar 

  64. Saberi-Movahed F, Rostami M, Berahmand K, Karami S, Tiwari P, Oussalah M, Band SS (2022) Dual regularized unsupervised feature selection based on matrix factorization and minimum redundancy with application in gene selection. Knowl-Based Syst 256:109884

    Article  Google Scholar 

Download references

Funding

This research was funded by JSPS Kakenhi 21K12034.

Author information

Author notes

  1. Veena Pamalla and Uday Kiran Rage are authors contributed equally to this work

Authors and Affiliations

  1. Department of Computer Science, Sri Balaji PG College, JNTU, Ananthapuram, 515001, Andhra Pradesh, India

    Veena Pamalla

  2. The University of Aizu, Fukushima, Japan

    Uday Kiran Rage, Ravikumar Penugonda, Likhitha Palla & Shrivastava Sourabh

  3. NICT, Tokyo, Japan

    Uday Kiran Rage & Koji Zettsu

  4. IIIT-Idupulapaya, RGUKT-AP, Vempalli, India

    Ravikumar Penugonda

  5. The University of Tokyo, Tokyo, Japan

    Yuto Hayamizu, Kazuo Goda & Masashi Toyoda

Authors
  1. Veena Pamalla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uday Kiran Rage
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ravikumar Penugonda
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Likhitha Palla
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yuto Hayamizu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Kazuo Goda
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Masashi Toyoda
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Koji Zettsu
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Shrivastava Sourabh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Uday Kiran Rage.

Ethics declarations

Conflicts of interests/Competing interests

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pamalla, V., Rage, U.K., Penugonda, R. et al. A fundamental approach to discover closed periodic-frequent patterns in very large temporal databases. Appl Intell 53, 27344–27373 (2023). https://doi.org/10.1007/s10489-023-04811-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04811-1

Keywords

Search

Navigation