Skip to main content
Springer Nature Link
Log in

3P-ECLAT: mining partial periodic patterns in columnar temporal databases

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Partial periodic pattern (3P) mining is a vital data mining technique that aims to discover all interesting patterns that have exhibited partial periodic behavior in temporal databases. Previous studies have primarily focused on identifying 3Ps only in row temporal databases. One can not ignore the existence of 3Ps in columnar temporal databases as many real-world applications, such as Facebook and Adobe, employ them to store their big data. This paper proposes an efficient single database scan algorithm, Partial Periodic Pattern-Equivalence Class Transformation (3P-ECLAT), to identify all 3Ps in a columnar temporal database. The proposed algorithm compresses the given database into a novel list-based data structure and mines it recursively to find all 3Ps. The 3P-ECLAT leverages the “downward closure property” and “depth-first search technique” to reduce the search space and the computational cost. Extensive experiments have been conducted on synthetic and real-world databases to demonstrate the efficiency of the 3P-ECLAT algorithm. The memory and runtime results show that 3P-ECLAT outperforms its competitor considerably. Furthermore, 3P-ECLAT is highly scalable and is superior to the previous approach in handling large databases. Finally, to demonstrate the practical utility of our algorithm, we provide two real-world case studies, one on analyzing traffic congestion during disasters and another on identifying the highly polluted areas in Japan.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability Statements

We have downloaded the above-mentioned databases from the well-known website [60]. We have also used two more real-world databases, Congestion and Drought. Unfortunately, we could not make these datasets public for confidentiality reasons.

Code Availability

To ensure the repeatability of our experiments, our algorithms were made available on GitHub [61]

Notes

  1. ACID stands for Atomicity, Consistency, Isolation, and Duration

  2. BASE stands for Basically Available, Soft state, and Eventually consistent

References

  1. MySQL: MySQL. https://www.mysql.com/

  2. PostGres: PostGres. https://www.postgresql.org/

  3. Snowflake: SnowFlake. https://www.snowflake.com/

  4. BigQuery: BigQuery. https://cloud.google.com/bigquery

  5. Agrawal R, Imieliński T, Swami A (1993) Mining association rules between sets of items in large databases. In: SIGMOD, pp 207–216

  6. Luna JM, Fournier-Viger P, Ventura S (2019) Frequent itemset mining: A 25 years review. Wiley Interdiscip. Rev Data Min Knowl Discov 9(6)

  7. Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. SIGMOD Rec 29(2):1–12. https://doi.org/10.1145/335191.335372

    Article  Google Scholar 

  8. Zaki MJ (2000) Scalable algorithms for association mining. IEEE Transactions on knowledge and data engineering 12(3):372–390

    Article  Google Scholar 

  9. Fournier-Viger P, Lin JC-W, Kiran RU, Koh YS (2017) A survey of sequential pattern mining. Data Sci Pattern Recog 1(1):54–77

    Google Scholar 

  10. Chui C-K, Kao B, Hung E (2007) Mining frequent itemsets from uncertain data. In: Zhou Z-H, Li H, Yang Q (eds) Advances in Knowledge Discovery and Data Mining. Springer, Berlin, Heidelberg, pp 47–58

    Chapter  Google Scholar 

  11. Yan X, Han J (2002) gspan: graph-based substructure pattern mining. In: 2002 IEEE International conference on data mining, 2002. Proceedings, pp 721–724. https://doi.org/10.1109/ICDM.2002.1184038

  12. Chang JH, Lee WS (2003) Finding recent frequent itemsets adaptively over online data streams. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining. KDD ’03, pp 487–492. Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/956750.956807

  13. Tanbeer SK, Ahmed CF, Jeong B-S, Lee Y-K (2009) Discovering periodicfrequent patterns in transactional databases. In: PAKDD, pp 242–253

  14. Amphawan K, Lenca P, Surarerks A (2009) Mining top-k periodic-frequent pattern from transactional databases without support threshold. In: Advances in information technology, pp 18–29

  15. Kiran RU, Kitsuregawa M, Reddy PK (2016) Efficient discovery of periodicfrequent patterns in very large databases. J Syst Softw 112:110–121. https://doi.org/10.1016/j.jss.2015.10.035

    Article  Google Scholar 

  16. Kiran RU, Shang H, Toyoda M, Kitsuregawa M (2017) Discovering partial periodic itemsets in temporal databases. In: Proceedings of the 29th international conference on scientific and statistical database management. SSDBM ’17

  17. Kiran RU, Veena P, Ravikumar P, Saideep C, Zettsu K, Shang H, Toyoda M, Kitsuregawa M, Reddy PK (2022) Efficient discovery of partial periodic patterns in large temporal databases. Electronics 11(10). https://doi.org/10.3390/electronics11101523

  18. Zuckerberg M (2023) Facebook.https://www.facebook.com. Accessed 14- Sep-2023

  19. Narayen S (2023) Adobe. https://www.adobe.com. Accessed 14-Sep-2023

  20. Paul Foeckler JR (2023) Victor Henning Mendeley. https://www.mendeley.com. Accessed 14-Sep-2023

  21. Ravikumar P, Raj VV, Likhitha P, Kiran RU, Watanobe Y, Ito S, Zettsu K, Toyoda M (2022) Towards efficient discovery of partial periodic patterns in columnar temporal databases. In: Nguyen NT, Tran TK, Tukayev U, Hong T-P, Trawiński B, Szczerbicki E (eds) Intelligent information and database systems. Springer, Cham, pp 141–154

    Chapter  Google Scholar 

  22. Chanda AK, Saha S, Nishi MA, Samiullah M, Ahmed CF (2015) An efficient approach to mine flexible periodic patterns in time series databases. Eng Appl Artif Intell 44:46–63. https://doi.org/10.1016/j.engappai.2015.04.014

    Article  Google Scholar 

  23. Han J, Gong W, Yin Y (1998) Mining segment-wise periodic patterns in time-related databases. KDD 98:214–218

    Google Scholar 

  24. Han J, Dong G, Yin Y (1999) Efficient mining of partial periodic patterns in time series database. In: Proceedings 15th international conference on data engineering (Cat. No. 99CB36337). IEEE, pp 106–115

  25. Kim H, Yun U, Vo B, Lin JC-W, Pedrycz W (2021) Periodicity-oriented data analytics on time-series data for intelligence system. IEEE Systems Journal 15(4):4958–4969. https://doi.org/10.1109/JSYST.2020.3022640

    Article  Google Scholar 

  26. Nishi MA, Ahmed CF, Samiullah M, Jeong B-S (2013) Effective periodic pattern mining in time series databases. Expert Syst Appl 40(8):3015–3027. https://doi.org/10.1016/j.eswa.2012.12.017

  27. Rasheed F, Alhajj R (2010) Stnr: A suffix tree based noise resilient algorithm for periodicity detection in time series databases. Applied Intelligence 32(3):267–278. https://doi.org/10.1007/s10489-008-0144-9

    Article  Google Scholar 

  28. Yang R, Wang W, Yu PS (2002) (2002) Infominer+: mining partial periodic patterns with gap penalties. 2002 IEEE International conference on data mining. Proceedings, IEEE pp, pp 725–728

    Google Scholar 

  29. Özden B, Ramaswamy S, Silberschatz A (1998) Cyclic association rules proceedings of the fourteenth international conference on data engineering. Orlando, FL, USA, IEEE Computer Society, Washington, 412–421

  30. Kiran RU, Reddy PK (2011) An alternative interestingness measure for mining periodic-frequent patterns. In: DASFAA (1), pp 183–192

  31. Kiran RU, Kitsuregawa M (2014) Novel techniques to reduce search space in periodicfrequent pattern mining. Database systems for advanced applications. Springer, Cham, pp 377–391

    Chapter  Google Scholar 

  32. Anirudh A, Kirany RU, Reddy PK, Kitsuregaway M (2016) Memory efficient mining of periodic-frequent patterns in transactional databases. In: 2016 IEEE Symposium series on computational intelligence (SSCI), pp 1–8. https://doi.org/10.1109/SSCI.2016.7849926

  33. Surana A, Kiran RU, Reddy PK (2011) An efficient approach to mine periodicfrequent patterns in transactional databases. In: PAKDD Workshops, pp 254–266

  34. Ravikumar P, Likhitha P, Venus Vikranth Raj B, Uday Kiran R, Watanobe Y, Zettsu K (2021) Efficient discovery of periodic-frequent patterns in columnar temporal databases. Electronics 10(12). https://doi.org/10.3390/electronics10121478

  35. Penugonda R, Palla L, Rage UK, Watanobe Y, Zettsu K (2021) Towards efficient discovery of periodic-frequent patterns in columnar temporal databases. In: Ali M (ed) Fujita H, Selamat A, Lin JC-W. Advances and Trends in Artificial Intelligence. Artificial Intelligence Practices. Springer, Cham, pp 28–40

    Google Scholar 

  36. Likhitha P, Ravikumar P, Kiran RU, Watanobe Y (2022) Discovering top-k periodic-frequent patterns in very large temporal databases. In: Roy PP, Agarwal A, Li T, Krishna Reddy P, Uday Kiran R (eds) Big Data Analytics. Springer, Cham, pp 200–210

    Chapter  Google Scholar 

  37. Fournier-Viger P, Yang P, Kiran RU, Ventura S, Luna JM (2021) Mining local periodic patterns in a discrete sequence. Inf Sci 544:519–548

    Article  MathSciNet  Google Scholar 

  38. Fournier-Viger P, Li Z, Lin JC, Kiran RU, Fujita H (2019) Efficient algorithms to identify periodic patterns in multiple sequences. Inf Sci 489:205–226

    Article  MathSciNet  Google Scholar 

  39. Kiran RU, Saideep C, Ravikumar P, Zettsu K, Toyoda M, Kitsuregawa M, Reddy PK (2020) Discovering fuzzy periodic-frequent patterns in quantitative temporal databases. In: 2020 IEEE International conference on fuzzy systems (FUZZ-IEEE), pp 1–8

  40. Kiran RU, Watanobe Y, Chaudhury B, Zettsu K, Toyoda M, Kitsuregawa M (2020) Discovering maximal periodic-frequent patterns in very large temporal databases. In: 2020 IEEE 7th International conference on data science and advanced analytics (DSAA), pp 11–20

  41. Kiran RU, Shang H, Toyoda M, Kitsuregawa M (2015) Discovering recurring patterns in time series. In: Proceedings of the 18th international conference on extending database technology, pp 97–108

  42. Fournier-Viger P, Wang Y, Yang P, Lin JC-W, Yun U, Kiran RU (2022) Tspin: Mining top-k stable periodic patterns. Appl Intell 52(6):6917–6938. https://doi.org/10.1007/s10489-020-02181-6

    Article  Google Scholar 

  43. Ishita SZ, Ahmed CF, Leung CK (2022) New approaches for mining regular high utility sequential patterns. Appl Intell 52(4):3781–3806. https://doi.org/10.1007/s10489-021-02536-7

  44. Ravikumar P, Kiran RU, Likhitha P, Chandrasekhar T, Watanobe Y, Zettsu K (2022) Discovering geo-referenced periodic-frequent patterns in georeferenced time series databases. In: 2022 IEEE 9th International conference on data science and advanced analytics (DSAA), pp 1–10. https://doi.org/10.1109/DSAA54385.2022.10032391

  45. Veena P, Ravikumar P, Kwangwari K, Kiran RU, Goda K, Watanobe Y, Zettsu K (2022) Discovering fuzzy geo-referenced periodic-frequent patterns in geo-referenced time series databases. In: 2022 IEEE International conference on fuzzy systems (FUZZ-IEEE), pp 1–8. https://doi.org/10.1109/FUZZ-IEEE55066.2022.9882785

  46. Zhang X, Qi Y, Chen G, Gan W, Fournier-Viger P (2022) Fuzzy-driven periodic frequent pattern mining. Inf Sci 618:253–269

    Article  Google Scholar 

  47. Fournier-Viger P, Yang P, Lin JC-W, Kiran RU (2019) Discovering stable periodic-frequent patterns in transactional data. In: Wotawa F, Friedrich G, Pill I, Koitz-Hristov R, Ali M (eds) Advances and Trends in Artificial Intelligence. From Theory to Practice, Springer, Cham, pp 230–244

    Google Scholar 

  48. Dao HN, Ravikumar P, Likitha P, Raj BVV, Kiran RU, Watanobe Y, Paik I (2022) Towards efficient discovery of stable periodic patterns in big columnar temporal databases. In: Fujita H, Fournier-Viger P, Ali M, Wang Y (eds) Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence, Springer, Cham, pp 831–843

    Google Scholar 

  49. Dao HN, Ravikumar P, Likhitha P, Rage UK, Watanobe Y, Paik I (2023) Finding stable periodic-frequent itemsets in big columnar databases. IEEE Access 11:12504–12524. https://doi.org/10.1109/ACCESS.2023.3241313

    Article  Google Scholar 

  50. Kiran RU, Venkatesh JN, Toyoda M, Kitsuregawa M, Reddy PK (2017) Discovering partial periodic-frequent patterns in a transactional database. J Syst Softw 125:170–182

    Article  Google Scholar 

  51. Nakamura S, Kiran RU, Likhitha P, Ravikumar P, Watanobe Y, Dao MS, Zettsu K, Toyoda M (2021) Efficient discovery of partial periodic-frequent patterns in temporal databases. In: Strauss C, Kotsis G, Tjoa AM, Khalil I (eds) Database and Expert Systems Applications. Springer, Cham, pp 221–227

    Chapter  Google Scholar 

  52. Rashid MM, Karim MR, Jeong BS, Choi HJ (2012) Efficient mining regularly frequent patterns in transactional databases. In: International conference on database systems for advanced applications (1), pp 258–271

  53. Fournier-Viger P, Wang Y, Yang P, Lin JC, Yun U, Kiran RU (2022) TSPIN: mining top-k stable periodic patterns. Appl Intell 52(6):6917–6938. https://doi.org/10.1007/s10489-020-02181-6

    Article  Google Scholar 

  54. Dao HN, Ravikumar P, Likhitha P, Kiran RU, Watanobe Y, Paik I (2023) Finding stable periodic-frequent itemsets in big columnar databases. IEEE Access 11:12504–12524

    Article  Google Scholar 

  55. Fournier-Viger P, Yang P, Kiran RU, Ventura S, Luna JM (2021) Mining local periodic patterns in a discrete sequence. Inf Sci 544:519–548

    Article  MathSciNet  Google Scholar 

  56. Xun Y, Wang L, Yang H, Cai J (2022) Mining relevant partial periodic pattern of multi-source time series data. Inform Sci 615:638–656. https://doi.org/10.1016/j.ins.2022.10.049

    Article  Google Scholar 

  57. National Center for Atmospheric Research (2013) University Corporation for Atmospheric Research: Standardized Precipitation Index (SPI) for Global Land Surface. Research Data Archive at the National Center for Atmospheric Research, Computational and Information Systems Laboratory, Boulder CO (1949–2012)

  58. Times TJ (2019) Air pollution deaths in Japan. https://www.japantimes.co.jp/life/2019/05/11/environment/reading-air-tokyo-still-work-air-pollution. Accessed 12-December-2020

  59. Environment J (2020) SORAMAME. http://soramame.taiki.go.jp. Accessed 12-December-2020

  60. Kiran RU (2023) PAttern MIning-Databases (PAMI-Databases). https://u-aizu.ac.jp/~udayrage/datasets.html. Accessed 17-April-2023

  61. Kiran RU (2023) PAttern MIning-Python Kit (PAMI-PyKit). https://github.com/udayRage/pami pykit/tree/master/traditional/3P-ECLAT. Accessed 17-April-2023

Download references

Funding

This research was funded by JSPS Kakenhi 21K12034.

Author information

Authors and Affiliations

  1. The University of Aizu, Fukushima, Japan

    Uday Kiran Rage, Ravikumar Penugonda, Likhitha Palla & Yutaka Watanobe

  2. IIIT-Idupulapaya, Amaravati, RGUKT-AP, India

    Ravikumar Penugonda & Venus vikranth raj Bathala

  3. National Institute of Information and Communications Technology, Tokyo, Japan

    Uday Kiran Rage, Sadanori Ito & Koji Zettsu

  4. The University of Tokyo, Tokyo, Japan

    Uday Kiran Rage & Masashi Toyoda

  5. Department of Computer Science, Sri Balaji PG college, Affiliated to JNTU, Ananthapuram, 515001, Andhra Pradesh, India

    Veena Pamalla

Authors
  1. Veena Pamalla
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Uday Kiran Rage
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ravikumar Penugonda
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Likhitha Palla
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yutaka Watanobe
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Sadanori Ito
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Koji Zettsu
    View author publications

    You can also search for this author inPubMed Google Scholar

  8. Masashi Toyoda
    View author publications

    You can also search for this author inPubMed Google Scholar

  9. Venus vikranth raj Bathala
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Uday Kiran Rage.

Ethics declarations

Competing interests

The authors have no competing interests to declare relevant to the content of this article

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pamalla, V., Rage, U.K., Penugonda, R. et al. 3P-ECLAT: mining partial periodic patterns in columnar temporal databases. Appl Intell 54, 657–679 (2024). https://doi.org/10.1007/s10489-023-05172-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-05172-5

Keywords