Skip to main content
Springer Nature Link
Log in

A novel hash-based approach for mining frequent itemsets over data streams requiring less memory space

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

In recent times, data are generated as a form of continuous data streams in many applications. Since handling data streams is necessary and discovering knowledge behind data streams can often yield substantial benefits, mining over data streams has become one of the most important issues. Many approaches for mining frequent itemsets over data streams have been proposed. These approaches often consist of two procedures including continuously maintaining synopses for data streams and finding frequent itemsets from the synopses. However, most of the approaches assume that the synopses of data streams can be saved in memory and ignore the fact that the information of the non-frequent itemsets kept in the synopses may cause memory utilization to be significantly degraded. In this paper, we consider compressing the information of all the itemsets into a structure with a fixed size using a hash-based technique. This hash-based approach skillfully summarizes the information of the whole data stream by using a hash table, provides a novel technique to estimate the support counts of the non-frequent itemsets, and keeps only the frequent itemsets for speeding up the mining process. Therefore, the goal of optimizing memory space utilization can be achieved. The correctness guarantee, error analysis, and parameter setting of this approach are presented and a series of experiments is performed to show the effectiveness and the efficiency of this approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Bocca JB, Jarke M, Zaniolo C (eds) Proceedings of the 20th international conference on very large databases (VLDB1994), Santiago, Chile, pp 487–499

  • Arlitt M, Williamson C (1996) Web server workload characterization: the search for invariants. In: Proceedings performance evaluation review, vol 24, No. 1, pp 126–137

  • Calders T, Dexters N, Goethals B (2006) Mining frequent items in a stream using flexible windows. In: Cama J, Klinkenberg R, Aguilar J (eds) Proceedings of ECML/PKDD 2006 workshop on knowledge discovery from data streams (IWKDDS), Berlin, Germany, pp 87–96

  • Calders T, Dexters N, Goethals B (2007) Mining frequent itemsets in a stream. In: Proceedings of the seventh IEEE international conference on data mining (ICDM’07), Omaha, USA, pp 83–92

  • Chang JH, Lee WS (2003) Finding recent frequent itemsets adaptively over online data streams. In: Getoor L, Senator TE, Domingos P, Faloutsos C (eds) Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining (KDD2003), Washington, DC, USA, pp 487–492

  • Charikar M, Chen K, Farach-Colton M (2002) Finding frequent items in data streams. In: Widmayer P, Ruis FT, Bueno RM, Hennessy M, Eidenbenz S, Conejo R (eds) Proceedings of the 29th international colloquium on automata, languages and programming (ICALP’02), Málaga, Spain, pp 693–703

  • Cheng J, Ke Y, Ng W (2006) Maintaining frequent itemsets over high-speed data streams. In: Ng WK, Kitsuregawa M, Li J, Chang K (eds) Proceedings of the 10th Pacific-Asia conference on knowledge discovery and data mining (PAKDD 2006), Singapore, pp 462–467

  • Chi Y, Wang H, Yu PS, Muntz RR (2004) Moment: maintaining closed frequent itemsets over a stream sliding window. In: Proceedings of the fourth IEEE international conference on data mining (ICDM’04), Brighton, UK, pp 59–66

  • Cormode G, Muthukrishnan S (2003) What’s hot and what’s not: tracking most frequent items dynamically. In: Proceedings of the 22nd ACM SIGMOD-SIGACT-SIGART symposium on principles of database systems (PODS2003), San Diego, CA, pp 296–306

  • Dang XH, Ng WK, Ong KL (2008) Online mining of frequent sets in data streams with error guarantee. Knowl Inf Syst 16(2): 245–258

    Article  Google Scholar 

  • Demaine E, Lopez-Ortiz A, Munro JI (2002) Frequency estimation of Internet packet streams with limited space. In: Möhring RH, Raman R (eds) Proceedings of the 10th European symposium on algorithms (ESA2002), Rome, Italy, pp 348–360

  • Fischer MJ, Salzberg SL (1982) Finding a majority among N votes: solution to problem 81-5. J Algorithm 3(4): 362–380

    Article  Google Scholar 

  • Giannella C, Han J, Pei J, Yan X, Yu PS (2004) Mining frequent patterns in data streams at multiple time granularities. In: Kargupta H, Joshi A, Sivakumar K, Yesha Y (eds) Data mining next generation challenges and future directions. AAAI Press, Menlo Park, CA, pp 191–212

  • Golab L, DeHaan D, Demaine ED, López-Ortiz A, Munro JI (2003) Identifying frequent items in sliding windows over on-line packet streams. In: Proceedings of the first ACM SIGCOMM Internet measurement conference (IMC’03), Florida, USA, pp 173–178

  • Han J, Pei J, Yin Y (2000) Mining frequent patterns without candidate generation. In: Chen W, Naughton JF, Bernstein PA (eds) Proceedings of the 2000 ACM SIGMOD international conference on management of data (SIGMOD’00), Dallas, TX, USA, pp 1–12

  • Jiang N, Gruenwald L (2006) CFI-stream: mining closed frequent itemsets in data streams. In: Eliassi-Rad T, Ungar LH, Craven M, Gunopulos D (eds) Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’06), Philadelphia, USA, pp 592–597

  • Jin R, Agrawal G (2005) An algorithm for in-core frequent itemset mining on streaming data. In: Proceedings of the fifth IEEE international conference on data mining (ICDM’05), Houston, TX, USA, pp 210–217

  • Jin C, Qian W, Sha C, Yu JX, Zhou A (2003) Dynamically maintaining frequent items over a data stream. In: Proceedings of the 12th ACM international conference on information and knowledge management (CIKM’03), New Orleans, LA, USA, pp 287–294

  • Karp RM, Papadimitriou CH, Shenker S (2003) A simple algorithm for finding frequent elements in streams and bags. ACM Trans Database Syst 28(1): 51–55

    Article  Google Scholar 

  • Lee D, Lee W (2005) Finding maximal frequent itemsets over online data streams adaptively. In: Proceedings of the fifth IEEE international conference on data mining (ICDM’05), Houston, TX, USA, pp 266–273

  • Lee LK, Ting HF (2006) A simpler and more efficient deterministic scheme for finding frequent items over sliding windows. In: Vansummeren S (ed) Proceedings of the 25th ACM SIGMOD-SIGACT-SIGART symposium on principles of database systems (PODS’06), Chicago, USA, pp 290–297

  • Leung CKS, Khan Q (2006) DSTree: a tree structure for the mining of frequent sets from data streams. In: Proceedings of the sixth IEEE international conference on data mining (ICDM’06), Hong Kong, China, pp 928–932

  • Li HF, Lee SY, Shan MK (2004) An efficient algorithm for mining frequent itemsets over the entire history of data streams. The first international workshop on knowledge discovery in data streams, in conjunction with ECML/PKDD 2004, Pisa, Italy

  • Li HF, Ho CC, Kuo FF, Lee SY (2006) A new algorithm for maintaining closed frequent itemsets in data streams by incremental updates. In: Proceedings of IEEE international workshop on mining evolving and streaming data (ICDM workshops 2006), Hong Kong, China, pp 672–676

  • Lin CH, Chiu DY, Wu YH, Chen ALP (2005) Mining frequent itemsets from data streams with a time-sensitive sliding window. 2005 SIAM international conference on data mining (SDM’05), Newport Beach, CA

  • Manku GS, Motwani R (2002) Approximate frequency counts over data streams. In: Proceedings of the 28th international conference on very large databases (VLDB2002), Hong Kong, China, pp 346–357

  • Mozafari B, Thakkar H, Zaniolo C (2008) Verifying and mining frequent patterns from large windows over data streams. In: Proceedings of IEEE 24th international conference on data engineering (ICDE’08), Cancún, México, pp 179–188

  • Wang SY, Hao XL, Xu HX, Hu YF (2007a) Finding frequent items in data streams using ESBF. In: Proceedings of the 2007 international workshop on high performance data mining and application (HPDMA 2007), in conjunction with PAKDD 2007, Nanjing, China, pp 244–255

  • Wang SY, Xu HX, Hu YF (2007b) Finding frequent items in sliding windows over data streams using EBF. In: Proceedings of the eighth ACIS international conference on software engineering, artificial intelligence, networking, and parallel/distributed computing (SNPD 2007), Qingdao, China, pp 682–687

  • Yu JX, Chong Z, Lu H, Zhou A (2004) False positive or false negative: mining frequent itemsets from high speed transactional data streams. In: Nascimento MA, Özsu MT, Kossmann D, Miller RJ, Blakeley JA, Schiefer KB (eds) Proceedings of the 30th international conference on very large databases (VLDB2004), Toronto, Canada, pp 204–215

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, National Tsing Hua University, Hsinchu, Taiwan, ROC

    En Tzu Wang

  2. Department of Computer Science, National Chengchi University, Taipei, Taiwan, ROC

    Arbee L. P. Chen

Authors
  1. En Tzu Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Arbee L. P. Chen
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Arbee L. P. Chen.

Additional information

Responsible editor: M.J. Zaki.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wang, E.T., Chen, A.L.P. A novel hash-based approach for mining frequent itemsets over data streams requiring less memory space. Data Min Knowl Disc 19, 132–172 (2009). https://doi.org/10.1007/s10618-009-0129-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-009-0129-2

Keywords