Mining Top-K Frequent Itemsets Through Progressive Sampling

Details Mining Top-K Frequent Itemsets Through Progressive Sampling Mining Top-K Frequent Itemsets Through Progressive Sampling

2010-06-27

arxiv.org/abs/1006.5235v1

Computer Science

Data Structures and Algorithms

16 pages, 2 figures, accepted for presentation at ECML PKDD 2010 and publication in the ECML PKDD 2010 special issue of the Da

Scientific paper

10.1007/s10618-010-0185-7

We study the use of sampling for efficiently mining the top-K frequent itemsets of cardinality at most w. To this purpose, we define an approximation to the top-K frequent itemsets to be a family of itemsets which includes (resp., excludes) all very frequent (resp., very infrequent) itemsets, together with an estimate of these itemsets' frequencies with a bounded error. Our first result is an upper bound on the sample size which guarantees that the top-K frequent itemsets mined from a random sample of that size approximate the actual top-K frequent itemsets, with probability larger than a specified value. We show that the upper bound is asymptotically tight when w is constant. Our main algorithmic contribution is a progressive sampling approach, combined with suitable stopping conditions, which on appropriate inputs is able to extract approximate top-K frequent itemsets from samples whose sizes are smaller than the general upper bound. In order to test the stopping conditions, this approach maintains the frequency of all itemsets encountered, which is practical only for small w. However, we show how this problem can be mitigated by using a variation of Bloom filters. A number of experiments conducted on both synthetic and real bench- mark datasets show that using samples substantially smaller than the original dataset (i.e., of size defined by the upper bound or reached through the progressive sampling approach) enable to approximate the actual top-K frequent itemsets with accuracy much higher than what analytically proved.

Affiliated with

Also associated with

No associations

Rating

Mining Top-K Frequent Itemsets Through Progressive Sampling does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Mining Top-K Frequent Itemsets Through Progressive Sampling, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Mining Top-K Frequent Itemsets Through Progressive Sampling will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-615650