Statistics – Machine Learning
Scientific paper
2011-08-03
Statistics
Machine Learning
Scientific paper
Minwise hashing is the standard technique in the context of search and databases for efficiently estimating set (e.g., high-dimensional 0/1 vector) similarities. Recently, b-bit minwise hashing was proposed which significantly improves upon the original minwise hashing in practice by storing only the lowest b bits of each hashed value, as opposed to using 64 bits. b-bit hashing is particularly effective in applications which mainly concern sets of high similarities (e.g., the resemblance >0.5). However, there are other important applications in which not just pairs of high similarities matter. For example, many learning algorithms require all pairwise similarities and it is expected that only a small fraction of the pairs are similar. Furthermore, many applications care more about containment (e.g., how much one object is contained by another object) than the resemblance. In this paper, we show that the estimators for minwise hashing and b-bit minwise hashing used in the current practice can be systematically improved and the improvements are most significant for set pairs of low resemblance and high containment.
Konig Christian
Li Ping
No associations
LandOfFree
Accurate Estimators for Improving Minwise Hashing and b-Bit Minwise Hashing does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Accurate Estimators for Improving Minwise Hashing and b-Bit Minwise Hashing, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Accurate Estimators for Improving Minwise Hashing and b-Bit Minwise Hashing will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-664517