Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters

Details Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters

1998-09-29

arxiv.org/abs/cs/9809121v1

ACM Transactions on Information Systems, 14, 2 (April 1996), 138-174

Computer Science

Information Retrieval

Scientific paper

We consider information retrieval when the data, for instance multimedia, is coputationally expensive to fetch. Our approach uses "information filters" to considerably narrow the universe of possiblities before retrieval. We are especially interested in redundant information filters that save time over more general but more costly filters. Efficient retrieval requires that decision must be made about the necessity, order, and concurrent processing of proposed filters (an "execution plan"). We develop simple polynomial-time local criteria for optimal execution plans, and show that most forms of concurrency are suboptimal with information filters. Although the general problem of finding an optimal execution plan is likely exponential in the number of filters, we show experimentally that our local optimality criteria, used in a polynomial-time algorithm, nearly always find the global optimum with 15 filters or less, a sufficient number of filters for most applications. Our methods do not require special hardware and avoid the high processor idleness that is characteristic of massive parallelism solutions to this problem. We apply our ideas to an important application, information retrieval of cpationed data using natural-language understanding, a problem for which the natural-language processing can be the bottleneck if not implemented well.

Affiliated with

Also associated with

No associations

Rating

Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Using Local Optimality Criteria for Efficient Information Retrieval with Redundant Information Filters will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-692689