Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL

Details Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL

2002-12-11

arxiv.org/abs/cs/0212033v1

Proceedings of the Twelfth European Conference on Machine Learning, (2001), Freiburg, Germany, 491-502

Computer Science

Learning

12 pages

Scientific paper

This paper presents a simple unsupervised learning algorithm for recognizing synonyms, based on statistical data acquired by querying a Web search engine. The algorithm, called PMI-IR, uses Pointwise Mutual Information (PMI) and Information Retrieval (IR) to measure the similarity of pairs of words. PMI-IR is empirically evaluated using 80 synonym test questions from the Test of English as a Foreign Language (TOEFL) and 50 synonym test questions from a collection of tests for students of English as a Second Language (ESL). On both tests, the algorithm obtains a score of 74%. PMI-IR is contrasted with Latent Semantic Analysis (LSA), which achieves a score of 64% on the same 80 TOEFL questions. The paper discusses potential applications of the new unsupervised learning algorithm and some implications of the results for LSA and LSI (Latent Semantic Indexing).

Affiliated with

Also associated with

No associations

Rating

Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.

If you have personal experience with Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Mining the Web for Synonyms: PMI-IR versus LSA on TOEFL will most certainly appreciate the feedback.

Rate now

Comments { 0 }

Profile ID: LFWR-SCP-O-637972