Computer Science – Information Retrieval
Scientific paper
2011-02-03
MSPT 2006 - 6th International Workshop on MSPT Proceedings, Republic of Korea
Computer Science
Information Retrieval
6 pages, 6 figures
Scientific paper
Today World Wide Web (WWW) has become a huge ocean of information and it is growing in size everyday. Downloading even a fraction of this mammoth data is like sailing through a huge ocean and it is a challenging task indeed. In order to download a large portion of data from WWW, it has become absolutely essential to make the crawling process parallel. In this paper we offer the architecture of a dynamic parallel Web crawler, christened as "WEB-SAILOR," which presents a scalable approach based on Client-Server model to speed up the download process on behalf of a Web Search Engine in a distributed Domain-set specific environment. WEB-SAILOR removes the possibility of overlapping of downloaded documents by multiple crawlers without even incurring the cost of communication overhead among several parallel "client" crawling processes.
Ghosh Soumya
Kar Saheli
Kim Young-Chon
Mukherjee Sajal
Mukhopadhyay Debajyoti
No associations
LandOfFree
Architecture of A Scalable Dynamic Parallel WebCrawler with High Speed Downloadable Capability for a Web Search Engine does not yet have a rating. At this time, there are no reviews or comments for this scientific paper.
If you have personal experience with Architecture of A Scalable Dynamic Parallel WebCrawler with High Speed Downloadable Capability for a Web Search Engine, we encourage you to share that experience with our LandOfFree.com community. Your opinion is very important and Architecture of A Scalable Dynamic Parallel WebCrawler with High Speed Downloadable Capability for a Web Search Engine will most certainly appreciate the feedback.
Profile ID: LFWR-SCP-O-427799