文章基本信息

标题：Design of a Parallel Migrating Web Crawler
本地全文：下载
作者：Abhinna Agarwal ; Durgesh Singh ; Anubhav Kedia 等
期刊名称：International Journal of Advanced Research In Computer Science and Software Engineering
印刷版ISSN：2277-6451
电子版ISSN：2277-128X
出版年度：2012
卷号：2
期号：4
出版社：S.S. Mishra
摘要：As the size of the Web grows exponentially, crawling the web using parallel crawlers poses certain drawbacks such as generation of large amount of redundant data and wastage of network bandwidth due to transmission of such useless data. Thus to overcome these inherent bottlenecks with traditional crawling techniques we have proposed the design of a parallel migrating web crawler. We first present detailed requirements followed by the architecture of a crawler
关键词：web crawler; parallel; migration; web database