文章基本信息

标题：A Study on Distributed Crawling-Based Overhead Optimization
本地全文：下载
作者：Han-Gil Kim ; Jong-Won Lee ; Tae-Hak Ban 等
期刊名称：International Journal of Software Engineering and Its Applications
印刷版ISSN：1738-9984
出版年度：2015
卷号：9
期号：3
页码：175-182
DOI：10.14257/ijseia.2015.9.3.17
出版社：SERSC
摘要：Network technology has been in the process of development with IT. Data on the network is increasing rapidly and there are also various data searching methods. One of these methods is a crawler. It finds data that users want and collects it. However, as data on the network increases exponentially, potential problems of the crawler are constantly emerging. This paper investigates that crawlers can operate in one system, decentralizing a crawler's main functions. By using/implementing a proposed system, the efficiency is compared, analyzed and verified with the existing problems.
关键词：Crawler; Distribute; Overhead; Optimization