Research and Application on Optimization of Multi-thread Download Technology for Enhanced Search Engine

DOI

10.2991/iccia.2012.365How to use a DOI?

Keywords

search engine, multi-threaded download, P2SP, MapReduce, protocol adaptation

Abstract

Multi-threaded file download as the key technology of content acquisition system for search engine, determines the efficiency and timeliness of content acquisition. In this paper, we do the research on optimization technologies which include multithreaded download based on P2SP, task scheduling based on MapReduce and download based on the protocol adaptation, designed to improve enhanced search engine efficiency. At last the result shows that the optimization method is successful for content acquisition.

Copyright

© 2013, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Yajuan Sun
AU  - Hong Lin
AU  - Baohui Wang
PY  - 2014/05
DA  - 2014/05
TI  - Research and Application on Optimization of Multi-thread Download Technology for Enhanced Search Engine
BT  - Proceedings of the 2012 2nd International Conference on Computer and Information Application (ICCIA 2012)
PB  - Atlantis Press
SP  - 1473
EP  - 1476
SN  - 1951-6851
UR  - https://doi.org/10.2991/iccia.2012.365
DO  - 10.2991/iccia.2012.365
ID  - Sun2014/05
ER  -