网络爬虫作为一个自动爬取网页的程序，为搜索引擎从网站上下载网页，是搜索引擎的重要组成部分。主题爬虫是专为查询某一主题或者某一领域应运而生的页面抓取工具。不同于通用搜索引擎，主题搜索引擎具有针对性，输入主题关键字，搜到的网页都是主题相关度极高的网页。-Web crawler as a Web page crawling procedures for the search engine　the website to download web pages, is an important part of the search engine. Theme crawler is designed to query a theme or a domain that came into being in the page crawler. Different　the general search engine, theme search engine has targeted, enter the theme of the keyword, the page is found to be highly relevant topics of the page.