Construction of Online English Corpus Based on Web Crawler Technology

<table class="table-group" id="tab1"><tr><td><table class="table"><tr><td class="thead-hr" colspan="2"><hr/></td></tr><tr class="thead"><td class="align_left">Language</td><td class="align_center">Web crawler framework</td></tr><tr><td class="thead-hr" colspan="2"><hr/></td></tr><tr><td class="align_left">JAVA</td><td class="align_center">Apache Nutch, webmagic, Heritriz3, WebCollector, craw1er4j, Spiderman, SeimiCrawler, jsoup-Gecco, and htmlunit</td></tr><tr><td class="align_left">Python</td><td class="align_center">Scrapy, pyspider, Newspaper, and Crawley</td></tr><tr><td class="align_left">PHP</td><td class="align_center">cola, Portia, python selenium, QueryList, phpspider, and PHPCrawl</td></tr><tr><td class="align_left">Go</td><td class="align_center">Beanbun, php selenium</td></tr><tr><td class="align_left">C#</td><td class="align_center">SmartSpider, Abot, xet, AngleSharp, HtmlAgilityPack, and CsQueryopen-source-sear ch-engine.Cobweb</td></tr><tr><td class="align_left">C/C++</td><td class="align_center">upton, Spidr, and Larbin</td></tr><tr><td class="align_left">Ruby</td><td class="align_center">wombat</td></tr><tr><td class="align_left">node.js</td><td class="align_center">node-crawler</td></tr><tr class="table-tr"><td colspan="2"><hr class="tbody-hr"/></td></tr></table></td></tr></table>

<div>Web crawler framework corresponding to different programming languages.</div>

Wireless Communications and Mobile Computing

Construction of Online English Corpus Based on Web Crawler Technology

Table 1