arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Browse Site by Tags

Showing related tags and posts across the entire site.
  • RE: Crawl Web Pages, RSS, Facebook & Twitter and store cleaned content to database

    Hi, My colleague and I are developing a web application which requires frequently updated web content from the sites that we want to monitor, from facebook and twitter. The type of content we are interested in is news and trending topics. We have a licensed version of arachnode. For that purpose, we...
    Posted to Plugins by Tred.S on Fri, Jun 15 2012
  • Extracting blogpost / forummessages by keywords from a single location

    Hello, About a week a go a got my licenced version of AN. First of all it is a great (and fast) classlibrary. I have already crawled a few sites but i am stuck trying to achieve the following: I would like to crawl one or a few sites and extract blogposts / forummessages that contain certain keywords...
    Posted to Plugins by KIK!T on Thu, Sep 23 2010
  • Re: Questions about the Templater etc'

    Hi Mike, Thanks for your last answer and advices. I'm currently trying to understand how creating a CrawlRule can help me to filter result web pages while crawling. I've searched and read posts I've found about rules in the forom but still didn't figure completely how to define one. Can...
    Posted to Plugins by Anat on Thu, Nov 5 2009
Page 1 of 1 (20 items)
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC