arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Search

  • Control arachnode configurations

    What is the proper way to control arachnode configuration? We have different configuration tables in AN database. Still when I do crawler.Start() it tries to load disallowed domains, disallowed uris and some other config stuff from xml files on the disk. I'm writing my console app for crawling, so I don't have any xmls with config. Could you
    Posted to General Questions (Forum) by victor on Wed, Jan 11 2017
  • Re: web service integration

    Awesome Thank you very much for such a quick answer! I'm trying to test services endpoints and having some troubles with Session. I click 'invoke' button on webservice test page And then get this: System.NullReferenceException: Object reference not set to an instance of an object. at Arachnode.Web.Value.AbstractClasses.AWebService.get_ArachnodeDAO
    Posted to General Questions (Forum) by victor on Mon, Jan 9 2017
  • web service integration

    I'm an AN newbie, so here to ask for some basic directions. I'd like to use AN to create a search feature in my existing main solution. I have a list of items with website as a property, so I want to search these items, based on keywords, that their websites contain. So I want to have a kind of web worker, crawling websites from my database
    Posted to General Questions (Forum) by victor on Sun, Jan 8 2017
Page 3 of 3 (23 items) < Previous 1 2 3 | More Search Options
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC