arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop
Search the Live Index Does arachnode.net scale? | Download the latest release

April 2015 - arachnode.net : blog

  • Performance Tuning

    From time to time I take a look at areas of AN which might be hastened. OVERVIEW TIPS: Disable ConsoleLogging: ApplicationSettings.EnableConsoleOutput = true; Disable Intellitrace. More threads does not necessarily mean better performance. Limit SQL's...
  • AJAX/Dynamic Content

    The question of "How do I interact with Dynamic/AJAX content" comes up enough such that it warrants a dedicated blog post. The Renderers execute out-of-process as either instances of AxWebBrowser or as the headless (no drawing of graphical content) variant...
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC