arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

http://arachnode.net 1.2 release +lucene.net

rated by 0 users
This post has 5 Replies | 1 Follower

Top 10 Contributor
Posts 1,905
arachnode.net Yes [Y] Posted: Wed, Jun 3 2009 6:10 PM

Hi all!

I wanted to let everyone know that we have merged branch-1.2 into the trunk.

This means we are very, very close to an official release!!!

Thank you to everyone who has helped with feature requests, suggestions, bug reports and documentation!!!

  • The majority of Version 1.2 was dedicated to installation and configuration, and some light refactoring of Actions and Rules. 
  • Free-crawling is now prioritized by pre-configured Domain priorities, and search results now gain the benefit of Popularity/Priority, thereby further enhancing search result accuracy.
  • Finally, we added a WebService for searching and clustering functionality for the search page.

Thanks again, and please get latest from the trunk, test and let's get 1.2 out the door!!!  :)

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
Male
Posts 101
Kevin replied on Wed, Jun 3 2009 6:23 PM

Great work!!!

Top 50 Contributor
Male
Posts 11

Any DB structural changes or just code?

Top 10 Contributor
Posts 1,905

Nothing too crazy - but I did segment the configuration tables into their own schema.

I did add the Priorities table.

When we give the AOK I will make a master DB update.

Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
Posts 1,905

Just fixed a few bugs related to Discovery restrictions and clustering of the results for Search.aspx are fixed and committed.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
Posts 1,905

Just updated the site to reflect SQL2008 instead of SQL2005.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (6 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC