arachnode.net
An open source .NET web crawler written in C# using SQL 2005/2008
IT Professionals & Windows Deployment Professionals: SmartDeploy Enterprise is the first hardware-independent imaging toolset that uses boot time driver-injection, simplifying deployment and easing distribution by reducing total image count. [LINK]

What do you use arachnode.net for?

rated by 0 users
This post has 6 Replies | 5 Followers

Top 10 Contributor
Posts 1,244
arachnode.net Posted: 15 Jun 2009 8:42 AM

Tell us what you currently use arachnode.net for, or intend to use arachnode.net for.

Thanks!
-Mike

For best service when you require assistance:  Big Smile

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

An open source .NET web crawler written in C# using SQL 2005/2008.

Twitter: http://twitter.com/arachnode_net

arachnode.net provides custom crawling and contracting resources.  Please ask.

C# crawler, C# web crawler, C# site crawler

Top 50 Contributor
Posts 4

I want to use arachnode.net + lucene.net to write a Small vertical search engine in c#.Arachnode.net is perfect and complex but I still know little about it ,can you tell me is arachnode.net a distributed crawler?

  Where can I get the latest arachnode.net 1.2 + lucene.net ?

 

 

Top 10 Contributor
Posts 1,244

Thanks for the compliment.  Big Smile

AN is currently designed for a single machine and there are a few others little things we're going to add in the way of user-requested plug-ins before we make the move to distributed crawling.

You can get the latest code from: https://arachnodenet.svn.sourceforge.net/svnroot/arachnodenet

For best service when you require assistance:  Big Smile

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

An open source .NET web crawler written in C# using SQL 2005/2008.

Twitter: http://twitter.com/arachnode_net

arachnode.net provides custom crawling and contracting resources.  Please ask.

C# crawler, C# web crawler, C# site crawler

Top 100 Contributor
Posts 1

I am using arachnode.net to learn how web crawlers and search engines works. According to the license of Arachnode.net, I can use it in a project in the company I work, can´t I?

Top 10 Contributor
Posts 1,244

Yes.  The license permits you to do whatever you want with the code.  We have no claims to whatever you create with it.  arachnode.net is bascially as free as it gets.  :)

For best service when you require assistance:  Big Smile

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

An open source .NET web crawler written in C# using SQL 2005/2008.

Twitter: http://twitter.com/arachnode_net

arachnode.net provides custom crawling and contracting resources.  Please ask.

C# crawler, C# web crawler, C# site crawler

Top 50 Contributor
Posts 5

Hello

I've been following your project with great interest. I hope to use it to try and aggregate different RFP opportunities in the public sector. I'm finding that the challenge is in trying to parse out the relevant pieces of data and put them in a database that would make sense to an application. I'm sure that's the struggle in doing anything like this.

Does anyone have any specific experience with something like that? Maybe someone who has done something similar with job postings or something?

I look forward to getting things working so I can test this out.

Top 25 Contributor
Posts 16

i thank the same as you , if you have good news or demo site , pls tell me  3QQ

Page 1 of 1 (7 items) | RSS
An open source .NET web crawler written in C# using SQL 2005/2008

copyright 2004-2010, arachnode.net LLC

Powered by Community Server (Non-Commercial Edition), by Telligent Systems