arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2005/2008/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop
Mongo/Raven/MySQL/Hadoop Does arachnode.net scale? | VS2008/2010/2012 & SQL2008/2012 | Download the latest release

Start working question

rated by 0 users
Answered (Verified) This post has 1 verified answer | 3 Replies | 5 Followers

Top 50 Contributor
10 Posts
Henry.Wen posted on Sun, Aug 30 2009 8:49 PM

Please pardon me , I would like to ask you a few stupid questions.

Now ,i want to get images,txt and so on,from the website of  http://www.ebaiy.com.

what should i edit the table of cfg.Configuration, cfg.CrawlActions and cfg.CrawlRules.

Now,the table view follows:

cfg.Configuration:

cfg.CrawlRules:

 

 cfg.CrawlAction:

thanks Mike.

Answered (Verified) Verified Answer

Top 10 Contributor
1,750 Posts
Verified by arachnode.net

By default, arachnode.net is configured to download images.  :)

Just start crawling!

-Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 50 Contributor
7 Posts

Yes, if there is an example , it will be very helpful!

waiting.........................

 

Top 50 Contributor
10 Posts

I need the code help me to get images from websit right now..Because, I add the arachnode to my project.

Top 10 Contributor
1,750 Posts
Verified by arachnode.net

By default, arachnode.net is configured to download images.  :)

Just start crawling!

-Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (4 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE

copyright 2004-2014, arachnode.net LLC