arachnode.net v2.0
An open source .NET web crawler written in C# using SQL 2005/2008
Forums » arachnode.net » General Questions » !!!How to restrict crawl to single domain? (Again)

!!!How to restrict crawl to single domain? (Again)

rated by 0 users
Not Answered This post has 0 verified answers | 1 Reply | 2 Followers

Top 25 Contributor
8 Posts
TileCheng posted on 02-17-2009 10:36 PM

I Crawl a List Page: Such As:  http://www.zhaojijiaju.com/news.asp

And It has Many sub pages : Such as: http://www.zhaojijiaju.com/view_news.asp?id=75

                                                                     http://www.zhaojijiaju.com/view_news.asp?id=25

                                                                    http://www.zhaojijiaju.com/view_news.asp?id=78

 

How I Can  Do  a CrawlRequest   In  The DataTable  "dbo.CrawlRequests"??

quickly,Thanks!!!!

 

All Replies

Top 10 Contributor
Male
920 Posts
Reply
arachnode.net replied on 05-26-2009 8:28 PM

I am not sure I understand the question.

I think you want to submit a depth of more than 1 to crawl deeper into a website's structure.

An open source .NET web crawler written in C# using SQL 2008.

Join the arachnode.net group on Facebook: http://www.facebook.com/groups.php?ref=sb#/group.php?gid=166721755872

Twitter: http://twitter.com/arachnode_net

arachnode.net is provides custom crawling and contracting resources.  Please ask.

http://bit.ly/TOFX4

Page 1 of 1 (2 items) | RSS
An open source .NET web crawler written in C# using SQL 2005/2008

copyright 2009, arachnode.net LLC

Powered by Community Server (Non-Commercial Edition), by Telligent Systems