arachnode.net
An open source .NET web crawler written in C# using SQL 2005/2008

Browse Forum Posts by Tags

Showing related tags and posts for the Forums application. See all tags in the site
  • Re: Web Shopping Pricing Bots solution...

    Address has been merged with AbsoluteUri. The AbsoluteUri CrawlRule references the following tables: If you wanted to Crawl msn.com, delete all rows in the tables above, insert msn.com into DisallowedDomains, and set IsDisallowed=true for msn.com and set the following setting in CrawlRules.config: negateIsDisallowedForAbsoluteUri...
    Posted to General Questions (Forum) by arachnode.net on 12 Apr 2009
Page 1 of 1 (1 items)
An open source .NET web crawler written in C# using SQL 2005/2008

copyright 2004-2010, arachnode.net LLC

Powered by Community Server (Non-Commercial Edition), by Telligent Systems