Showing related tags and posts across the entire site.
-
Megetron, I have a situation where I build lists of particular web sites I want to crawl. I do it via a sql script that will submit the crawl requests, but it may give you some ideas. In the sql I just do something like this: declare @id bigint declare @url varchar(255) declare @depth int declare @restrictcrawl...
-
They correspond to the enum: namespace Arachnode.SiteCrawler.Value.Enums Flags ] { [ public enum UriClassificationType : byte { None = 0, Domain = 1, Extension = 2, FileExtension = 4, Host = 8, Scheme = 16 } } RestrictCrawlTo means that the Crawl won't crawl WebPages that aren't the same UriClassificationType...