Just going through the Demo version of AN, I would like to know how this tool indexing the data? Is the tool do the Data Mining of the crawled data? I would like to discuss in detail in this regard.
My issue is that I've to crawl specific data from a authenticated web page and do the analytics on that. Will this tool can help me out?
Indexing is performed either through SQL Full Text indexing and/or Lucene.NET.
There are a large number of functions and sections of helper code for use in data mining.
Here: Look at 'UserDefinedFunctions' -> http://arachnode.net/search/SearchResults.aspx?q=UserDefinedFunctions
Yes, AN can crawl authenticated web sites.
For best service when you require assistance: