-
Milan Solanki: > What would be the best method of parsing html I do this daily. I use biterscripting for parsing our own web pages, and extracting all kinds of info from it in all kinds of formats. You can start with the sample script posted at http://www.biterscripting.com/SS_WebPageToText.html as...
-
Templater.cs is ALPHA code. 1.) HtmlAgilityPack is a memory hog, and should only be used when you absolutely need it. It is used in the templater code because I need XPATH support. 2.) ExtractText does a much, much better job of stripping out tags than the HtmlAgilityPack does, and it's faster as...
-
arachnode.net already contains support for the HtmlAgilityPack - however, the HtmlAgilityPack is a HUGE memory hog and has an extremely negative impact on crawling rate. If you can avoid it, don't use it. If you have to use it, change the configuration setting for 'ExtractWebPageMetaData'...