-
Just for others that read this post... when troubleshooting AN, start by looking in the database tables 'Exceptions' and 'DisallowedAbsoluteUris'. For your question: Take a look at cfg.AllowedDataTypes. This table controls what you are allowed to crawl. First step it to make sure that...
-
Another condition that can occur that throw a 406 is when you ask for file such as a GIF : But we actually get a JAVASCRIPT back, which be default, isn't allowed. The reason for the bait and switch could be tracking, or to save bandwidth for relatively unfamiliar crawlers, or due to a coding error...
-
This is due to certain WebServers not handling the HttpRequest header 'Accept-Types' properly. You really are going to make me fix all of my little things, huh? I'll take a look... Just for a little background on why you may get this error. Arachnode.net uses a very specific set of configurables...
-
You are very welcome. It can, but indexing is currently limted (from the lucene.net side, not the SQL FTI side) to text. Check the table AllowedDataTypes.