arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Parameter is not valid - may be 304 response issue for images and maybe more

rated by 0 users
Answered (Verified) This post has 1 verified answer | 2 Replies | 2 Followers

Top 10 Contributor
Male
101 Posts
Kevin posted on Fri, Sep 4 2009 3:09 PM

In reviewing Parameter is not valid errors in Exceptions table, it appears that this may be due to 304 responses.

In testing, I get this error on images returned.  Using firebug to see what exactly comes back when these particular absoluteuri images are requested, I see that the first time I get a 200 and on a refresh I get a 304 not returned probably due to caching on the server end.

Debugging ImageManager, I think what is happening is that the image is actually not being returned, so line 96 is trying to do a FromStream call when the stream is a 0 byte stream (nothing returned so nothing there).

So need to look at how 304 respones are handled for images and maybe other things as well.

Kevin

 

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Not all WebServers handle the 'Accept-Type' HttpHeader properly.  Fixed in, 1.2 or greater.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 10 Contributor
1,905 Posts

I wonder if this is due to making a HEAD request first?

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Not all WebServers handle the 'Accept-Type' HttpHeader properly.  Fixed in, 1.2 or greater.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (3 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC