I keep getting this:Unable to read data from the transport connection: The connection was closed.
I made a research and it seems like it has something to do with a proxy.
I found nice link:
a guy named mayank.prajapati offers a solution. maybe ti can help.
Please let me know if you find this helpfull for the problem?
What is the code that fixes your problem?
Do you need to access a proxy server?
For best service when you require assistance:
At start you crawl fine, but after a while you get this:Unable to read data from the transport connection:
After googling a bit:"Turned out that if the sites failing used cookies – and therefore my request failed. So the correct syntax to fetch pages is instead"http://aspcode.net/unable-to-read-data-from-the-transport-connection
If you would add this to AN it will eorks fine:
CookieContainer CC = new CookieContainer();HttpWebRequest.Proxy = null;HttpWebRequest.UseDefaultCredentials = true;HttpWebRequest.CookieContainer = CC;
I checked my crawl that has been going for a month and I found 2 of the error messages you received. Wonder why you have so many more?
I made the change and checked it in.
do not make any changes...this is still under investigation.
seems like it still happens, and this code only delays it from happens....
Someone suggested to lower the MTU.
I crawl using a multi-server option. I wi try to crawl differently and let you know.
After changing the MTU(through registry) i still recieve error.
I found that this error repeat itself only when you crawls from another machine inside a network, where the SQLServer machine hosted in a different machine.
Foe example: PC1 host the sqlserver, and using dialup connection.
PC2 run the AN executable, and connects to internet through a shared internet connection of PC1.
This scenario will popups this error.
I don't have a solution for this. but the code above doess NOT solve, so check it out please.
At the meantime I must continue with thw crawk, so I will do the crawl through PC1 from the example above.
If you think you know what the issue, post code here, so I can test this,
Can you post all of the stack traces for the exceptions?