arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop
Search the Live Index Does arachnode.net scale? | Download the latest release

Crawl Freezes In the Middle of Crawl

rated by 0 users
Answered (Verified) This post has 0 verified answers | 17 Replies | 1 Follower

Top 10 Contributor
35 Posts
flash posted on Sat, Jan 7 2012 11:48 PM

Hi,

I have been running the crawl for a year now on a 4 sites.

Recently the crawls seems to be freezing in the middle of the crawl on all sites.
This issue first started occurring on one of the sites and now all are "Infected".
I have tried running the Crawler on the console application for testing, but the issue is not resolved.

I get no exceptions nor errors, The window just stops updating (no new lines of data appear) while the console window is still alive (I can sent "Enter" manually).

Thank you in advance,
Runny

 

Page 1 of 2 (16 items) 1 2 Next > | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC