arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Making Progress...but...

rated by 0 users
Answered (Verified) This post has 1 verified answer | 5 Replies | 3 Followers

Top 75 Contributor
6 Posts
drewehart posted on Mon, Jul 13 2009 3:52 PM

Ok, first thanks for the SQL 2008 instructions - they worked well.  I setup the database, execute the stored procedure, and build and run the applications.  I get the "console.exe" with the following error:

The following configuration settings are mising from database table 'cfg.Configuration'
ConoleOutputLogsDirectory
DownloadedFilesDirectory
DownloadedImagesDirectory
DownloadedWebPagesDirectory
Examine the 'Application' event log.

My knowledge is limited so any help would be appreciated.  Many thanks in advance,

Drew

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Check the cfg.CrawlActions table.

-Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 75 Contributor
6 Posts

OK, so I creaated directories and added their location to the configuration file, so that fixed it.  Now I get:

the following configuration setting is missing from database table 'cfg.CrawlActions' for CrawlAction 'Arachnode.Plugins.CrawlActions.ManageLuceneDotNetIndexes'
LuceneDotNetIndexDirectory

Examine table 'dbo.exceptions'

I am working on this error - if you have any solutions, please let me know.

Thanks

Drew

Top 75 Contributor
6 Posts

For now I just disabled the Lucene...

I believe it is in the documentation folder, but I am not sure...

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Check the cfg.CrawlActions table.

-Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
229 Posts

I am having the same issue. I disabled the LuceneDotNetIndex through the crawlactions table and it works ok, but now I dont create the index files which I will need in the future. what is the reason for such error? I already set the LuceneDotNetIndexDirectory in the configuration table, what else needs to be done?

Top 10 Contributor
1,905 Posts

Check the CrawlActions table.  Smile

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (6 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC