arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Adding custom field to index

rated by 0 users
Answered (Verified) This post has 1 verified answer | 1 Reply | 2 Followers

Top 25 Contributor
23 Posts
victor posted on Tue, Jan 17 2017 3:36 PM

Is it possible to add any kind of custom field in index when crawling? To make it easier to match website domains to some entities in my system, when performing search queries later..

Thank you,

Vic

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Yes: http://arachnode.net/blogs/arachnode_net/archive/2010/04/08/adding-custom-fields-to-the-lucene-net-indexes.aspx

Also: https://workingcopy.arachnode.net/Plugins/CrawlActions/WorkingCopy_CustomManageLuceneDotNetIndexes.txt

OK,
Mike 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (2 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC