arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE
Does arachnode.net scale? | VS2008/2010/2012 & SQL2008/2012 | Download the latest release

Max No Of Docs To Return Per Search Question

rated by 0 users
Answered (Verified) This post has 1 verified answer | 6 Replies | 2 Followers

Top 25 Contributor
19 Posts
ptrennum posted on Wed, Jul 13 2011 3:53 PM

I am having a strangeness regarding the MaximumNumberOfDocumentsToReturnPerSearch.

Does this value in the config Table need to be changed prior to a crawl?  What is happening is that I change the number in the DB from 200 to 1000 and the pager control updates showing 60 plus pages but any of the pages over 20 display no results.  Only 200 results are displayed no matter what the value is or how many result pages are returned?

Thanks

Answered (Verified) Verified Answer

Top 10 Contributor
1,692 Posts
Verified by arachnode.net

Strange... I don't have this line.

I will manually force the update of the locations in code that have this reference and hopefully (or not) it is an SVN hiccup.

This update will come with the fix from the storage requirements thread.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 10 Contributor
1,692 Posts

Hmm...

I just flushed a test index locally... I will put on a crawl and let you know.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 25 Contributor
19 Posts

Cool let me know what you get.  Any pages from 21 onwards are just blank for me....

Thanks!

Top 10 Contributor
1,692 Posts

Hmm...

I'm not seeing this.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 25 Contributor
19 Posts

Yeah I am still getting this problem.  I have updated the configuration table to allow 1000 results returned and any page from 21 onwards is blank.  Is there anywhere else I need to update this value other than the configuration table?

THanks!

Top 25 Contributor
19 Posts

Just found the other place, for some reason the value at the end was hard coded to 200 when I updated it to 1000 it worked fine

SearchResults<Document> searchResults = SearchManager.GetDocuments(Global.DefaultQueryParser, Global.CustomQueryParser, Global.IndexSearcher, query, (DiscoveryType)Enum.Parse(typeof(DiscoveryType), discoveryType), pageNumber, 10, shouldDocumentsBeClustered, null, 200);

thanks

Top 10 Contributor
1,692 Posts
Verified by arachnode.net

Strange... I don't have this line.

I will manually force the update of the locations in code that have this reference and hopefully (or not) it is an SVN hiccup.

This update will come with the fix from the storage requirements thread.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (7 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE

copyright 2004-2013, arachnode.net LLC