arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Encoding problems?

rated by 0 users
Answered (Verified) This post has 1 verified answer | 10 Replies | 2 Followers

Top 50 Contributor
Male
8 Posts
Vitaly posted on Sat, Apr 3 2010 5:42 AM

Hi, Mike! Do you plan to fix this? The same problem in database table 'Exceptions' (if the language .NET Framework is differs from English).

Thanks, Vitaly.

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Answered (Verified) arachnode.net replied on Sat, Apr 3 2010 1:46 PM
Verified by Vitaly

Cool bug find!  Thanks so much!  :)

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 10 Contributor
1,905 Posts

Yes.  Thanks for pointing it out.

(I am looking into it now...)

...OK, easy fixes... rebuilding the index now...

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 10 Contributor
1,905 Posts
Answered (Verified) arachnode.net replied on Sat, Apr 3 2010 1:46 PM
Verified by Vitaly

Cool bug find!  Thanks so much!  :)

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 50 Contributor
Male
8 Posts

Оk, thanks. Thank you for your prompt response. I hope that you make changes to the demo version.

Vitaly.

Top 10 Contributor
1,905 Posts

Thanks again for finding this bug.  There appears to be one other condition that creates the "..." for the SearchResult summary.  Looks like a problem with plurality and the Lucene.net summarizer.  More on this later.

I will likely roll it into the demo version once I finish another feature I am working on. 

(Why do you need it in the demo?  Did you figure out how to crack the demo?  Stick out tongue  Let me know if you have - I'll give you $50.  Big Smile)

 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 50 Contributor
Male
8 Posts

Hi, Mike!!! Only $ 50?.. Big Smile 

Of course I'm kidding Smile

In fact, I would like to see the demo version to see features of the system (if I'll like it, I'm happy to buy a license that interests me). I would be glad to cooperate with you. Smile

 

Thanks, Vitaly

Top 10 Contributor
1,905 Posts

Hehe - yeah - only $50 - I don't want to encourage bad behavior too much.  Smile

You have downloaded the demo version, right?  Does the demo version show you enough of what you need to see regarding the encoding bug that you found?

I am working on one Lucene.net bug related to summarization and one other new feature and after I wrap those two issues I will update the demo.

Thanks again for finding what you found - very, very helpful to me.  It is much appreciated.

Mike

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 50 Contributor
Male
8 Posts
Vitaly replied on Mon, Apr 5 2010 12:17 PM

Smile Thanks, Mike. Unfortunately or fortunately, I'm not a hacker Smile

Yes, I used a demo version. The bug was easy to find, as I would like to use your system and a Lucene.NET for searching in Russian. 

I suspect that the problem in the Lucene. I think it is appropriate to include the Lucene.NET project (I mean Lucene.NET source code) in the arachnode.net solution with your corrections.

I hope you consider my request. It is important to fully test the capabilities arachnode.net + Lucene.Net.

 

- Vitaly

Top 50 Contributor
Male
8 Posts
Vitaly replied on Fri, Apr 9 2010 12:05 PM

Hi, Mike!

I found another small bug Smile

Top 10 Contributor
1,905 Posts

Cool. 

The red box I am working on ATM - lucene.net (the highlighter) isn't picking up 'politic', that site has 'politics' in it.

The yellow is because politic is found in the tags, but not the InnerText.

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 50 Contributor
Male
8 Posts
Vitaly replied on Sat, Apr 10 2010 1:44 AM

Ok. Thanks, Mike

Page 1 of 1 (11 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC