arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

Proxies

rated by 0 users
Answered (Verified) This post has 1 verified answer | 3 Replies | 2 Followers

Top 25 Contributor
23 Posts
JCrawl posted on Wed, Jul 15 2015 3:44 AM

Hello 

I am just getting started with proxies. My proxies require authentication, how do I set that up>. I entered the proxies as 12.12.12.12:80:UserName:Password however I am sure that is not the proper format.. can you please help me.

Thank you

 

Answered (Verified) Verified Answer

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Easy: webProxy.Credentials = new NetworkCredential("username", "password");

https://msdn.microsoft.com/en-us/library/system.net.webproxy.credentials(v=vs.110).aspx

Thanks!
Mike 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

All Replies

Top 10 Contributor
1,905 Posts
Verified by arachnode.net

Easy: webProxy.Credentials = new NetworkCredential("username", "password");

https://msdn.microsoft.com/en-us/library/system.net.webproxy.credentials(v=vs.110).aspx

Thanks!
Mike 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Top 25 Contributor
23 Posts
JCrawl replied on Sun, Aug 16 2015 2:41 PM

Thank you Mike for the Quick reply. Above you showed how to connect for the console project but now I am looking at the Renderer and I am not sure how to pass credentials to the proxy project. Can you shed some light on this. 

If you need access to private proxies, let me know.

Thank you 

 

JCrawl

Top 10 Contributor
1,905 Posts

The Renderers receive Proxies by default.

Thanks,
Mike 

For best service when you require assistance:

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

Skype: arachnodedotnet

Page 1 of 1 (4 items) | RSS
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC