arachnode.net
An open source .NET web crawler written in C# using SQL 2005/2008
IT Professionals & Windows Deployment Professionals: SmartDeploy Enterprise is the first hardware-independent imaging toolset that uses boot time driver-injection, simplifying deployment and easing distribution by reducing total image count. [LINK]

Question about the creation of folders for download files

rated by 0 users
Not Answered This post has 0 verified answers | 2 Replies | 2 Followers

Top 10 Contributor
Male
46 Posts
Massimo Ghidoni posted on 3 Feb 2010 9:24 AM

Hi,

I have to download files from an Url like

 http://comune.accadia.fg.it/sito/bandi/Urb_PIP/  (between Urb and PIP there is the caracter "_" )

Arachnode create the folders:

http -> comune -> accadia -> fg -> it -> sito -> bandi -> urb -> pip

Why AN generate 2 folder for "Urb_PIP" and not only one folder with that name?

And If in the URL there are some special caracter (such as "%" or "&") wath's happen in the folder creation?

Thank you in advance...

Massimo

All Replies

Top 10 Contributor
1,244 Posts

AN strips out all non-alphanumeric chars when creating the dirs.

If you want to change this you can do so here: http://workingcopy.arachnode.net/Functions/ExtractDirectory.cs

Thanks!
Mike

For best service when you require assistance:  Big Smile

  1. Check the DisallowedAbsoluteUris and Exceptions tables first.
  2. Cut and paste actual exceptions from the Exceptions table.
  3. Include screenshots.

An open source .NET web crawler written in C# using SQL 2005/2008.

Twitter: http://twitter.com/arachnode_net

arachnode.net provides custom crawling and contracting resources.  Please ask.

C# crawler, C# web crawler, C# site crawler

Top 10 Contributor
Male
46 Posts

Perfect!

Thanks!! Big Smile

Massimo

Page 1 of 1 (3 items) | RSS
An open source .NET web crawler written in C# using SQL 2005/2008

copyright 2004-2010, arachnode.net LLC

Powered by Community Server (Non-Commercial Edition), by Telligent Systems