arachnode.net v2.0
An open source .NET web crawler written in C# using SQL 2005/2008

Question about the creation of folders for download files

rated by 0 users
Not Answered This post has 0 verified answers | 2 Replies | 2 Followers

Top 25 Contributor
9 Posts
Massimo Ghidoni posted on 02-03-2010 9:24 AM

Hi,

I have to download files from an Url like

 http://comune.accadia.fg.it/sito/bandi/Urb_PIP/  (between Urb and PIP there is the caracter "_" )

Arachnode create the folders:

http -> comune -> accadia -> fg -> it -> sito -> bandi -> urb -> pip

Why AN generate 2 folder for "Urb_PIP" and not only one folder with that name?

And If in the URL there are some special caracter (such as "%" or "&") wath's happen in the folder creation?

Thank you in advance...

Massimo

All Replies

Top 10 Contributor
Male
922 Posts

AN strips out all non-alphanumeric chars when creating the dirs.

If you want to change this you can do so here: http://workingcopy.arachnode.net/Functions/ExtractDirectory.cs

Thanks!
Mike

An open source .NET web crawler written in C# using SQL 2008.

Join the arachnode.net group on Facebook: http://www.facebook.com/groups.php?ref=sb#/group.php?gid=166721755872

Twitter: http://twitter.com/arachnode_net

arachnode.net is provides custom crawling and contracting resources.  Please ask.

http://bit.ly/TOFX4

Top 25 Contributor
9 Posts

Perfect!

Thanks!! Big Smile

Massimo

Page 1 of 1 (3 items) | RSS
An open source .NET web crawler written in C# using SQL 2005/2008

copyright 2009, arachnode.net LLC

Powered by Community Server (Non-Commercial Edition), by Telligent Systems