Hi there,
First of all - please excuse me for asking dumb questions but for some reason I cannot get this working like the rest of the community.
I am deploying arachnode for a college project demonstrating the usage of web crawlers alongside databases. I have installed it completely and the program will compile however - I am not sure how to configure it.
Am I required to configure using the SQL tables only or do I edit the configuration project > ApplicationSettings.cs?
Ok time for another question: When the application compiles it will run a web application. When I try to execute this application I get a directory view of [arachnode.net root]\web - should this be displaying the demonstration page (just like on this website?)
I thank you for your help - I hope I am not wasting too much of your time :)
Regards,Patrick(New Zealand)
(also is the descriptions missing on the documentation page?)
You'll need to edit cfg.Configuration, cfg.CrawlActions and/or cfg.CrawlRules.
If you see the directory view then you haven't selected a startup page (Search.aspx) - StartupPage is a solution option, which I don't check in because your breakpoints are not my breakpoints (and other options) and vice versa... :)
Which documentation page are you referring to?
MAIN INSTALLATION DOCUMENTATION LINK: http://arachnode.net/Content/InstallationInstructions.aspx
Mike
An open source .NET web crawler written in C# using SQL 2005/2008.
Join the arachnode.net group on Facebook: http://www.facebook.com/groups.php?ref=sb#/group.php?gid=166721755872
Twitter: http://twitter.com/arachnode_net
arachnode.net provides custom crawling and contracting resources. Please ask.
http://bit.ly/TOFX4
C# crawler, C# web crawler, C# site crawler
Hay Mike,
Thanks for the swift reply - I had a look in those table and until I can get this going I am happy enough with the default settings. But I am still having a hard time getting this going (I feel silly)
SO I un commented your code within the test crawl block in Program.CS but I still cannot get this to run - it looks exactly like it does in the screenshot. I was under the impression it would insert a row into crawlrequests (since I cant really manually do this)
Another silly Question - How do I go about checking in the startup page - IF I can get this going It would be great to have an example search engine running for my project.
As for the documentation, http://arachnode.net:8081/ the descriptions are missing (or is this meant to be like this?)
Regards,
Patrick
I bet your SQL connection is hanging... check the first call to GetVersion in Crawler.cs...
/// <summary> /// Checks for exceptions. /// </summary> private void CheckForExceptions() { ArachnodeDataSet.VersionDataTable versionDataTable = _arachnodeDAO.GetVersion();
Here's how to set the Start Page:
Documentation, yes - they are supposed to be blank - supposed to remind me to add them. AN really needs a small book as every so often I'll forget how something works - yes, there is a lot arachnode.net can do...
You can insert rows into the table directly - I HIGHLY recommend using the API tho - ArachnodeDAO.InsertCrawlRequest(...);
Also, I'm fixing some "less-than-minor" things that user:megetron found - so, please get the latest from the trunk and check back often. :)