arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE
Sign in
|
Join
|
Help
Home
Features
Compare
Download
License
Live Demonstration
Support
Contact
API Documentation
Blogs
Creating Plugins
Forums
Frequently Asked Questions
Installation Instructions
Working Copy
All Tags
»
404
»
crawlrequests
(
RSS
)
Does arachnode.net scale?
| VS2008/2010/2012 & SQL2008/2012 |
Download the latest release
Browse Forum Posts by Tags
Showing related tags and posts for the
General Questions
forum. See
all tags in the site
crawling specific web sites
404 is returned because trailing slash is not used
When I crawl this site: http://www.jenkinskling.com the following response is returned: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <title>R a z o r B a l l</title> <meta http-equiv="Content-Type" content="text...
Posted to
General Questions
(Forum)
by
canuckbbp
on Thu, Nov 10 2011
Page 1 of 1 (1 items)
An Open Source C# web crawler with Lucene.NET search using SQL 2005/2008/CE
copyright 2004-2013, arachnode.net LLC