arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does arachnode.net scale? | Download the latest release

arachnode.net + AN.Next | Commercial License | $499

  • 90-day unconditional return
  • Multi-server capabilities
  • AN.Next - a lightweight crawler not requiring MS-SQL
  • Access to future releases
  • Actively developed
  • AJAX/JavaScript/DOM integration/interaction
  • Asynchronous content processing
  • Automatic Proxy Server Rotation
  • Console application
  • Customization friendly
  • .doc/.pdf/.ppt/.xls indexing
  • Distributed crawling/multiple machines
  • Dynamic data administration
  • Extensible architecture
  • Forms application
  • Full lucene.net integration
  • Graphical user interface
  • GZip decompression
  • High-performance multi-threaded crawling engine
  • HtmlAgilityPack integration
  • Intelligent download throttling
  • International language support
  • JavaScript parsing/rendering
  • Link analysis and reporting
  • Local debugging proxy
  • Low memory/processor footprint
  • Multi-crawler, multi-database capabilities
  • Performance counters
  • Start/stop/pause/continue/save crawls
  • Storage direct to SQL 2005/2008 and/or to disk
  • Supports continuous crawling/indexing/searching
  • Unlimited forum/email support
  • Use from any .NET language
  • Web and webservice interface for search and direct data access
  • arachnode.cache - distributed disk-backed MRU cache
  • arachnode.tfidf - RAM-based tf/idf search engine



phone: (650) 76A-NODE (762-6633)

email: arachnode.net ( @ ) gmail.com

skype: arachnodedotnet

Address:
75-5737 Kuakini Hwy
Kailua-Kona, HI 96740

License Terms:

Copyright (c) 2004-2017 http://arachnode.net, arachnode.net, LLC

Permission is hereby granted, upon purchase, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, merge and modify copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

1.) The right to publish, distribute, sublicense, and/or sell is allowed but must retain the 'arachnode.net' designation in the source code.  You may not redistribute the arachnode.net source code in uncompiled format.  You are free to rename the binaries such that the 'arachnode.net' designation is not viewable to end users.

2.) The license fee is paid only once and secures rights to all current and future arachnode.net source code.

3.) The license information here supersedes all other license information posted elsewhere.

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC