An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop

Completely Open Source @ GitHub

Does scale? | Download the latest release


I can’t say enough good things about Arachnode! I was looking for a well-developed web scraper for a project I was working on. I first ended up using WebHarvy because of their nice looking UI. After using it for a day or two, I realized how long it would take me to get the data I needed. The software was very under-developed, slow, and often didn’t get all the data I needed it to. I returned the software and went to look for something else. I came across Arachnode and sent Mike an e-mail seeing if his software could do what I needed it to do. I was shocked when I woke up the next day to already see a long reply answering every question I had asked and even included Skype info to talk to him if I had any further questions. This software really does everything I need it to do and more. It uses multithread and works fast! The tech support is BAR NONE! Whenever I have a question for Mike or need him to do a little coding for me, he gets the work done fast, usually in less than 24 hours.  The cost of software is worth it in tech support alone. There is nothing worse than running into an issue and having to send an e-mail to the company to wait for a reply, which usually took 24+ hours to get. Nothing beats talking to someone in real time! Another great thing about this software is its all open source, so if you are up to the task of editing the code yourself to meet the requirement you need from the software, you can!  

-Brian V., Chicago, Illinois

"I notice that is an excellent and open-source project"
-wumengge on forums

"I've been playing with 1.2 for a few months now and I have to say it's GREAT!!!"
-hoska on forums

"great tool thanks"
-megetron on forums

"Hi. I think this is a great product, and I'm excited to start crawling with it"
-Sagie on forums

"you have been great help, Kudos!!"
-vishal on forums

"thank you for your great work!"
-lingyun2003 on forums

"Great stuff - Thank you!!"
-grahamsmith on forums

"thanks again for a great app!"
-hoska on forums

"BTW.  great job :)"
-anonymous on forums

An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, LLC