arachnode.net
An Open Source C# web crawler with Lucene.NET search using SQL Server 2008/2012/2014/2016/CE An Open Source C# web crawler with Lucene.NET search using MongoDB/RavenDB/Hadoop
Search the Live Index Does arachnode.net scale? | Download the latest release

Downloads

Name
Blue Tag
Users Comments
Add Comment
UbiCrawler: a scalable fully distributed Web crawler
Boldi, P., Codenotti, B., Santini, M., and Vigna, S. (2004a). UbiCrawler: a scalable fully distributed...
Synchronizing a database to improve freshness
Cho, J. and Garcia-Molina, H. (2000). Synchronizing a database to improve freshness. In Proceedings...
Modeling and managing content changes in text databases
Ipeirotis, P., Ntoulas, A., Cho, J., Gravano, L. (2005) Modeling and managing content changes in...
Mercator: A scalable, extensible Web crawler
Heydon, A. and Najork, M. (1999). Mercator: A scalable, extensible Web crawler. World Wide Web,...
Focused crawling using context graphs
Diligenti, M., Coetzee, F., Lawrence, S., Giles, C. L., and Gori, M. (2000). Focused crawling using...
Do your worst to make the best: Paradoxical effects in pagerank...
Boldi, P., Santini, M., and Vigna, S. (2004b). Do your worst to make the best: Paradoxical effects...
Design and implementation of a high performance distributed...
Shkapenyuk, V. and Suel, T. (2002). Design and implementation of a high performance distributed...
Design and implementation of a distributed crawler and filtering...
Zeinalipour-Yazti, D. and Dikaiakos, M. D. (2002). Design and implementation of a distributed crawler...
Crawling the Web
Pant, G., Srinivasan, P., Menczer, F. (2004). "Crawling the Web". Web Dynamics: Adapting...
Crawling a Country: Better Strategies than Breadth-First...
Baeza-Yates, R., Castillo, C., Marin, M. and Rodriguez, A. (2005). Crawling a Country: Better Strategies...
Breadth-first crawling yields high-quality pages
Marc Najork and Janet L. Wiener. Breadth-first crawling yields high-quality pages. In Proceedings...
Balancing volume, quality and freshness in web crawling
Baeza-Yates, R. and Castillo, C. (2002). Balancing volume, quality and freshness in web crawling...
Page 2 of 2 (35 items) < Previous 1 2
An Open Source C# web crawler with Lucene.NET search using SQL 2008/2012/CE

copyright 2004-2017, arachnode.net LLC