Hi,
I am new to arachnode.net. I have a question before I go in deep with crawler.
Is it possible to integrate it with any other web application ?
Here is the scenario.
I have one web application developed using asp.net & sql server 2008.
Now I want to crawl few websites through crawler and I want to store information like link, description, keywords etc in my application database.
I hope my question is understandable..
Please reply asap.
Thanks in advance.
Welcome! The quick answer is sure, you can integrate by having them use the same db. Use AN for what it's good at - crawling and storing the particular type of data you need. Then, review the db schema and pull out what you need for use in your app.
You can also go deeper. Write custom plug-ins that do whatever work you need during the crawl and store that data in any custom db schema you might need. Choose Lucene for your text indexing, or go with SQL Server's FTI. And with the full source, take it to wherever you need it!
Hope that helps :)
Thanks for your reply.
I will dig into it and will be back with list of questions.
Please be ready to answer them.... :)
Thanks.
No problem hope things go well for you. AN is a great product with lots of potential to take it wherever you need to.
