Web Crawler 05.10.08
I need the following small crawler project:
1. login to a specific website (I will provide the user name and password and url)
2. this website contains a database of about 9000 objects.
3. go through the 9000 objects and write the results in one big excel file. (about 30-40 excel lines per each db object).
4. just before reached the excel 65000 lines limit, open a new excel, and continue writing in the new excel.
5. all data should be written to the A of the excel.
6. before and after each db object write a special tag in the excel, like (beginObj, endObj), so it will be easy to read the excel.
7. no need for GUI, just a simple progress bar.
8. Excel output can be a simple set of files: output1.xls, output2.xls in the current directory.
9. it has to be dot net.
10. demo app is required.
11. url = http://www.ivc-online.com/
12. the crawling is in this format: http://www.ivc-online.com/searchResultDetails.asp?Ptype=PFV&ObjectType=1&fObjectID=7986
with the fObjectID chaning for each object.
13. - you can create your own temporary account there for demo purposes, and I will provide the real user name and password, later on.
thanks
