Crawling Data From Wikipedia 07.05.08
I need to crawl in a MySQL database some data from all countries in Wikipedia: en.wikipedia.org/wiki/List_of_countries
The data I need to crawl is this, taking as a sample Australia:
en.wikipedia.org/wiki/Australia
- Description of the country, as it’s on Wiki. The first text area we find.
- History
- Politics
- Capital
- Largest city (together with coordinates)
- Official Languages
- Governement: data here. and then Monarch, Governor, king and prime minister (when they exist)
- Area: total and water
- Population: estimate, census and density
- GDP (PPP) (total, per capita)
- GDP (nominal) (total, per capita)
- HDI
- Currency
- Time Zone
This database should be built in such a way so that we can import it later in our database. You’d probably exchange a message with my coder in order to ask him how he wants it exactly. but i guess that a simple mysql database with correct fields will suffice.
