I'm trying to download multiple data sets from a public source, the European Union Transaction Log.
I'm using the "Allocation / Compliance" search tool (in the left blue pane).
From there I must select a country and hit "search". I then have access to various years (2010, 2011, 2012, etc...). I want to download the files for all countries (28) and all previous years (from 2005 to 2013), meaning 252 requests at least.
I tried to figure out how I could directly use the download URL. The final URL (in this instance, for Bulgaria in 2010), is the following.
From this URL, I can:
- strip the "&installationAllowance=35266538" parameter
- strip the "®istryCodeLookup=Bulgaria" parameter
- change the "&periodYear=2010" parameter (to change the reference date)
- change the "®istryCode=BG" parameter (to change the country).
My only problem so far is that each link requires a validation from the download page (a click on a "OK" button).
In the source code of the page, I see that the function triggered is the following:
The "OK" button code is the following:
There is also a block of JavaScript in the page source.
How could I mass download these files? (I was thinking of creating a bunch of links, replacing the years and countries with a RegEx, and then using the "DownThemAll" extension for Firefox, with the option "download from file").
No comments:
Post a Comment