Friday, May 12, 2017

firefox - Multiple downloads from public source (web site), how to bypass "click on OK button" triggered by Javascript?

I'm trying to download multiple data sets from a public source, the European Union Transaction Log.



I'm using the "Allocation / Compliance" search tool (in the left blue pane).



From there I must select a country and hit "search". I then have access to various years (2010, 2011, 2012, etc...). I want to download the files for all countries (28) and all previous years (from 2005 to 2013), meaning 252 requests at least.



I tried to figure out how I could directly use the download URL. The final URL (in this instance, for Bulgaria in 2010), is the following.



From this URL, I can:





  • strip the "&installationAllowance=35266538" parameter

  • strip the "®istryCodeLookup=Bulgaria" parameter

  • change the "&periodYear=2010" parameter (to change the reference date)

  • change the "®istryCode=BG" parameter (to change the country).



My only problem so far is that each link requires a validation from the download page (a click on a "OK" button).



In the source code of the page, I see that the function triggered is the following:







The "OK" button code is the following:






There is also a block of JavaScript in the page source.




How could I mass download these files? (I was thinking of creating a bunch of links, replacing the years and countries with a RegEx, and then using the "DownThemAll" extension for Firefox, with the option "download from file").

No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...