Skip to Content

problems webscraping with RCurl

I basically have two questions/ problems any help is highly appreciated. I am trying to webscrape the following page:

I tried the following:

webpage = getURL(url)
page = postForm(url, GasQuality="L",
                   .opts = curlOptions(
                   verbose = TRUE,
                   header = TRUE,
                   followLocation = TRUE,
                   useragent = "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv: Gecko/20101203 Firefox/3.6.13"
cat(page, file = "page.html") 

If you look at the HTML source, you can see that I omitted some of the name=value pairs. but including them doesnt make a difference either.