rscripts

Using Rselenium to overcome Rvest limitations

Rvest vs Rselenium Its very often the case that data crawling needs to accomodate cookie requests from the website and/or by pass login authorizations. While rvest is a great tool to process html text, accomodating cookies and login requests can be quite troublesome. The main problem comes with the internals of read_html() function which builds on curl type of requests. This connection is often stateless and may result to 301 http errors.