The ScrapeOps Proxy API enables you to tell the headless browser to wait for a specific period of time before returning the HTML response.
This is useful if you are scraping dynamic websites that take a while to load all the data onto the page.
To enable the ScrapeOps Proxy's Wait Functionality, you just need to add the
wait query parameter to your request, along with the number of milliseconds you want to wait.
For example, adding
wait=3000 to the request will tell the headless browser to wait 3,000 milliseconds before returning the HTML response. Giving the website enough time to load the target data onto the page.
curl -k "https://proxy.scrapeops.io/v1/?api_key=YOUR_API_KEY&url=http://example.com/&wait=3000"
wait to the request will automatically enable a headless browser (equivalent to adding
render_js=true to the request). Using the Headless Browser functionality will consume 10 API Credits.