Wait Time

The ScrapeOps Proxy API Aggregator enables you to tell the headless browser to wait for a specific period of time before returning the HTML response.

This is useful if you are scraping dynamic websites that take a while to load all the data onto the page.

To enable the ScrapeOps Proxy's Wait Functionality, you just need to add the wait query parameter to your request, along with the number of milliseconds you want to wait.

For example, adding wait=3000 to the request will tell the headless browser to wait 3,000 milliseconds before returning the HTML response. Giving the website enough time to load the target data onto the page.

curl -k "https://proxy.scrapeops.io/v1/?api_key=YOUR_API_KEY&url=http://example.com/&wait=3000"

Automatically Enables Javascript Rendering

Adding a wait to the request will automatically enable a headless browser (equivalent to adding render_js=true to the request). Using the Headless Browser functionality will consume 10 API Credits.