Wait For Element
The ScrapeOps Proxy API enables you to tell the headless browser to wait for a specific element to appear on the page before returning the HTML response.
This is useful if you are scraping dynamic websites that take a while to load all the data onto the page.
To enable the ScrapeOps Proxy's Wait Functionality, you just need to add the
wait_for query parameter to your request, along with a CSS selector of the page element you want to wait for.
For example, adding
wait_for=.loading-done to the request will tell the headless browser to wait until the element with the CSS selector
.loading-done before returning the HTML response. Giving the website enough time to load the target data onto the page.
curl -k "https://proxy.scrapeops.io/v1/?api_key=YOUR_API_KEY&url=http://example.com/&wait_for=.loading-done"
wait_for to the request will automatically enable a headless browser (equivalent to adding
render_js=true to the request). Using the Headless Browser functionality will consume 10 API Credits.