Skip to main content

POST/PUT Requests

The ScrapeOps Residential Proxy Aggregator supports sending POST and PUT requests in order to scrape forms or API endpoints.

When scraping POST and PUT endpoints, you need to figure out what data and headers are required to be sent with the request to give the correct response.

Scraping POST and PUT endpoints can be more tricky as it is normally easier for them to block requests as it is highly dependent on whether the data and headers you are sending contain all the correct information.

POST Requests​

Simply send the POST request as normal, specifying the Content-Type in the headers.

Here is an example of how to send POST requests to the Proxy endpoint using Python Requests:


import requests

proxies = {
  "http": "http://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181"
}
response = requests.post('http://httpbin.org/anything', proxies=proxies, json={'key': 'value'}, headers={'Content-Type': 'application/json'}, verify=False)

response.json()

PUT Requests​

Simply send the PUT request as normal, specifying the Content-Type in the headers.

Here is an example of how to send PUT requests to the Proxy endpoint using Python Requests:


import requests

proxies = {
  "http": "http://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181",
  "https": "https://scrapeops:YOUR_API_KEY@residential-proxy.scrapeops.io:8181"
}
response = requests.put('http://httpbin.org/anything', proxies=proxies, json={'key': 'value'}, headers={'Content-Type': 'application/json'}, verify=False)

response.json()