Skip to main content

Response Formats

The ScrapeOps Proxy Aggregator offers two possible formats:

  1. Target Server Response (Default)
  2. JSON Response

Target Server Response (Default)

The default response from our Proxy API endpoint and Proxy Port is the response returned by the target URL you request.

This response could be in HTML, JSON, XML, etc. format depending on the response returned by the websites server.

Example response:


<html>
<head>
...
</head>
<body>
...
</body>
</html>

The response will contain the HTML, etc. response and any headers (Note: cookies aren't returned.)


JSON Response

If you add the parameter json_response=true to your request, then the proxy will return an extended JSON response with additional information about the request and response.

You can use this functionality when you would like to access additional response information such as cookies and XHR requests/responses.

The following are parameters returned.

KeyDescription
successfulBoolean value indicating if the request was successful or not. true if successful.
bodyThe HTML, JSON, XML, etc. response from the target website.
urlThe requested URL.
status_codeThe ScrapeOps status code.
sops_api_creditsThe number of ScrapeOps API credits consumed for the request.
content_typeThe content type of the websites response.
headersAny headers returned with the server response.
cookiesAny cookies returned with the server response.
xhrA array of the XHR requests/responses made by the headless browser when making the request. Only works when render_js=true is enabled.

The following is an example response:


{
"successful": true,
"url": "https://www.example.com/",
"content_type": "text/html;charset=UTF-8",
"sops_api_credits": 1,
"status_code": 200,
"headers": {
"Accept-Ch": "ect,rtt,downlink,device-memory,sec-ch-device-memory,viewport-width,sec-ch-viewport-width,dpr,sec-ch-dpr,sec-ch-ua-platform,sec-ch-ua-platform-version",
"Content-Language": "en-GB",
"Content-Security-Policy": "upgrade-insecure-requests;report-uri https://metrics.media-amazon.com/",
"Content-Type": "text/html;charset=UTF-8",
"X-Amz-Rid": "C347MNWAT7SE9XS4MJCN",
"X-Cache": "Miss from cloudfront",
"X-Content-Type-Options": "nosniff",
"X-Frame-Options": "SAMEORIGIN",
"X-Ua-Compatible": "IE=edge",
"X-Xss-Protection": "1;"
},
"xhr": null,
"cookies": [
{
"session-id": "258-3775233-4286404;"
}
],
"body": "<html><head>...</head><body>...</body></html>",
}