Skip to main content

Residential Sticky Sessions

The ScrapeOps Residential Proxy Aggregator enables you to use the same IP address for multiple requests using the sticky_session query parameter.

For example, adding sticky_session=7 to a group of requests will tell the API to use the same proxy IP for all of these requests. The session number you set can be any number from 0 to 10000.

This is useful if you are scraping a website where you need to chain together multiple requests from the same IP address to get the data you need, such as:

  • Maintaining login sessions
  • Following pagination links
  • Multi-step form submissions
  • Shopping cart operations

If no sticky_session parameter has been set then the proxy will use a new IP address every time.

Example Usage​

cURL​

curl -k -x "http://scrapeops.sticky_session=7:YOUR_API_KEY@residential-proxy.scrapeops.io:8181" "https://httpbin.org/ip"

Python​

import requests

proxy = {
    'http': 'http://scrapeops.sticky_session=7:YOUR_API_KEY@residential-proxy.scrapeops.io:8181',
    'https': 'http://scrapeops.sticky_session=7:YOUR_API_KEY@residential-proxy.scrapeops.io:8181'
}

# These requests will use the same IP address
response1 = requests.get('https://httpbin.org/ip', proxies=proxy)
response2 = requests.get('https://httpbin.org/ip', proxies=proxy)

Custom Session Timeout​

By default, sticky sessions remain active for 10 minutes from the last request. You can customize this duration using the session_timeout parameter, which accepts a value in minutes.

For example, session_timeout=2 will set the session to expire after 2 minutes of inactivity:

cURL​

curl -k -x "http://scrapeops.sticky_session=7.session_timeout=2:YOUR_API_KEY@residential-proxy.scrapeops.io:8181" "https://httpbin.org/ip"

Python​

import requests

proxy = {
    'http': 'http://scrapeops.sticky_session=7.session_timeout=2:YOUR_API_KEY@residential-proxy.scrapeops.io:8181',
    'https': 'http://scrapeops.sticky_session=7.session_timeout=2:YOUR_API_KEY@residential-proxy.scrapeops.io:8181'
}

response1 = requests.get('https://httpbin.org/ip', proxies=proxy)
response2 = requests.get('https://httpbin.org/ip', proxies=proxy)
info

The session_timeout parameter is only used in combination with sticky_session. It has no effect on its own.

Important Notes​

Session Duration​

  • Sessions remain active for 10 minutes from the last request by default
  • Use session_timeout=X to customize the timeout (in minutes)
  • After the timeout period of inactivity, a new IP will be assigned
  • There is no limit to how long you can keep a session active with regular requests

Best Practices​

  • Use different session IDs for different scraping tasks
  • Keep session IDs consistent within the same scraping flow
  • Use a shorter session_timeout for quick scraping jobs to free up IPs sooner
  • Consider implementing retry logic for expired sessions
tip

For high-volume scraping, consider rotating through multiple session IDs to prevent overloading any single IP address.