Using a Proxy with Python Requests

Author: Newtum

This guide shows how to route your HTTP requests through a proxy server using the Python Requests library. This is useful for accessing resources from a different IP address or bypassing network restrictions.

What is a Proxy?

A proxy server acts as an intermediary for requests from clients seeking resources from other servers. When you use a proxy, your request goes to the proxy first, and the proxy then forwards the request to the destination server.

How to Use a Proxy

You can specify a proxy by passing a dictionary to the `proxies` parameter. The keys should be the protocol (e.g., 'http', 'https') and the values should be the proxy URL.

import requests

# This is a public proxy, it may not always be available
proxy_url = 'http://167.71.5.83:80'

proxies = {
    'http': proxy_url,
    'https': proxy_url, # Use the same proxy for HTTP and HTTPS
}

try:
    # httpbin.org/ip will return the IP address of the request origin
    response = requests.get('https://httpbin.org/ip', proxies=proxies, timeout=5)
    print("Request was made from proxy IP:", response.json()['origin'])
except requests.exceptions.ProxyError as e:
    print(f"Failed to connect to proxy: {e}")
except requests.exceptions.RequestException as e:
    print(f"An error occurred: {e}")

Proxies with Authentication

If your proxy requires a username and password, you can include them in the proxy URL.

import requests

# Format: http://user:password@host:port
proxies = {
   'http': 'http://user:pass@10.10.1.10:3128/',
}

# This is a conceptual example and will not run without a real authenticated proxy.
# response = requests.get('https://httpbin.org/ip', proxies=proxies)
# print(response.status_code)

print("Proxy with authentication would be configured as shown.")