Getting Started with Squid Proxy Server in the Cloud

Squid is a high-performance caching and forwarding web proxy that offers powerful control over internet access and traffic flow within a network. Often used to cache content to speed up response times and reduce bandwidth usage, Squid is equally valuable as an API forward proxy for microservices and web applications. Acting as a middleman between client applications and external API endpoints, Squid allows fine-grained control, monitoring, and optimization of HTTP requests and responses. This guide will walk you through deploying and configuring Squid on AWS and Azure, enabling it to serve as an API forward proxy for your microservices architecture.

Benefits of Using a Forward Proxy for Microservices

Utilizing a forward proxy in microservices architectures brings several advantages:

1. Enhanced Security: Squid allows for detailed access control, enabling you to restrict outbound API calls to only the necessary endpoints. This helps secure your application against unintended external communications.
2. Traffic Monitoring: Squid’s logging capabilities provide valuable insights into API traffic patterns, request times, and error tracking. These logs can be used to troubleshoot issues and optimize API communication.
3. Reduced Latency and Bandwidth Usage: By caching responses from external APIs, Squid can reduce redundant calls and serve cached data, leading to lower response times and bandwidth costs.
4. Load Balancing: Squid can distribute outbound API requests, balancing loads across different endpoints, thereby optimizing network performance.
5. Auditing and Compliance: Detailed logs ensure you have records of all outbound requests, helping meet compliance requirements and monitor for suspicious activity.

Deploying Squid Server on AWS and Azure

To deploy a Squid Proxy server on AWS or Azure, choose your preferred platform below to launch a pre-configured Squid Proxy server instance. This simplifies setup, making it quick and straightforward. Our team has optimized these images to provide a seamless experience with Squid, streamlining the deployment process to save you time and reduce setup complexity.

After launching your instance, access it using SSH, following the access guidelines of your cloud provider. Typically, this involves an SSH key with a specific username or, in some cases, a password. Next, open the firewall for port 3128, Squid’s default port, but ensure it’s only accessible to internal hosts on your network. Exposing this port to the entire internet could unintentionally allow public access to your proxy.

For further customization, you can edit Squid’s configuration in /etc/squid/squid.conf. However, the default configuration should work out of the box. Once everything is set, configure your client applications to use this Squid Proxy server as a forward proxy to access the internet.

Testing Access to through the proxy

To verify that your proxy is working correctly, use curl from another machine within the same network or any device with access to the Squid server. In the example below, we test the proxy by making a curl request to the U.S. government’s weather service.

$ curl --proxy http://1.2.3.4:3128 https://api.weather.gov/points/39.7456,-97.0892

Replace 1.2.3.4 with the actual IP address of your Squid Proxy. To confirm the connection, you can monitor Squid’s access log /var/log/squid/access.log while making the request to see the log entry generated by the call to the weather service.

[08/Nov/2024:05:42:16 +0000] url="api.weather.gov:443" status=200 http_method=CONNECT request_protocol=HTTP/1.1 http_user_agent="curl/7.68.0" response_time=159 dns_lookup_time=36 src_ip=192.168.0.124 src_port=63114 dest_ip=23.196.44.171 dest_port=443 bytes=10220 bytes_in=985 bytes_out=9235 user_ident="-" user="-" http_referrer="-" vendor_action=TCP_TUNNEL dest_status=HIER_DIRECT

Here’s an explanation of some of the fields in the log entry above:

1. url: "api.weather.gov:443"
   • The URL or destination address that the proxy is connecting to, including the port number. Here, it indicates the HTTPS endpoint for api.weather.gov.
2. status: 200
   • The HTTP status code returned to the client. A 200 code indicates that the connection was successfully established.
3. http_method: CONNECT
   • The HTTP method used for the request. CONNECT is typically used to create a tunnel for HTTPS connections.
4. http_user_agent: "curl/7.68.0"
   • The user agent string sent by the client, which identifies the software and version initiating the request. Here, it’s curl version 7.68.0.
5. response_time: 159
   • The total time, in milliseconds, that Squid took to process the request and send a response back to the client.
6. dns_lookup_time: 36
   • The time, in milliseconds, taken by Squid to resolve the DNS for the destination address. This is typically measured from when the request is received to when the DNS lookup completes.
7. src_ip: 192.168.0.124
   • The IP address of the client making the request to the Squid proxy.
8. src_port: 63114
   • The source port on the client machine used to initiate the connection.
9. dest_ip: 23.196.44.171
   • The IP address of the destination server that Squid is connecting to. This is the resolved IP for api.weather.gov.
10. dest_port: 443
    • The destination port on the server, which is 443 for HTTPS traffic.
11. bytes: 10220
    • The total number of bytes transferred for this request, including both the request and response data.
12. bytes_in: 985
    • The number of bytes that Squid received from the client as part of the request.
13. bytes_out: 9235
    • The number of bytes that Squid sent to the client as part of the response.

These fields help administrators analyze details about each request and understand aspects like client source, destination, timing, and data size for troubleshooting or optimizing proxy performance.

Configuring Proxy for Some Common Client Systems

Configuring clients to route traffic through a Squid Proxy server can help manage, monitor, and secure network traffic. Here’s how to configure proxy settings for several common environments.

Linux Systems

For Linux, you can configure proxy settings both through the command line and via the GUI.

1. Command-Line Interface (CLI)

To set the proxy for command-line applications, add environment variables for HTTP and HTTPS proxies.

# Set HTTP and HTTPS proxies for the current session
export http_proxy="http://<proxy_ip>:3128"
export https_proxy="http://<proxy_ip>:3128"

# Verify the settings
curl -I http://www.google.com

To make these changes permanent, add the lines above to your .bashrc or .bash_profile.

2. Graphical User Interface (GUI)

For GUI applications, configure the proxy settings in your Linux desktop environment:

• GNOME: Go to Settings > Network > Network Proxy, select Manual, and enter your Squid server’s IP and port 3128.
• KDE Plasma: Go to System Settings > Network Settings > Proxy, choose Manual Configuration, and enter the proxy details.

After setting the proxy in the GUI, your web browser and other GUI-based applications should route traffic through the Squid server.

Windows Systems

You can configure proxy settings on Windows both through the GUI and via PowerShell.

1. Graphical User Interface (GUI)

1. Open Settings.
2. Go to Network & Internet > Proxy.
3. Under Manual proxy setup, toggle on Use a proxy server.
4. Enter the Squid server’s IP in the Address field and 3128 as the Port.
5. Save changes.

2. PowerShell

You can also configure proxy settings using PowerShell, which is useful for scripting or remote setups.

# Set the system-wide HTTP and HTTPS proxy settings
[System.Net.WebRequest]::DefaultWebProxy = New-Object System.Net.WebProxy("http://<proxy_ip>:3128")
[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

# To persist these settings across PowerShell sessions:
$env:http_proxy="http://<proxy_ip>:3128"
$env:https_proxy="http://<proxy_ip>:3128"

These settings apply to PowerShell and any applications using .NET’s WebRequest class for HTTP requests.

.NET Applications

To configure a .NET application to use a proxy, you can specify the proxy settings within the application’s configuration code.

Example in C#:

using System;
using System.Net;

class Program
{
    static void Main()
    {
        // Define proxy server
        WebProxy proxy = new WebProxy("http://<proxy_ip>:3128", true);

        // Assign the proxy to the default web request
        WebRequest.DefaultWebProxy = proxy;

        // Example of using an HttpClient with proxy settings
        var httpClientHandler = new HttpClientHandler { Proxy = proxy };
        var client = new HttpClient(httpClientHandler);

        // Make a request to verify
        var response = client.GetAsync("http://api.example.com").Result;
        Console.WriteLine(response.StatusCode);
    }
}

Replace <proxy_ip> with your Squid Proxy IP address. This approach ensures that all HTTP requests made by the application route through the specified proxy.

Docker Containers

To configure Docker containers to use a proxy, set environment variables for HTTP and HTTPS proxies in the Docker configuration.

1. Configuring Docker Daemon (Persistent Proxy)

Edit the Docker daemon configuration file:

sudo mkdir -p /etc/systemd/system/docker.service.d

# Create a file to store environment variables
sudo tee /etc/systemd/system/docker.service.d/proxy.conf <<EOF
[Service]
Environment="HTTP_PROXY=http://<proxy_ip>:3128"
Environment="HTTPS_PROXY=http://<proxy_ip>:3128"
EOF

# Reload and restart Docker service
sudo systemctl daemon-reload
sudo systemctl restart docker

2. Setting Proxy for Individual Containers

To set proxy variables for a specific container, you can specify them directly in the docker run command:

docker run -e HTTP_PROXY="http://<proxy_ip>:3128" -e HTTPS_PROXY="http://<proxy_ip>:3128" your_image

Alternatively, add proxy settings to a Dockerfile for a custom image:

# Set environment variables for HTTP and HTTPS proxies
ENV HTTP_PROXY="http://<proxy_ip>:3128"
ENV HTTPS_PROXY="http://<proxy_ip>:3128"

With these configurations, Docker containers will route their traffic through the Squid Proxy.

These settings allow various client environments to route their traffic securely through a Squid Proxy server, enhancing control and monitoring capabilities for network traffic.

Conclusion

Setting up Squid as an API forward proxy on AWS or Azure enhances control, security, and performance for microservices architectures. Squid’s configurable ACLs, caching rules, and robust logging make it an ideal choice for monitoring and controlling outbound API traffic. By following this guide, you’ll be equipped to deploy, configure, and troubleshoot Squid effectively, ensuring optimal API performance and security for your applications.