Exploring Go’s HTTP RoundTripper - A few use cases

I’ve had quite a few instances where I needed to modify the default configurations of an HTTP client. One very regular instance is when I need to debug what goes on in a request. I typically would modify the client such that every request is routed through the specified proxy server — I should probably write a separate article on this.

Recently, I’ve been exploring a more intricate configuration of Go’s HTTP client and its use cases thorough the http.RoundTripper interface.

Understanding HTTP RoundTripper in Go

Not necessarily sure how widely discussed it is but the http.RoundTripper interface is quite an interesting portion of Go's HTTP client and standard library at large.

What is an HTTP RoundTripper?

Fundamentally, an HTTP RoundTripper is a Go interface that specifies the requests and answers that an HTTP client may send and receive. Think of it as a middleware but for the client.

Understanding the Basics — The RoundTrip Method

The http.RoundTripper interface is deceptively simple:

type RoundTripper interface {
    RoundTrip(*http.Request) (*http.Response, error)
}

Again, deceptively – this interface is the backbone of HTTP communication in Go. The RoundTripper's job is to execute a single HTTP transaction. It acts as a middleman between an application and the server, handling all the finer details of the HTTP protocol.

The heart of the RoundTripper interface is the RoundTrip method. This function accepts a pointer to a http.Request as input and outputs an error along with a pointer to a http.Response.

The RoundTrip technique usually accomplishes the following:

It takes a prepared request.
Sends it to the server using appropriate protocols.
Reads the server's response.
Constructs an http.Response object from the server's reply.
Returns this response (or an error if something went wrong).

Why does this matter?

Understanding RoundTripper is crucial for several reasons:

Customization: It lets you modify the behaviour of the HTTP client to suit your requirements.
Middleware: You can create middleware-like functionality by chaining RoundTrippers.
Testing: Custom RoundTrippers make it easy to mock HTTP responses for testing.
Performance: By optimizing a RoundTripper, you can improve the performance of your HTTP requests.

The good thing about this design is its flexibility. Implementing a custom RoundTripper allows us customize how requests are sent and how responses are handled.

The http.Transport: The Default RoundTripper

While understanding the RoundTripper interface is nice, it’s equally important to understand its default implementation and how it works: http.Transport. This struct does most of the heavy-lifting behind Go’s HTTP client, handling the smaller details of HTTP communication.

What is http.Transport?

http.Transport implements the RoundTripper interface. It's designed to be used as the Transport field in http.Client, but it can also be used standalone for fine-grained control over HTTP requests.

Here's a basic example of how to create and use an http.Transport:

transport := &http.Transport{}
client := &http.Client{Transport: transport}

resp, err := client.Get("https://example.com")

Creating Custom RoundTrippers

Why Customize?

Customizing the RoundTripper allows you to add functionality to every HTTP request your application makes. Common use cases include:

Use case 1 - Logging

In this example, I’ll create a LoggingRoundTripper that logs the details of every request and response.

type LoggingRoundTripper struct {
    rt http.RoundTripper
}

func (lrt *LoggingRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) {
// Log the request
    log.Printf("Request: %s %s", req.Method, req.URL)

// Perform the HTTP request
    resp, err := lrt.rt.RoundTrip(req)

// Log the response or error
    if err == nil {
        log.Printf("Response: %s", resp.Status)
    } else {
        log.Printf("Error: %v", err)
    }

    return resp, err
}

This LoggingRoundTripper provides a simple way to log all HTTP requests and responses.

Structure: The LoggingRoundTripper struct embeds another RoundTripper. This allows it to wrap any existing RoundTripper, including the default one.
RoundTrip Method: This is where the magic happens. It implements the RoundTripper interface:
- It first logs the incoming request's method and URL.
- Then it calls the embedded RoundTripper's RoundTrip method to perform the actual HTTP request.
- Finally, it logs either the response status or any error that occurred.
Wrapping: By returning the response and error from the embedded RoundTripper, it maintains the expected behavior while adding logging functionality.

Usage:

func main() {
    client := &http.Client{
        Transport: &LoggingRoundTripper{
            rt: http.DefaultTransport,
        },
    }

    resp, err := client.Get("https://example.com")
    if err != nil {
        log.Fatal(err)
    }
    defer resp.Body.Close()

// Process the response...
}

Use Case 2: Token Refresh Mechanism

Why Refresh Tokens?

In many authentication systems, access tokens have a limited lifespan for security reasons. When these tokens expire, rather than requiring users to log in again, a refresh token mechanism allows the application to obtain a new access token seamlessly. This approach improves user experience and maintains security.

Extending the AuthTransport for Tokn Refresh

To implement a token refresh mechanism, we'll create an AuthTransport that extends the basic http.RoundTripper. This transport will automatically refresh the token when it receives a 401 Unauthorized response.

// TokenRefresher defines the interface for token refresh operations
type TokenRefresher interface {
    RefreshToken() (string, error)
}

// AuthTransport implements http.RoundTripper with token refresh capabilities
type AuthTransport struct {
    Token          string// Current access token
    Transport      http.RoundTripper// Underlying RoundTripper to make HTTP requests
    TokenRefresher TokenRefresher// Interface to refresh the token
    mutex          sync.Mutex// Mutex to ensure thread-safe token updates
}

// RoundTrip implements the http.RoundTripper interface
func (at *AuthTransport) RoundTrip(req *http.Request) (*http.Response, error) {
// Clone the request to avoid modifying the original
    newReq := req.Clone(req.Context())

// Add the current token to the request header
    newReq.Header.Set("Authorization", "Bearer "+at.Token)

// Use default transport if none provided
    if at.Transport == nil {
        at.Transport = http.DefaultTransport
    }

// Perform the HTTP request
    resp, err := at.Transport.RoundTrip(newReq)

// Check for unauthorized response (indicating expired token)
    if err == nil && resp.StatusCode == http.StatusUnauthorized {
// Attempt to refresh the token
        newToken, err := at.TokenRefresher.RefreshToken()
        if err != nil {
            return resp, err
        }

// Update the token in a thread-safe manner
        at.mutex.Lock()
        at.Token = newToken
        at.mutex.Unlock()

// Retry the request with the new token
        newReq.Header.Set("Authorization", "Bearer "+newToken)
        return at.Transport.RoundTrip(newReq)
    }

    return resp, err
}

This implementation provides a solution for handling token refreshes in a way that's transparent to the rest of an application.

The TokenRefresher interface allows us to define custom token refresh logic based on your authentication system.
The AuthTransport struct encapsulates all necessary components: the current token, the underlying transport, the token refresher, and a mutex for thread safety.
The RoundTrip method:
- Adds the current token to each request.
- Performs the request using the underlying transport.
- If it receives a 401 Unauthorized response, it:
  - Refreshes the token
  - Updates the stored token safely
  - Retries the request with the new token

To use this AuthTransport, we would need to implement the TokenRefresher interface according to your authentication system, then set up your HTTP client like this:

refresher := YourTokenRefresher{}// Your implementation of TokenRefresher
authTransport := &AuthTransport{
    Token:          initialToken,
    TokenRefresher: refresher,
}
client := &http.Client{Transport: authTransport}

This setup ensures that all requests made through this client will automatically handle token refreshing when needed, providing a seamless authentication experience in your application.

Recap

Throughout this writeup, I've covered a few key points:

Started with the basics, understanding what a RoundTripper is and its role in Go's HTTP client operations.
Explored custom RoundTripper implementations, starting with a simple logging example and progressing to a more advanced token refresh mechanism.

Further Learning

Understanding the HTTP RoundTripper opens up possibilities for optimizing and extending a Go applications' network communication. Here are a few resources I've used and you might find helpful:

Official Go http package documentation: Go HTTP Package
The Go Blog: "HTTP/2 Server Push": HTTP/2 Server Push
Steve Francia's talk on common Go mistakes: YouTube - 7 common mistakes in Go and when to avoid them
Go Time Podcast Episode 140: "Networking, HTTP Routers, WebSockets": Go Time - Networking