APIs for Load and Stress Testing

Unit tests tell you your API behaves correctly for one request at a time. They tell you nothing about what happens when a thousand requests arrive per second, or what breaks first when traffic keeps climbing past what you tested. Load testing answers "does this hold up at expected traffic," while stress testing answers "where does this fall over, and how does it fail." Both are things you should measure deliberately, in a controlled environment, rather than discovering for the first time in production.

Writing a load generator in Go

Go's goroutines and channels make it a genuinely good language for writing your own load generator when you need something more tailored than a generic CLI tool. A worker-pool pattern that fires a fixed number of concurrent requests and collects latency and status statistics:

package main

import (
	"io"
	"net/http"
	"sync"
	"time"
)

type result struct {
	status   int
	duration time.Duration
	err      error
}

func loadTest(url string, concurrency int, totalRequests int) []result {
	results := make(chan result, totalRequests)
	jobs := make(chan struct{}, totalRequests)
	for i := 0; i < totalRequests; i++ {
		jobs <- struct{}{}
	}
	close(jobs)

	var wg sync.WaitGroup
	client := &http.Client{Timeout: 5 * time.Second}

	for w := 0; w < concurrency; w++ {
		wg.Add(1)
		go func() {
			defer wg.Done()
			for range jobs {
				start := time.Now()
				resp, err := client.Get(url)
				elapsed := time.Since(start)
				if err != nil {
					results <- result{
						duration: elapsed, err: err,
					}
					continue
				}
				io.Copy(io.Discard, resp.Body)
				resp.Body.Close()
				results <- result{
					status:   resp.StatusCode,
					duration: elapsed,
				}
			}
		}()
	}

	wg.Wait()
	close(results)

	var all []result
	for r := range results {
		all = append(all, r)
	}
	return all
}

concurrency here caps the number of in-flight requests, which matters more than the total request count for understanding real-world behavior — a service that handles 1,000 sequential requests fine can still fall over at 50 concurrent ones if it has a small connection pool or a lock contention problem that only shows up under parallelism.

A custom load generator on the same machine as your test client shares its network stack

If your Go load generator and the service under test run on the same box, you're measuring loopback performance, not real network behavior, and you can exhaust the client machine's own ephemeral port range or CPU before the server even notices load. For anything beyond quick local sanity checks, run the generator from a separate machine (or several).

Reaching for a purpose-built tool: vegeta

For anything beyond a quick sanity check, tsenart/vegeta is a mature, widely used Go load testing tool available both as a CLI and as a library you can embed directly in your own test code:

import (
	"fmt"
	"time"

	vegeta "github.com/tsenart/vegeta/v12/lib"
)

func runVegetaAttack(targetURL string) {
	rate := vegeta.Rate{Freq: 100, Per: time.Second} // 100 requests/sec
	duration := 30 * time.Second
	targeter := vegeta.NewStaticTargeter(vegeta.Target{
		Method: "GET",
		URL:    targetURL,
	})

	attacker := vegeta.NewAttacker()
	var metrics vegeta.Metrics
	for res := range attacker.Attack(targeter, rate, duration, "load-test") {
		metrics.Add(res)
	}
	metrics.Close()

	fmt.Printf("success ratio: %.2f%%\n", metrics.Success*100)
	fmt.Printf("p95 latency: %s\n", metrics.Latencies.P95)
	fmt.Printf("throughput: %.2f req/s\n", metrics.Throughput)
}

vegeta.Rate is the key parameter that separates a proper load test from just "hammer it as fast as possible" — it fixes the request rate independent of how fast responses come back, which is what actually models real user traffic. Ramping Freq up across successive runs (100, then 500, then 1000) and watching where metrics.Success starts dropping or P95 latency spikes is how you find your API's actual capacity rather than guessing at it.

What to measure, not just how

A load test that only reports "it survived" tells you little. The metrics worth tracking on every run:

Latency percentiles, not averages — a p50 of 20ms with a p99 of 4 seconds means one in a hundred users is having a terrible time, and an average would hide that completely.
Error rate under load — does the error rate stay near zero as concurrency rises, or does it climb sharply past some threshold (a strong signal you've found a resource limit — a connection pool, a goroutine limit, a downstream dependency).
Resource usage on the server, gathered alongside the client-side metrics — CPU, memory, open file descriptors, database connection pool utilization. The load test tool tells you what the client experienced; your service's own metrics (see the observability chapters ahead) tell you why.

Stress testing means testing past the breaking point on purpose

Load testing validates you meet a target (say, 500 requests/second at acceptable latency). Stress testing deliberately exceeds that target to find where and how the system fails — does it degrade gracefully (slower responses, rejecting new requests with 429) or does it fail catastrophically (crash, cascade failures to dependent services)? Run stress tests against a staging environment that mirrors production's resource limits, never against production itself, since the entire point is to find the breaking point.

Load testing your own client code, not just the server

The same worker-pool pattern is useful in reverse: if your Go service calls a downstream API, load-test your own client against a mock server that introduces latency and occasional errors, to verify your timeouts, retries, and circuit breakers (from the API gateway chapter) actually behave correctly under load rather than only in the happy-path unit tests.

Making load tests part of the deploy pipeline

A load test run manually once a quarter tells you about the system as it existed a quarter ago. Running a smaller, fast version of the same test automatically against every release candidate — a two-minute run at a fixed moderate rate, checked against a latency and error-rate budget — catches performance regressions at the same point in the pipeline where you'd catch a failing unit test:

func TestPerformanceRegression(t *testing.T) {
	metrics := runVegetaAttackForTest(testServerURL, 50, 20*time.Second)
	if metrics.Latencies.P95 > 200*time.Millisecond {
		t.Fatalf("p95 latency regressed: got %s, budget 200ms",
			metrics.Latencies.P95)
	}
	if metrics.Success < 0.99 {
		t.Fatalf("success ratio regressed: got %.2f%%, budget 99%%",
			metrics.Success*100)
	}
}

This doesn't replace a full stress test against production-like infrastructure before a major release, but it catches the common case — a new database query added without an index, a synchronous call added where an asynchronous one used to be — before it ships, rather than after an on-call engineer notices latency creeping up in production metrics.

Frequently Asked Questions

Why does loadTest cap concurrency separately from totalRequests instead of just firing everything at once? Because concurrency, not total volume, is what actually exposes real-world problems — a service can happily churn through a thousand sequential requests but fall over at fifty concurrent ones if it has a small connection pool or a lock contention issue that only shows up under parallelism. Capping in-flight requests is what makes the test representative of actual traffic instead of a burst nothing production would ever see.

Why does the chapter warn against running the Go load generator on the same machine as the service under test? Because at that point you're measuring loopback performance and your own client machine's limits — CPU contention or exhausting the ephemeral port range — rather than anything resembling real network behavior. For anything beyond a quick local sanity check, run the generator from a separate machine so the numbers reflect what actual users experience.

Why does vegeta.Rate fix the request rate instead of just sending requests as fast as possible? Because "as fast as possible" models nothing real — actual user traffic arrives at some rate independent of how quickly your server happens to respond, and a naive hammer-it-as-fast-as-you-can approach conflates throughput with latency in a way that doesn't tell you where your real capacity limit is. Fixing Freq and ramping it across successive runs is what lets you find the point where success ratio drops or P95 latency spikes.

Why track latency percentiles instead of just the average response time? Because an average hides exactly the users having a bad time — a p50 of 20ms sitting next to a p99 of 4 seconds means one in a hundred requests is badly broken, and that fact disappears completely once you average it in with 99 fast ones. Percentiles (especially p95 and p99) are what actually surface a problem an average would smooth over.

Is a two-minute load test run in CI on every release candidate a replacement for a full stress test? No — it's a cheap regression check against a fixed latency and error-rate budget, good for catching the common case like a missing index or an accidentally synchronous call, but it's not exercising the system past its breaking point the way a proper stress test does. Keep both: the fast CI check on every release candidate, and a full stress test against production-like infrastructure before major releases.

A system's real capacity is not what it does on your laptop with one client — it's the point where latency and error rate both start climbing, and you only find that point by deliberately looking for it.