net/go.book
All Parts Marketing

Big Data, AI, and Streaming Networks

"An airport baggage system doesn't wait for every suitcase from every flight to arrive before starting to sort — it's a continuous flow of belts, scanners, and diverters, built to keep moving even when one flight dumps ten times the usual load. Streaming data systems are built the same way: designed for continuous flow and uneven bursts, not for a single batch that arrives all at once."

Batch vs. Stream, and Why the Network Design Differs

Traditional big-data processing (think a nightly job that reads a day's worth of logs) is a batch: bounded, known size, processed once. Streaming systems process data as it arrives, indefinitely, with no natural endpoint — log events, sensor readings, clickstreams, or tokens from an AI model generating a response one piece at a time. That difference changes the networking requirements:

  • Backpressure becomes essential: if consumers are slower than producers, something has to absorb or push back on the excess, or memory grows without bound.
  • Delivery semantics matter explicitly: at-most-once, at-least-once, or exactly-once, because a stream has no "reprocess the whole batch" fallback.
  • Fan-out is common: many consumers reading the same stream independently (analytics, alerting, and an ML feature pipeline might all read the same event stream).

Go's Role: Fast, Real Client Libraries

Go doesn't need to reinvent Kafka or NATS — it needs to talk to them well, and the ecosystem has mature, widely used, real client libraries for exactly that. Two worth knowing:

  • segmentio/kafka-go — a pure-Go Kafka client (no cgo, no dependency on librdkafka), which makes it easy to deploy and cross-compile.
  • nats-io/nats.go — the official Go client for NATS, a lightweight pub/sub messaging system with an at-most-once core and an optional persistent (JetStream) mode for at-least-once/exactly-once delivery.

A Kafka Consumer with kafka-go

// Illustrative use of a real package: github.com/segmentio/kafka-go
package main

import (
	"context"
	"log"

	"github.com/segmentio/kafka-go"
)

func consume(ctx context.Context, brokers []string, topic, groupID string) {
	reader := kafka.NewReader(kafka.ReaderConfig{
		Brokers: brokers,
		Topic:   topic,
		// consumer group: partitions are load-balanced across members
		GroupID: groupID,
	})
	defer reader.Close()

	for {
		msg, err := reader.ReadMessage(ctx)
		if err != nil {
			log.Println("read error:", err)
			return
		}
		log.Printf("partition=%d offset=%d value=%s",
			msg.Partition, msg.Offset, msg.Value)
	}
}

kafka.ReaderConfig.GroupID is what turns this into a proper consumer-group member: Kafka automatically assigns it a subset of the topic's partitions and rebalances if other consumers join or leave — the same fan-out and load-sharing behavior a hand-rolled system would have to build from scratch.

Backpressure with Buffered Channels and Rate Limiting

Whether you're consuming from Kafka or building an internal pipeline, the same principle applies: a fast producer feeding a slow consumer needs somewhere for the excess to go, deliberately, rather than accidentally. Go's buffered channels give you a bounded queue for free, and golang.org/x/time/rate (the real, standard extended-library rate limiter) lets you cap throughput explicitly:

package main

import (
	"context"
	"time"

	"golang.org/x/time/rate"
)

// process applies a hard cap on how fast events are handled, so a burst
// upstream doesn't overwhelm a downstream system (a database, an API, a GPU).
func process(ctx context.Context, events <-chan []byte, limiter *rate.Limiter) {
	for {
		select {
		case ev, ok := <-events:
			if !ok {
				return
			}
			if err := limiter.Wait(ctx); err != nil {
				return // context canceled while waiting for a token
			}
			handle(ev)
		case <-ctx.Done():
			return
		}
	}
}

func handle(ev []byte) {
	_ = ev // real processing goes here
}

func main() {
	// 500 events/sec, burst of 100
	limiter := rate.NewLimiter(rate.Limit(500), 100)
	// bounded buffer absorbs short bursts
	events := make(chan []byte, 1000)
	go process(context.Background(), events, limiter)
	time.Sleep(time.Second) // placeholder for real producer lifetime
}

The buffered channel absorbs short bursts without blocking the producer immediately; the rate limiter enforces a hard ceiling so a sustained burst degrades gracefully (producers block on a full channel, or you drop) instead of exhausting downstream resources.

Streaming AI Output Over the Network

Modern AI inference APIs — including this book's own subject matter, in a sense — commonly stream generated tokens back to the client as they're produced, rather than waiting for the full response. The standard library gives you two real, well-supported ways to do this over HTTP:

  • Server-Sent Events (SSE) — a simple, one-directional text stream over a normal HTTP response, well suited to token-by-token output.
  • gRPC server-streaming (covered in depth earlier in this section) — a good fit when the client is another service rather than a browser.

A Minimal SSE Endpoint

http.Flusher (standard library, part of net/http) is what makes this possible: it lets a handler push partial output to the client immediately instead of buffering the whole response.

package main

import (
	"fmt"
	"net/http"
	"time"
)

func streamTokens(w http.ResponseWriter, r *http.Request) {
	flusher, ok := w.(http.Flusher)
	if !ok {
		http.Error(w,
			"streaming unsupported", http.StatusInternalServerError)
		return
	}

	w.Header().Set("Content-Type", "text/event-stream")
	w.Header().Set("Cache-Control", "no-cache")
	w.Header().Set("Connection", "keep-alive")

	tokens := []string{"The", " quick", " brown", " fox", " jumps."}
	for _, tok := range tokens {
		// SSE message framing: "data: ...\n\n"
		fmt.Fprintf(w, "data: %s\n\n", tok)
		flusher.Flush() // push this chunk to the client now
		// simulate token generation latency
		time.Sleep(100 * time.Millisecond)
	}
}

func main() {
	http.HandleFunc("/stream", streamTokens)
	http.ListenAndServe(":8090", nil)
}

A browser client (or any HTTP client that reads the response body incrementally) sees each token as soon as flusher.Flush() is called, rather than waiting for the handler to return — the same mechanism that makes chat-style AI interfaces feel responsive instead of freezing until the whole answer is ready.

Frequently Asked Questions

Why does a stream need backpressure when a batch job never does? A batch job has a known, bounded size, so a slow consumer just takes longer to finish. A stream has no natural endpoint — if the producer keeps sending faster than the consumer can keep up, something has to absorb or push back on the excess, or memory grows without bound. That's exactly what the buffered channel and rate.Limiter in this chapter's process function are doing together: one absorbs short bursts, the other enforces a hard ceiling.

Do I need to run Kafka or NATS myself to follow this chapter? No — the chapter's point is that Go doesn't need to reimplement the broker, just talk to one well through mature client libraries like segmentio/kafka-go and nats-io/nats.go. Understanding kafka.ReaderConfig.GroupID and how consumer groups get partitions assigned matters more here than standing up a Kafka cluster yourself.

What's the difference between at-most-once, at-least-once, and exactly-once, and why does it matter for a stream? It's about what happens when something goes wrong mid-delivery: at-most-once can silently drop a message, at-least-once can redeliver the same message twice, and exactly-once guarantees neither. A stream can't just "reprocess the whole batch" the way a batch job can, so picking the wrong semantic for your use case (billing events versus a live dashboard, say) has real consequences that only show up under failure.

Why use Server-Sent Events instead of WebSockets for streaming AI tokens? Because token-by-token AI output is inherently one-directional — the client isn't sending anything back mid-response — and SSE rides on plain HTTP, so it works through the same proxies and load balancers as any other endpoint and reconnects automatically in browsers. WebSockets are bidirectional and lower-overhead per message, but that flexibility isn't needed here, which is why the DeepDive in this chapter calls SSE the simpler, more compatible choice for one-way streams.

What does http.Flusher actually do, and why does the handler check for it with a type assertion? flusher.Flush() is what pushes a partial chunk of the response to the client immediately instead of letting it sit in a buffer until the handler returns — that's the entire mechanism behind streamTokens feeling responsive token by token. The type assertion (w.(http.Flusher)) exists because not every http.ResponseWriter implementation supports flushing, so the handler needs to fail gracefully instead of panicking if it doesn't.

Key Takeaways

  • Streaming systems are designed for continuous, unbounded flow, which makes backpressure and delivery semantics first-class networking concerns.
  • segmentio/kafka-go and nats-io/nats.go are real, widely used pure-Go clients for production streaming platforms — Go doesn't need to reimplement the broker.
  • Buffered channels absorb short bursts; golang.org/x/time/rate enforces a hard throughput ceiling for sustained load.
  • http.Flusher (standard library) is the real mechanism behind Server-Sent Events, and it's exactly how token-by-token AI streaming responses work over plain HTTP.
  • SSE trades bidirectionality for simplicity and infrastructure compatibility — the right trade for one-way streams like AI output or log feeds.