io.Reader, io.Writer, and Streaming Data

Every networked program in Go moves bytes. A TCP connection reads bytes from the network and writes bytes back. An HTTP handler reads a request body and writes a response body. A file server reads a file from disk and writes it to a socket. All of these are the same operation — reading from something, writing to something — and Go models that operation with two interfaces so central that almost every package in the standard library depends on them.

If you understand io.Reader and io.Writer, you understand how data flows through every Go program — network or otherwise.

The io.Reader Interface

type Reader interface {
    Read(p []byte) (n int, err error)
}

One method. Read fills the caller-provided buffer p with up to len(p) bytes, returns how many bytes it actually wrote into p, and optionally an error. The caller owns the buffer — Read does not allocate. ([]byte is a byte slice; slices, arrays, and maps get their own chapter next.)

Key contract details:

• Read may return n > 0 and a non-nil error (including io.EOF) in the same call. Code that checks the error first and discards n when err != nil silently drops the last chunk of data.
• Read is allowed to return fewer than len(p) bytes even when more data is available — a short read is not an error.
• When no more data exists, Read returns 0, io.EOF. A well-behaved reader returns 0, io.EOF on every subsequent call.

Common io.Reader Implementations

Every one of these satisfies io.Reader:

// All three of these satisfy io.Reader — your code that calls Read
// doesn't care which one is underneath.
var r io.Reader
r = os.Stdin                  // terminal input
r = strings.NewReader("abc")  // a string
r = bytes.NewBuffer([]byte{1, 2, 3}) // a buffer

The io.Writer Interface

type Writer interface {
    Write(p []byte) (n int, err error)
}

The mirror image. Write takes a caller-owned byte slice and writes len(p) bytes to the underlying sink. It returns how many bytes were written — the caller must check that n == len(p), because a short write with a non-nil error means not everything was written.

Common io.Writer Implementations

io.ReadAll, io.Copy, and Friends

The io package provides convenience functions built on Reader and Writer:

// Read everything from a Reader into memory
data, err := io.ReadAll(r)

// Copy everything from a Reader to a Writer, in 32KB chunks,
// without ever holding the full content in memory.
written, err := io.Copy(w, r)

// Copy exactly N bytes
written, err := io.CopyN(w, r, 1024)

// Read at least min bytes into buf, or fill it entirely
n, err := io.ReadAtLeast(r, buf, min)

// Read exactly len(buf) bytes
n, err := io.ReadFull(r, buf)

io.Copy is the workhorse. When a Go HTTP proxy forwards a response, a file server sends a file to a client, or a TCP relay pipes data between two connections, the implementation is io.Copy:

func handleConn(client net.Conn, upstream string) {
    defer client.Close()
    server, err := net.Dial("tcp", upstream)
    if err != nil {
        return
    }
    defer server.Close()

    // Pipe data in both directions concurrently
    go io.Copy(server, client)
    io.Copy(client, server)
}

io.Copy uses a 32KB internal buffer and never grows beyond that — it streams data in chunks instead of loading everything into memory first. This is the difference between a file server that handles any file size and one that crashes on a 2GB download.

Composing Readers

Go's io package provides composable readers — wrappers that take an existing Reader and add behavior without the wrapped type needing to know:

// LimitReader stops after N bytes — protects against unbounded input.
r := io.LimitReader(conn, 1<<20) // 1 MB max

// TeeReader duplicates reads: everything read from r is also written to w.
// Useful for logging or inspecting data as it passes through.
r := io.TeeReader(conn, os.Stdout)

// MultiReader concatenates multiple readers into one stream.
r := io.MultiReader(
    strings.NewReader("prefix\n"),
    file,
    strings.NewReader("\nsuffix"),
)

These compose because they all implement io.Reader. A TeeReader wrapping a LimitReader wrapping a net.Conn is still an io.Reader — the caller calling Read never knows the chain exists.

// Log everything a client sends, but cap it at 1 MB.
r := io.LimitReader(io.TeeReader(conn, os.Stdout), 1<<20)
io.Copy(io.Discard, r) // reads, logs, and discards — all through one pipe

io.Pipe

An io.Pipe creates a synchronous in-memory pipe: one side is a *io.PipeReader, the other a *io.PipeWriter. Writes on one end block until the other end reads — it is an io.Reader/io.Writer pair connected together, no goroutine required:

r, w := io.Pipe()

go func() {
    defer w.Close()
    fmt.Fprintf(w, "Hello from the pipe!\n")
}()

data, _ := io.ReadAll(r)
fmt.Print(string(data))

io.Pipe is useful for connecting a function that expects a Writer to one that expects a Reader without staging data through a temporary file or buffer.

Why This Matters for Networking

Every net.Conn is an io.Reader and an io.Writer. This means:

• io.Copy(os.Stdout, conn) prints everything the connection sends.
• io.Copy(conn, os.Stdin) sends everything you type.
• bufio.NewScanner(conn) reads lines from a network connection — Scanner wraps any io.Reader.
• json.NewDecoder(conn).Decode(&v) parses JSON from a network stream — Decoder wraps any io.Reader.
• io.LimitReader(conn, 1<<20) caps a request body at 1 MB — critical for any HTTP handler that reads r.Body.
• conn.SetReadDeadline(time.Now().Add(5 * time.Second)) makes the next Read time out — the interface stays the same, but the concrete net.Conn underneath enforces a deadline.

Go's networking primitives don't need special types for every data source and sink. They need two interfaces, and everything — files, connections, buffers, encoders, decompressors — agrees to implement them.

Try It Yourself: A Line Echo Server with bufio.Scanner

package main

import (
    "bufio"
    "fmt"
    "log"
    "net"
)

func main() {
    ln, err := net.Listen("tcp", ":8080")
    if err != nil {
        log.Fatal(err)
    }
    defer ln.Close()
    fmt.Println("Listening on :8080...")

    for {
        conn, err := ln.Accept()
        if err != nil {
            log.Println("accept error:", err)
            continue
        }
        go func(c net.Conn) {
            defer c.Close()
            scanner := bufio.NewScanner(c) // c is an io.Reader
            for scanner.Scan() {
                line := scanner.Text()
                fmt.Fprintf(c, "echo: %s\n", line) // c is an io.Writer
            }
        }(conn)
    }
}

bufio.NewScanner wraps the net.Conn and scans it line by line. fmt.Fprintf writes back on the same connection. The Scanner handles buffering, line splitting, and partial reads — no manual byte-slice management required.

Frequently Asked Questions

Why would Read ever return n > 0 and a non-nil error in the same call — isn't that contradictory? It's not a contradiction, it's the contract: Read is telling you "here are the last n bytes I have, and also, that was the end." A common bug is checking err != nil first and returning immediately without looking at n, which silently drops that final chunk of data. Always process p[:n] before checking whether err says to stop.

If io.EOF isn't an error, why does it satisfy the error interface at all? Because Go's error handling has no separate "signal" type — io.EOF is a sentinel value that happens to implement error so it can travel through the same return slot every other failure uses. The distinction isn't in the type, it's in what you do with it: code that treats io.EOF as "stop the loop, everything's fine" versus log.Fatal-ing on every other error is the standard, correct pattern.

When should I reach for io.LimitReader instead of just calling io.ReadAll and checking the length afterward? As soon as the data source isn't fully trusted — which, for networked code, is almost always. io.ReadAll has no way to stop early; it keeps growing its buffer until io.EOF or the process runs out of memory, so checking the length after the call is checking it after the damage is already done. io.LimitReader caps the read itself, before a single byte over the limit is ever allocated.

Why does io.Pipe need a goroutine on each side instead of just working like a buffer? Because it's synchronous by design — a write blocks until a read consumes it, with no internal storage in between. That's different from bytes.Buffer, which happily stores everything you write to it with no reader required. io.Pipe exists specifically for cases where you want backpressure (the writer can't get ahead of the reader), and backpressure requires two independent goroutines actually running at the same time, not just two variables.

Where This Goes From Here

The next chapter covers slices, arrays, and maps — the data structures you will use to hold the bytes, connections, and routing tables that io.Reader and io.Writer stream through. Every networked program in this book, from the simplest echo server to the largest production API, is built on these two interfaces.