Handling JSON and XML over HTTP: Concepts and Go Implementation

"Imagine you're sending a package (data) through the mail (HTTP). JSON and XML are like the forms you fill out to describe what's inside—structured, standardized, and understood by everyone."

Why Use JSON and XML in HTTP?

• Data Interchange: JSON and XML are the most common formats for exchanging data between clients and servers.
• Human-Readable: Both formats are text-based and easy to debug.
• Widely Supported: Every major programming language can read and write JSON and XML.
• Analogy: Like using a customs form—everyone knows how to read it, no matter where it's from.

Content-Type tells the reader how to parse what you sent; get it wrong and every decoder downstream fails silently or loudly.

JSON vs XML: Quick Comparison

Go in Action: Serving and Consuming JSON (with Nested Data)

Let's build a server that returns a list of users (with nested fields) as JSON, and a client that fetches and posts users.

Server Example: Serve and Accept JSON

The handler pair below serves the in-memory users slice as JSON and accepts new users posted to the same route, guarding the slice with a sync.Mutex since both handlers can run concurrently on different goroutines:

type User struct {
    ID    int    `json:"id"`
    Name  string `json:"name"`
    Email string `json:"email"`
}

var (
    users = []User{
        {ID: 1, Name: "Alice", Email: "alice@example.com"},
        {ID: 2, Name: "Bob", Email: "bob@example.com"},
    }
    mu sync.Mutex
)

func getUsers(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/json")
    mu.Lock()
    defer mu.Unlock()
    json.NewEncoder(w).Encode(users)
}

func addUser(w http.ResponseWriter, r *http.Request) {
    var u User
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "Invalid body", http.StatusBadRequest)
        return
    }
    if err := json.Unmarshal(body, &u); err != nil {
        http.Error(w, "Invalid JSON", http.StatusBadRequest)
        return
    }
    mu.Lock()
    defer mu.Unlock()
    u.ID = len(users) + 1
    users = append(users, u)
    w.WriteHeader(http.StatusCreated)
    json.NewEncoder(w).Encode(u)
}

The full file (imports, the /users method-routing main) is json_server.go.

Client Example: Fetch and Post JSON

The matching client fetches the list with http.Get, decodes it straight from the response body, then encodes and posts a new User:

func main() {
    resp, err := http.Get("http://localhost:8080/users")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    var users []User
    if err := json.NewDecoder(resp.Body).Decode(&users); err != nil {
        panic(err)
    }
    for _, u := range users {
        fmt.Printf("- ID: %d, Name: %s, Email: %s\n", u.ID, u.Name, u.Email)
    }

    newUser := User{Name: "Charlie", Email: "charlie@example.com"}
    data, _ := json.Marshal(newUser)
    resp2, err := http.Post(
        "http://localhost:8080/users", "application/json",
        bytes.NewBuffer(data),
    )
    if err != nil {
        panic(err)
    }
    defer resp2.Body.Close()
    var created User
    body, _ := io.ReadAll(resp2.Body)
    json.Unmarshal(body, &created)
    fmt.Println("Created user:", created)
}

Full file: json_client.go.

Key Details:

• Use json.NewEncoder(w).Encode(data) to send JSON from a handler.
• Use json.NewDecoder(r.Body).Decode(&target) to parse JSON from a request.
• For nested/child fields, use Go slices or structs (see the User struct for examples).
• Always set the Content-Type: application/json header.

Testing:

• Use curl to test GET: curl http://localhost:8080/users
• Use curl to test POST: curl -X POST -H "Content-Type: application/json" -d '{"name":"Test","email":"test@example.com"}' http://localhost:8080/users

A Closer Look: Marshal/Unmarshal, Struct Tags, and Exported Fields

Before decoding a real HTTP body, it helps to see the raw building blocks encoding/json gives you: Marshal/Unmarshal work entirely in memory on []byte, while Encoder/Decoder stream to and from any io.Writer/io.Reader (like w and r.Body in a handler).

package main

import (
    "encoding/json"
    "fmt"
)

type User struct {
    ID       int    `json:"id"`
    Name     string `json:"name"`
    Email    string `json:"email,omitempty"`
    password string // unexported: encoding/json can't see it at all
}

func main() {
    u := User{ID: 1, Name: "Ada", password: "secret"}

    data, err := json.Marshal(u)
    if err != nil {
        fmt.Println("marshal error:", err)
        return
    }
    fmt.Println(string(data)) // Email omitted, password never appears

    var decoded User
    if err := json.Unmarshal(data, &decoded); err != nil {
        fmt.Println("unmarshal error:", err)
        return
    }
    fmt.Printf("%+v\n", decoded)
}

Streaming Decode: Why json.NewDecoder(r.Body) Beats ReadAll+Unmarshal

ioutil.ReadAll(r.Body) followed by json.Unmarshal first buffers the entire request body into a []byte, then parses it — for a large body that's two full copies of the data sitting in memory before you've validated a single field. json.NewDecoder(r.Body).Decode(&v) instead parses directly from the stream, token by token, without ever holding the whole body in memory at once. Combined with http.MaxBytesReader, this also caps the worst case memory a single request can consume:

package main

import (
    "encoding/json"
    "net/http"
)

type CreateUserRequest struct {
    Name  string `json:"name"`
    Email string `json:"email"`
}

const maxBodyBytes = 1 << 20 // 1 MiB

func createUserHandler(w http.ResponseWriter, r *http.Request) {
    r.Body = http.MaxBytesReader(w, r.Body, maxBodyBytes)

    var req CreateUserRequest
    dec := json.NewDecoder(r.Body)
    dec.DisallowUnknownFields() // reject fields the struct doesn't have

    if err := dec.Decode(&req); err != nil {
        http.Error(w, "invalid JSON: "+err.Error(),
            http.StatusBadRequest)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    json.NewEncoder(w).Encode(map[string]string{
        "status": "created",
        "name":   req.Name,
    })
}

Go in Action: Serving and Consuming XML (with Nested Elements)

Let's build a server that returns a list of products (with nested tags) as XML, and a client that fetches and posts products.

Server Example: Serve and Accept XML

The Tags []string \xml:"tags>tag"`tag is the nested-element part: it tellsencoding/xmlto wrap each tag in aelement, one` child per entry.

type Product struct {
    XMLName xml.Name `xml:"product"`
    ID      int      `xml:"id"`
    Name    string   `xml:"name"`
    Tags    []string `xml:"tags>tag"`
}

type ProductList struct {
    XMLName  xml.Name  `xml:"products"`
    Products []Product `xml:"product"`
}

var (
    products = []Product{
        {ID: 1, Name: "Widget", Tags: []string{"gadget", "tool"}},
        {ID: 2, Name: "Gizmo", Tags: []string{"device"}},
    }
    mu sync.Mutex
)

func getProducts(w http.ResponseWriter, r *http.Request) {
    w.Header().Set("Content-Type", "application/xml")
    mu.Lock()
    defer mu.Unlock()
    xml.NewEncoder(w).Encode(ProductList{Products: products})
}

func addProduct(w http.ResponseWriter, r *http.Request) {
    var p Product
    body, err := io.ReadAll(r.Body)
    if err != nil {
        http.Error(w, "Invalid body", http.StatusBadRequest)
        return
    }
    if err := xml.Unmarshal(body, &p); err != nil {
        http.Error(w, "Invalid XML", http.StatusBadRequest)
        return
    }
    mu.Lock()
    defer mu.Unlock()
    p.ID = len(products) + 1
    products = append(products, p)
    w.WriteHeader(http.StatusCreated)
    xml.NewEncoder(w).Encode(p)
}

Full file: xml_server.go.

Client Example: Fetch and Post XML

func main() {
    resp, err := http.Get("http://localhost:8081/products")
    if err != nil {
        panic(err)
    }
    defer resp.Body.Close()
    var plist ProductList
    if err := xml.NewDecoder(resp.Body).Decode(&plist); err != nil {
        panic(err)
    }
    for _, p := range plist.Products {
        fmt.Printf("- ID: %d, Name: %s, Tags: %v\n", p.ID, p.Name, p.Tags)
    }

    newProduct := Product{Name: "Thingamajig", Tags: []string{"novelty", "fun"}}
    data, _ := xml.Marshal(newProduct)
    resp2, err := http.Post(
        "http://localhost:8081/products", "application/xml",
        bytes.NewBuffer(data),
    )
    if err != nil {
        panic(err)
    }
    defer resp2.Body.Close()
    var created Product
    body, _ := io.ReadAll(resp2.Body)
    xml.Unmarshal(body, &created)
    fmt.Println("Created product:", created)
}

Full file: xml_client.go.

Key Details:

• Use xml.NewEncoder(w).Encode(data) to send XML from a handler.
• Use xml.NewDecoder(r.Body).Decode(&target) to parse XML from a request.
• For nested/child elements, use Go slices with struct tags (see the Product struct and tags>tag).
• Always set the Content-Type: application/xml header.

Testing:

• Use curl to test GET: curl http://localhost:8081/products
• Use curl to test POST: curl -X POST -H "Content-Type: application/xml" -d '<product><name>Test</name><tags><tag>foo</tag></tags></product>' http://localhost:8081/products

Content Negotiation: One Endpoint, JSON or XML

Real APIs often serve both formats from the same route, choosing the response based on the client's Accept header while trusting Content-Type to describe whatever the client actually sent:

package main

import (
    "encoding/json"
    "encoding/xml"
    "net/http"
    "strings"
)

type Product struct {
    XMLName xml.Name `xml:"product" json:"-"`
    ID      int      `xml:"id" json:"id"`
    Name    string   `xml:"name" json:"name"`
}

func productHandler(w http.ResponseWriter, r *http.Request) {
    products := []Product{{ID: 1, Name: "Widget"}}

    accept := r.Header.Get("Accept")
    switch {
    case strings.Contains(accept, "application/xml"):
        w.Header().Set("Content-Type", "application/xml")
        xml.NewEncoder(w).Encode(products)
    default:
        w.Header().Set("Content-Type", "application/json")
        json.NewEncoder(w).Encode(products)
    }
}

Try It Yourself: Safe Decoding and Content Negotiation

1. Wire up createUserHandler from earlier behind /users and confirm a valid POST body succeeds while DisallowUnknownFields rejects a body with an extra, misspelled field (e.g. "naem" instead of "name").
2. Send a POST body larger than maxBodyBytes (a 2 MiB JSON blob works) and confirm the handler now returns an error instead of silently allocating an unbounded buffer — inspect the error message to see the MaxBytesReader limit kick in.
3. Wire up productHandler and request the same URL twice: once with curl -H "Accept: application/json" ... and once with curl -H "Accept: application/xml" .... Confirm the Content-Type of each response matches what you asked for.
4. Bonus: remove the err != nil check after a Decode call on purpose and feed the handler garbage input — observe how the zero value flows silently through the rest of the program until it fails somewhere confusing, far from the real bug.

Practical Exercise Files

• JSON Server Example
• JSON Client Example
• XML Server Example
• XML Client Example

Frequently Asked Questions

Why did my struct field disappear from the JSON output with no error at all? Almost always it's an unexported field — one starting with a lowercase letter, like the password field in the User example earlier. Reflection simply cannot see unexported fields from outside their own package, so encoding/json and encoding/xml skip them silently rather than failing loudly. If a field genuinely needs to stay out of the wire format, export it and tag it json:"-" instead of leaving it lowercase — that way the omission is a documented choice, not a mystery.

Should I use json.Unmarshal or json.NewDecoder(r.Body).Decode(&v) in an HTTP handler? Reach for Decoder/Encoder whenever you're working directly with r.Body or http.ResponseWriter. Unmarshal needs the whole body already sitting in a []byte, which means an extra ioutil.ReadAll pass and two full copies of the data in memory before you've validated a single field. Decoder.Decode streams straight from the connection, and pairs naturally with http.MaxBytesReader to cap how much a single request can cost you.

My XML decode succeeded but the struct came back completely empty — what happened? Check whether the incoming document declares an xmlns. A struct tag like xml:"product" only matches an element named product in the default (no) namespace, so a namespaced <product xmlns="urn:example:catalog"> silently fails to match and Decode returns no error — just zero values. The fix is to include the namespace URI in the tag (xml:"urn:example:catalog product") or decode into a type with an XMLName xml.Name field so you can inspect what namespace actually arrived.

Why does the sample content-negotiation handler check Accept for XML but Content-Type for what it just wrote? Because they answer two different questions. Accept on an incoming request is the client saying what it would prefer to receive back — a hint you can honor or ignore. Content-Type on the response is your server truthfully describing what it just encoded, and every client-side decoder depends on that being accurate. Mixing the two up is exactly the class of bug that works fine in curl (where you set headers explicitly) but breaks with real browsers or libraries that default to Accept: */*.

Is DisallowUnknownFields actually necessary, or just extra caution? It earns its keep the first time a client typos a field name — say "naem" instead of "name" — and your handler would otherwise decode that request into a struct with the correct field silently left at its zero value, no error raised. Combined with http.MaxBytesReader, it turns two of the most common "the request looked fine but the data was wrong" bugs into an immediate, explicit 400 Bad Request instead of a debugging session two layers downstream.

Key Takeaways

• Use JSON for most web APIs—compact, fast, and easy in Go.
• Use XML when you need to interoperate with older systems or standards.
• Always set the correct Content-Type header.
• Use Go's encoding/json and encoding/xml for safe, automatic encoding/decoding.
• For nested/child data, use slices and struct tags.
• Test with curl or Postman for real-world scenarios.
• Prefer Decoder/Encoder streaming over ReadAll+Unmarshal for HTTP bodies, and always check the returned error.
• Guard request bodies with http.MaxBytesReader to prevent unbounded memory use from a single request.
• Remember only exported struct fields are encoded/decoded—unexported fields vanish silently.
• Watch for XML namespace mismatches: a wrong xmlns fails silently, leaving zero values instead of an error.