Zig HTTP Sentinel

A production-grade HTTP client library for Zig 0.16.0-dev.1303+, extracted from high-frequency trading systems.

Built on Modern Zig: Uses std.Io.Threaded architecture for true thread-safe concurrent operations

Developed by QUANTUM ENCODING LTD Contact: [email protected]

Features

Modern Zig Architecture: Built on std.Io.Threaded for reliable concurrent operations
Client-Per-Worker Pattern: Each thread owns its HTTP client - zero contention, true parallelism
Memory-Safe: RAII-style cleanup with explicit ownership
Full HTTP Support: GET, POST, PUT, PATCH, DELETE, HEAD methods
Automatic GZIP Decompression: Transparent handling of compressed responses
Configurable: Request options for timeouts and body size limits
Production-Tested: Running in live trading systems handling thousands of requests/second

Quick Start

Installation

Add to your build.zig.zon:

.dependencies = .{
    .http_sentinel = .{
        .url = "https://github.com/YOUR_USERNAME/zig-http-sentinel/archive/refs/tags/v1.0.0.tar.gz",
        .hash = "YOUR_HASH_HERE",
    },
},

Then in your build.zig:

const http_sentinel = b.dependency("http_sentinel", .{
    .target = target,
    .optimize = optimize,
});
exe.root_module.addImport("http-sentinel", http_sentinel.module("http-sentinel"));

Basic Usage

const std = @import("std");
const HttpClient = @import("http-sentinel").HttpClient;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    // Create client with std.Io.Threaded backend
    var client = try HttpClient.init(allocator);
    defer client.deinit();

    // Make request
    const headers = [_]std.http.Header{
        .{ .name = "Accept", .value = "application/json" },
    };

    var response = try client.get("https://api.example.com/data", &headers);
    defer response.deinit();

    std.debug.print("Status: {}\n", .{response.status});
    std.debug.print("Body: {s}\n", .{response.body});
}

Core Architecture

The std.Io.Threaded Foundation

HTTP Sentinel is built on Zig's modern std.Io.Threaded architecture:

pub const HttpClient = struct {
    allocator: std.mem.Allocator,
    io_threaded: *std.Io.Threaded,  // ← Key: Heap-allocated I/O subsystem
    client: http.Client,

    pub fn init(allocator: std.mem.Allocator) !HttpClient {
        const io_threaded = try allocator.create(std.Io.Threaded);
        io_threaded.* = std.Io.Threaded.init(allocator);
        const io = io_threaded.io();

        return .{
            .allocator = allocator,
            .io_threaded = io_threaded,
            .client = http.Client{
                .allocator = allocator,
                .io = io,
            },
        };
    }
};

Why This Matters:

Each HttpClient owns its I/O subsystem
Enables true thread-safe operation
No hidden shared state
Foundation of the client-per-worker pattern

📖 See MODERN_ZIG_PATTERNS.md for complete implementation details

The Client-Per-Worker Pattern

✅ Correct: Each Thread Owns Its Client

const Worker = struct {
    allocator: std.mem.Allocator,

    fn run(self: @This()) void {
        // Each worker creates its own client
        var client = HttpClient.init(self.allocator) catch unreachable;
        defer client.deinit();

        // Make requests - no contention!
        var response = client.get(url, &.{}) catch return;
        defer response.deinit();

        // Process response...
    }
};

// Launch workers
for (&threads) |*thread| {
    thread.* = try std.Thread.spawn(.{}, Worker.run, .{worker});
}

Benefits:

✅ Zero contention (no mutexes needed)
✅ True parallelism
✅ Scales linearly with CPU cores
✅ No race conditions
✅ Simple, clear ownership

❌ Incorrect: Sharing Clients (Don't Do This)

// This pattern is fundamentally broken in Zig 0.16
var shared_client = HttpClient.init(allocator);
var mutex = std.Thread.Mutex{};

fn workerThread(client: *HttpClient, mutex: *std.Thread.Mutex) void {
    mutex.lock();
    defer mutex.unlock();
    // Even with mutex, internal state can race!
    const response = client.get(...);  // ← Race conditions possible
}

Why It Fails:

std.Io.Threaded manages thread-local I/O resources
Internal buffers and state not protected by your mutex
Connection pooling state can race
TLS state is per-thread

API Reference

Response Structure

pub const Response = struct {
    status: http.Status,
    body: []u8,
    allocator: std.mem.Allocator,

    pub fn deinit(self: *Response) void;
};

HTTP Methods

All methods follow RAII cleanup pattern:

// GET
var response = try client.get(url, headers);
defer response.deinit();

// POST
var response = try client.post(url, headers, body);
defer response.deinit();

// PUT
var response = try client.put(url, headers, body);
defer response.deinit();

// PATCH
var response = try client.patch(url, headers, body);
defer response.deinit();

// DELETE
var response = try client.delete(url, headers);
defer response.deinit();

// HEAD
var response = try client.head(url, headers);
defer response.deinit();

Request Options

pub const RequestOptions = struct {
    max_body_size: usize = 10 * 1024 * 1024,  // Default: 10MB
    timeout_ns: u64 = 0,  // 0 = no timeout
};

// Custom options
const options = HttpClient.RequestOptions{
    .max_body_size = 50 * 1024 * 1024,  // 50MB
    .timeout_ns = 30 * std.time.ns_per_s,  // 30 seconds
};

var response = try client.getWithOptions(url, headers, options);
defer response.deinit();

Advanced Features

Automatic GZIP Decompression

HTTP Sentinel automatically detects and decompresses gzip-encoded responses:

// Server sends: Content-Encoding: gzip
var response = try client.get(url, &.{});
defer response.deinit();

// response.body is automatically decompressed
std.debug.print("Decompressed body: {s}\n", .{response.body});

Custom Headers

const headers = [_]std.http.Header{
    .{ .name = "Authorization", .value = "Bearer YOUR_TOKEN" },
    .{ .name = "Content-Type", .value = "application/json" },
    .{ .name = "User-Agent", .value = "MyApp/1.0" },
};

var response = try client.post(url, &headers, json_body);
defer response.deinit();

Error Handling

const response = client.get(url, &.{}) catch |err| {
    std.debug.print("Request failed: {}\n", .{err});
    return err;
};
defer response.deinit();

// Check status before processing
if (response.status != .ok) {
    std.debug.print("HTTP error: {}\n", .{response.status});
    return error.HttpError;
}

// Safe to parse body
const data = try std.json.parseFromSlice(..., response.body, .{});

Examples

Basic GET Request

zig build run-basic

const std = @import("std");
const HttpClient = @import("http-sentinel").HttpClient;

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();

    var client = try HttpClient.init(gpa.allocator());
    defer client.deinit();

    var response = try client.get("https://httpbin.org/get", &.{});
    defer response.deinit();

    std.debug.print("{s}\n", .{response.body});
}

Concurrent Requests

zig build run-concurrent

const Worker = struct {
    id: usize,
    allocator: std.mem.Allocator,
    success_count: *std.atomic.Value(u32),

    fn run(self: @This()) void {
        var client = HttpClient.init(self.allocator) catch return;
        defer client.deinit();

        var i: u32 = 0;
        while (i < 10) : (i += 1) {
            var response = client.get(url, &.{}) catch continue;
            defer response.deinit();

            if (response.status == .ok) {
                _ = self.success_count.fetchAdd(1, .monotonic);
            }
        }
    }
};

pub fn main() !void {
    var success_count = std.atomic.Value(u32).init(0);

    var threads: [4]std.Thread = undefined;
    for (&threads, 0..) |*thread, i| {
        const worker = Worker{
            .id = i,
            .allocator = allocator,
            .success_count = &success_count,
        };
        thread.* = try std.Thread.spawn(.{}, Worker.run, .{worker});
    }

    for (&threads) |*thread| {
        thread.join();
    }

    std.debug.print("Completed: {}\n", .{success_count.load(.monotonic)});
}

AI Client (Anthropic Claude)

export ANTHROPIC_API_KEY=your_key_here
zig build run-anthropic

Demonstrates:

JSON payload construction
API authentication
Response parsing
Multi-turn conversations

See examples/anthropic_client.zig for full implementation.

Testing

Run the test suite:

zig build test

Run examples:

zig build run-basic        # Basic GET/POST
zig build run-concurrent   # Concurrent workers
zig build run-anthropic    # AI client demo

Performance

Benchmarks (4 workers, 20 total requests)

Pattern: Client-Per-Worker
Workers: 4
Requests per worker: 5

Latency: 150-300ms per request (network dependent)
Throughput: 40-60 requests/second
Memory: ~2MB per worker (includes TLS state)
CPU: <5% utilization (I/O bound)
Contention: Zero (no mutexes)

Memory Overhead

HttpClient instance: ~1.5KB
  - std.Io.Threaded: ~512 bytes
  - http.Client: ~1KB
  - Bookkeeping: ~128 bytes

Per-request: 100KB-10MB
  - Response body: variable (default limit: 10MB)
  - Transfer buffer: 8KB (stack)
  - Internal buffers: ~2KB

Production Deployment

Requirements

Zig Version: 0.16.0-dev.1303+ (run zig version to check)
OS: Linux, macOS, Windows (TLS support required)
Memory: ~2-4MB per concurrent worker thread
Network: HTTPS/TLS support enabled

Production Checklist

Use std.heap.GeneralPurposeAllocator in debug mode (detects leaks)
Use std.heap.c_allocator or arena in production (performance)
Configure RequestOptions.max_body_size based on your APIs
Set RequestOptions.timeout_ns for all requests
Always check response.status before parsing body
Use defer response.deinit() immediately after request
One HttpClient per worker thread (never share!)
Profile memory under load with your actual workload

Error Handling Strategy

pub fn fetchData(allocator: std.mem.Allocator) ![]u8 {
    var client = try HttpClient.init(allocator);
    defer client.deinit();

    var response = try client.get(url, &.{});
    defer response.deinit();

    // Validate status
    if (response.status != .ok) {
        return error.HttpError;
    }

    // Validate content type
    // (would need to store headers in Response for this)

    // Return owned copy
    return try allocator.dupe(u8, response.body);
}

Migration from Older Versions

From Zig 0.11/0.12

Key changes in Zig 0.16:

// OLD (0.11/0.12)
const uri = try std.Uri.fromString(url);  // ❌ Removed

// NEW (0.16)
const uri = try std.Uri.parse(url);  // ✅ Use this

// OLD (0.11/0.12)
const module = b.createModule(.{
    .source_file = .{ .path = "src/lib.zig" },  // ❌ Old API
});

// NEW (0.16)
const module = b.addModule("name", .{
    .root_source_file = b.path("src/lib.zig"),  // ✅ New API
});

📖 See MODERN_ZIG_PATTERNS.md for complete migration guide

Common Pitfalls

1. Sharing Clients Across Threads ❌

// DON'T DO THIS
var global_client = try HttpClient.init(allocator);
for (threads) |*t| {
    t.* = try std.Thread.spawn(.{}, worker, .{&global_client});
}

Fix: Create client per thread (see Client-Per-Worker Pattern above)

2. Forgetting defer ❌

// Memory leak!
var response = try client.get(url, &.{});
return;  // ← Leaked response.body

Fix: Always use defer response.deinit();

3. Not Checking Status ❌

var response = try client.get(url, &.{});
defer response.deinit();
const data = try std.json.parseFromSlice(..., response.body);  // Might be error HTML!

Fix: Check response.status before parsing

4. Wrong Header Type ❌

const headers = [_]std.http.Header{...};
var response = try client.get(url, headers);  // Type error

Fix: Pass slice &headers not array

Documentation

MODERN_ZIG_PATTERNS.md - Complete implementation patterns for Zig 0.16
examples/ - Working code examples
src/http_client.zig - Core implementation
API Docs: Run zig build-lib src/lib.zig -femit-docs for generated documentation

Contributing

Contributions welcome! Please ensure:

All tests pass (zig build test)
Code follows Zig style conventions
New features include tests and examples
Documentation updated
Runs on Zig 0.16.0-dev.1303+

License

MIT License - See LICENSE file for details

Copyright © 2025 QUANTUM ENCODING LTD
Website: https://quantumencoding.io
Contact: [email protected]

Acknowledgments

This library emerged from production high-frequency trading systems at QUANTUM ENCODING LTD, where reliability and performance under extreme load are non-negotiable. The patterns documented here represent lessons learned from processing millions of requests in live trading environments.

The std.Io.Threaded architecture is a fundamental shift in how HTTP clients work in Zig 0.16, and this library demonstrates the correct patterns for leveraging it in production.

Support

Issues: GitHub Issues
Email: [email protected]
Docs: MODERN_ZIG_PATTERNS.md

Built with ❤️ for the Zig community by QUANTUM ENCODING LTD

README