Zig ZBuild

this project explores the potential of custom zig build steps. It contains several addons and can be added to projects using zig's package manager.

Disclaimer: some of it may break!

Generated Documentation

Source on Gitlab

for the impatient, this will build and serve the autodoc of ZBuild:

max@muster:~/zig-webterm$ zig build run
steps [54/56] zig build-exe zigtty Debug native... LLVM Emit Object...
Open http://127.0.0.1:3000 in your browser

zig build zb

This Repository's Zig Package can add a zig build zb step to your build command by adding and calling it in build.zig:

// build.zig.zon
.dependencies = .{
    .ZBuild = .{
        .url = "https://gitlab.com/dasimmet/zbuild/-/archive/e6a55b6739caad97f0abd1a10f4925f4c6b2ab15/zbuild.tar.gz",
        .hash = "1220b57901ab8ec55fab947461397e817349b88b1f92ec06bcfb9b14bb2c36eb0629",
    },
},

// build.zig
const ZBuild: type = @import("ZBuild").ZBuild;

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});
    _ = ZBuild.init(.{
        .owner = b,
    });
    
    // rest of your build.zig code here
}

Then, running zig build zb will build and run integrate with your build:

tobi@lap:~/repos/zbuild$ zig build zb
cli: /home/tobi/repos/zbuild/zig-cache/o/d70288effb4d6bd4bd4e78d729411a09/zb
usage: zig build zb -- <subcommand>

# Available Subcommands

Among others supports subcommands for managing your build.zig.zon

ZBuild.Step.Fetch

This step exposes the new (in zig 0.12.0) zig fetch subcommand by receiving the url and hash as input, and returning a GeneratedFile of the package's path in the zig global cache.

Like dependencies in build.zig.zon, it will fail if the hash was not provided and print the stdout of zig fetch.

ZBuild.Step.Emsdk

an example use case of ZBuild.Step.Fetch is fetching and setting up an SDK that may not contain a build.zig file. It cannot be used to pass dependencies, but only needs to be executed when another step actually depends on it. the contained .fetch member can be added to a separate top level step (e.g. fetch), so off-the-grid builds are possible after running zig build fetch.

this step will fetch a specified version of Emsdk and install it, all managed in zig's cache.

ZBuild.Step.Download

The module has a Download build step to fetch a file into the zig-cache either project locally or globally. This is similar to how the package manager works.

The input url of Download accepts a ZBuild.LazyType, a generic version of std.Build.LazyPath. The output_file is a GeneratedFile path to the downloaded file in zig's cache.

ZBuild.Step.JZon

the src/build/Step/JZon.zig step converts a json file from a LazyPath into a .zig file with a .zon-like syntax, but assigned to a single identifier data:

this allowes a file to be @import -ed ans used as a comptime lookup table.

pub const data = <converted json data in ZON format here>;

It can then be imported as a module in subsequent steps and processed at comptime.

The example zig build mimedata -Dmime-indent=2 fetches a list of mimetypes per extension and a list of mimetype magic numbers using a Download step, then convert them into a .zig file in src.

The result is imported in ZBuild.CompressMap and in comptime maps file extensions to the appropriate Content-Type header.

ZBuild.Step.Compress

the repo holds a src/build/Step/Compress.zig to bundle the responses from the generated ZBuild documentation into a std.StaticStringMap zig source code file. While this intermediate representation is not too efficient for large binaries, it allows comptime as well as runtime access to the compressed assets body as well as some metadata including:

• uncompressed filesize
• filesystem create,access and modify timestamps
• hex encoded md5 and sha256 checksums
• http mimetypes based on file extension and/or magic numbers

The Step's output_file can be added as a module to other compilations and ships with all code needed for decompression outside the zig standard library.

const ZBuild = @import("ZBuild");
const assets = ZBuild.Step.Compress.init(
    b,
    .{ .path = "src/assets" },
    "assets",
);
assets.method = .Deflate;
// the method enum will enable compression on the file entries.
// it will be shipped with the resulting zig file and can be switched on.
// compressors not supported by the standard library will use the
// common command line tools (gzip, xz, zstd) and fail if they are missing.
// TODO: provide a way to set compression method on a per-file basis.

const mod = assets.module(b);

Now, the assets can be imported and accessed with their relative path to the base directory:

const assets = @import("assets");
const path = "index.html";
if (assets.map().get(path)) |res| {
    return Response.render(r, path, res);
}

as modern browsers support Content-Encoding and the zig standard library has built-in Deflate-Support, the content can be served without even decompressing once on the server.

the default example builds the static autodoc documentation of the build Library and serves them.

ZBuild.Step.Serve

the compressed assets contain all information to embed responses into a static webserver binary using zap:

var zb = ZBuild.Step.Serve.init(
    .{
        .owner = b,
        // a module generated by the `ZBuild.Step.Compress` step
        .assets = compress.module(b),
        .dependency = zbuild_dep,
        .name = "serve",

        // build options for the compilation
        .options = std.Build.ExecutableOptions{
            .name = "zbuild-serve",
            // root_source_file is non-optional here, but will
            // be overwritten internally with a path to the dependency's
            // directory.
            .root_source_file = .{ .path = "" },
            .target = target,
            .optimize = optimize,
        },

        // an optional module to handle requests not covered
        // by assets. It needs to provide the same
        // interface as: `src/srv/Api.zig`
        .api = b.addModule(
            "Api",
            .{
                .source_file = .{
                    .path = "src/Api.zig",
                },
            },
        ),
    },
);

The result is a single static binary webserver serving precompressed assets from memory ;-D.

Perspective and Ideas?

• Provide more preprocessing as part or follow-ups of the ZBuild.Step.Compress Step.
• Create a in-memory Filesystem replacement for wasm compilation targets based on the ZBuild.Step.Compress step
• Replace zap with standard library based http server. Not sure wich is better, but zap only supports a subset of standard http headers as of now I think. At Least the If-None-Match header is not being picked up by the server, to http Etags are not handled correctly, and Windows is not supported.
• Ship a complete OAuth2 handler for headless applications including the required webservice, frontend and token storage.
• Expand on the Idea of zig build zb to add custom subcommands to zig build, especially for common package manager tasks
• Maybe zbuild eval can also set it's own (and zig's) autocompletion.