mzg

mzg is a MessagePack library for Zig with no allocations, and the API favors the streaming usage.

How to use

Run the following command to add this project as a dependency

zig fetch --save git+https://github.com/uyha/mzg.git#v0.1.3

In your build.zig, add the following

const mzg = b.dependency("mzg", .{
    .target = target,
    .optimize = optimize,
});
// Replace `exe` with your actual library or executable
exe.root_module.addImport("mzg", mzg.module("mzg"));

API

High level functions

Packing functions

pack is for packing Zig values using a writer
packAdapted is like pack but it accepts a map that defines how certain types should be packed.
packWithOptions is like pack but it accepts a mzg.PackOptions parameter to control the packing behavior.
packAdaptedWithOptions is like packWithOptions but it accepts a map that defines how certain types should be packed.

Unpacking functions

unpack is for unpacking a MessagePack message into a Zig object and it does not allocate memory.
unpackAdapted is like unpack but it accepts a map that defines how certain types should be unpacked.
unpackAllocate is like unpack but it accepts an Allocator that allows the unpacking process to allocate memory.
unpackAdaptedAllocate is like unpackAllocate but it accepts a map that defines how a particular type should be packed.

Adapters

adapter.packArray and adapter.unpackArray pack and unpack structs like std.ArrayListUnmanaged
adapter.packMap and adapter.unpackMap pack and unpack structs like std.ArrayHashMapUnmanaged.
adapter.packStream and adapter.unpackStream pack and unpack structs in a stream like manner (length is not specified in the beginning).

Building block functions

There are a set of pack* and unpack* functions that translate between a precise set of Zig types and MessagePack. These functions can be used when implementing custom packing and unpacking.

Examples

All examples live in examples directory.

Simple back and forth

Converting a byte slice to MessagePack and back

pub fn main() !void {
    const allocator = std.heap.page_allocator;
    var buffer: std.ArrayListUnmanaged(u8) = .empty;
    defer buffer.deinit(allocator);

    try mzg.pack(
        "a string with some characters for demonstration purpose",
        buffer.writer(allocator),
    );

    var string: []const u8 = undefined;
    const size = try mzg.unpack(buffer.items, &string);
    std.debug.print("Consumed {} bytes\n", .{size});
    std.debug.print("string: {s}\n", .{string});
}

const std = @import("std");
const mzg = @import("mzg");

Default packing and unpacking for custom types

Certain types can be packed and unpacked by default (refer to Mapping for more details)

pub fn main() !void {
    const allocator = std.heap.page_allocator;
    var buffer: std.ArrayListUnmanaged(u8) = .empty;
    defer buffer.deinit(allocator);

    try mzg.pack(
        Targets{ .position = .init(2000), .velocity = .init(10) },
        buffer.writer(allocator),
    );

    var targets: Targets = undefined;
    const size = try mzg.unpack(buffer.items, &targets);

    std.debug.print("Consumed {} bytes\n", .{size});
    std.debug.print("Targets: {}\n", .{targets});
}

const Position = enum(i32) {
    _,
    pub fn init(raw: std.meta.Tag(@This())) @This() {
        return @enumFromInt(raw);
    }
};
const Velocity = enum(i32) {
    _,
    pub fn init(raw: std.meta.Tag(@This())) @This() {
        return @enumFromInt(raw);
    }
};
const Targets = struct {
    position: Position,
    velocity: Velocity,
};

const std = @import("std");
const mzg = @import("mzg");
const adapter = mzg.adapter;

Adapters for common container types

Many container types in the std library cannot be packed or unpacked by default, but the adapter functions make it easy to work with these types.

pub fn main() !void {
    const allocator = std.heap.page_allocator;
    var buffer: std.ArrayListUnmanaged(u8) = .empty;
    defer buffer.deinit(allocator);

    var in: std.ArrayListUnmanaged(u32) = .empty;
    defer in.deinit(allocator);
    try in.append(allocator, 42);
    try in.append(allocator, 75);
    try mzg.pack(adapter.packArray(&in), buffer.writer(allocator));

    var out: std.ArrayListUnmanaged(u32) = .empty;
    defer out.deinit(allocator);
    const size = try mzg.unpack(
        buffer.items,
        adapter.unpackArray(&out, allocator),
    );
    std.debug.print("Consumed {} bytes\n", .{size});
    std.debug.print("out: {any}\n", .{out.items});
}

const std = @import("std");
const mzg = @import("mzg");
const adapter = mzg.adapter;

Mapping

Default packing from Zig types to MessagePack

Zig Type	MessagePack
`void`, `null`	`nil`
`bool`	`bool`
integers (<=64 bits)	`int`
floats (<=64 bits)	`float`
`?T`	`nil` if value is `null`, pack according to `T` otherwise
enums	`int`
enum literals	`str`
tagged unions	`array` of 2 elements: `int` and the value of the union
packed structs	`int`
structs and tuples	`array` of fields in the order they are declared
`[N]`, `[]`, `[:X]`, and `@Vec` of `u8`	`str`
`[N]`, `[]`, `[:X]`, and `@Vec` of `T`	`array` of `T`
`Ext`	`ext`
`Timestamp`	`Timestamp`
`*T`	`T`

Default unpack from MessagePack to Zig types

MessagePack	Compatible Zig Type
`nil`	`void`, `?T`
`bool`	`bool`
`int`	integers, enums
`float`	floats
`str`	`[]const u8`
`bin`	`[]const u8`
`array`	The length can be read into integers
`map`	The length can be read into integers
`ext`	`Ext`
`Timestamp`	`Timestamp`

Customization

Packing

When an enum/union/struct has an mzgPack function that returns mzg.PackError(@TypeOf(writer))!void can be called with

// Value cares about the options passed by the caller
value.mzgPack(options, map, writer);

With the arguments being

value is the enum/union/struct being packed.
options is mzg.PackOptions.
map is a tuple of 2 element tuples whose 1st element being a type and 2nd element being a packing adapter function.
writer is anytype that can be called with writer.writeAll(&[_]u8{});.

The function will be called when the mzg.pack function is used.

Unpacking

When an enum/union/struct has

mzgUnpack function that returns UnpackError!usize and can be called with
```
out.mzgUnpack(map, buffer);
```
mzgUnpackAllocate function that returns UnpackError!usize and can be called with
```
out.mzgUnpackAllocate(allocator, map, buffer);
```