sackosoft/sanctum
Sanctum is an event streaming and storage platform which can be scripted by 'spells' written in Lua.
Welcome, sorcerer, to the Sanctum. Write spells in Lua for transforming and acting on event streams of magical energy.
Sanctum is an event streaming, storage and processing platform. It lets you, the sorcerer, craft powerful spells in Lua to process and transform streams of events. Sanctum handles the complexities of storage, state and routing these energies across the ethereal planes, so that you can focus on writing the spells for your use case.
For more detailed reference information and documentation, refer to the docunomicon
.
Options for installing and hosting Sanctum can be found in the docunomicon
.
Sanctum uses the just
command runner. Refer to CONTRIBUTING.md
for more information about
dependencies. The commonly used commands are described below.
just test
just freeze
just debug
Sanctum is designed for realtime event streaming workloads. The switch from Lua 5.4 (provided by ziglua
) to LuaJIT
(provided by zig-luajit
) has improved a target spell casting benchmarks by 36%. The overhead for Sanctum to cast
Lua spells is around ~500 nanoseconds, based on January 2025 benchmarks. As a result, Sanctum can cast millions of spells per second
per core, even on modest hardware like my Intel Xeon E5-2680 -- a processor released in 2013.
Sanctum will work best if your workload is bound by the flow of magical energy rather than the processing of those energies.
Demonstrates event processing with Spells on a throw-away runtime. Sanctum runs as a CLI application and covered by a suite of regression tests to enable building and refactoring the core engine. Runs on a single node (no networking, no persistence).
Docunomicon - Definition and example of Lua spells
Minimal documentation for spells in the runtime.
Spell Casting
Spells can be loaded dynamically into the sanctum from files.
Spells are loaded and unloaded with the prepare and unprepare lifecycle hooks.
Counting loop spell is added with a reusable regression test.
Many reusable regression tests can be defined.
Regression tests are fully self-contained. Seed events, the initial event loop event,
can be specified in Lua and loaded by the runtime.
Malformed spells (spell parse failure) are handled gracefully.
Handle parsing failing line number from errors Lua message.
Unstable magic (internal spell failures) is handled gracefully.
Add support to serialize and deserialize magical energies as MessagePack buffers.
Spells can produce magical energy or act as a terminal action, producing nothing.
Spells can be bound to energy streams subscriptions.
Support for topic-based subscriptions
Support for filter-based subscriptions
Migrate from Lua 5.4 to LuaJIT.
Build a proof of concept sanctum application for demo/validation.
Should sanctum spells be bundled/packaged into an app?
Remove this section from the README, create release notes and initial release artifacts.
Upgrades the runtime with persistence for events and spell state. Sanctum can be used for real workloads with no guarantees. Runs on a single node (no networking).
Storage Engine MVP
Durable event logging.
Durable spell state
Spells can leverage a library for saving and loading data.
Spells can use a key value store.
Spells can leverage range queries (e.g. SELECT * FROM state WHERE v > 10 AND v < 20). I.e. support clustering in state store.
(Option) Link SQLite?
(Option) Provide simple data structures, serialize and save to disk?
Add initial logging and metrics to identify performance characteristics.
Sanctum fully featured as an MVP.
Production Readiness
Crash recovery guarantees.
Limits defined and scale targets established.
Platform Stability
API contracts
Documentation
Performance optimization
Wrap the Lua VM so that detailed telemetry can be emitted, per instance.
- memory usage, spell execution count+time, event input and output counters.
Support N:M event inputs and outputs for spells (currently only supports 1:0 and 1:1).
Novice sorcers may struggle to uncover the mysteries of the Sanctum:
Mystical Term | Technical Meaning |
---|---|
Sanctum | A scriptable event streaming and storage platform. |
Sorcerers | Developers building on Sanctum. |
Spells | User-defined modules invoked with events in an event stream. Spells may cause new events to be created. |
Energy Streams | Ordered event streams with immutable data. |
Runes | Configuration options. |
Runeset | YAML configuration file of options which can be used by the Sanctum or its Spells. |
Thank you, brave scholar of the dark arts, please refer to CONTRIBUTING.md.
Incubation.
This mystical undertaking is still in its early phases.