scout

A fast, local-first search engine built in Zig

scout lets you crawl and index websites locally, then search them offline using full-text search powered by SQLite FTS5.

Getting Started

1. Clone the Repository

git clone https://github.com/KDesp73/scout && cd scout

2. Build with Zig

zig build

3. Initialize the Database

scout init

4. Start Crawling

scout crawl --seed "wikipedia.org" --infinite

5. Search the Index

scout query --input "your search terms here"

6. Serve a website version of the search engine

scout serve --ip 127.0.0.1 --port 8080

Features

• Website crawler with depth control
• Indexing of page titles, descriptions, and full content
• Local full-text search via SQLite FTS5
• CLI-first interface with easy commands

Requirements

• Zig 0.14.1
• SQLite with FTS5 support
• A Unix-like system (Linux/macOS)

Example Commands

# Crawl a single host
scout crawl --seed "example.com" --depth 2

# View visited pages
scout list --pages

# Check the crawl queue
scout list --queue

# Perform a search
scout query --input "example term"

# Run just the parser
scout parse --host "example.com"

License

MIT