A pure Rust LLM trained on a Publii CMS blog, using Candle (Hugging Face's Rust ML framework).

Works with any Publii db.sqlite — not just blog.lewman.com. Point it at your own blog database and train a model on your writing.

Quickstart

# Prepare: extract text, clean, train BPE tokenizer
cargo run -- prepare --db db.sqlite --output-dir data --vocab-size 32768

# Download Gutenberg corpus for pre-training
cargo run -- prepare-corpus --output-dir data

# Pre-train (learns English from Gutenberg books)
cargo run --features cuda --release -- pretrain --data-dir data --artifact-dir pretrain_artifacts

# Fine-tune (learns your voice from blog posts)
cargo run --features cuda --release -- finetune \
  --pretrain-dir pretrain_artifacts --data-dir data --artifact-dir artifacts

# Generate text
cargo run --features cuda --release -- generate --prompt "Hello world" --artifact-dir artifacts --data-dir data

Architecture

• blogllm-core — reads db.sqlite, strips HTML, cleans markdown artifacts (links, images, URLs), trains a BPE tokenizer, and implements TF-IDF RAG retrieval
• blogllm-model — custom causal transformer (GPT-style) with weight tying, manual layer normalization, and manual AdamW training loop. Supports checkpoint save/load for two-stage training.
• blogllm (CLI) — prepare, pretrain, finetune, generate, rag-generate subcommands with full hyperparameter control

Two-Stage Training

Why two stages?

A typical blog has ~275K tokens — not enough to learn English grammar from scratch. Pre-training on ~100 Gutenberg books (~10M tokens) teaches the model English. Fine-tuning on the blog adapts it to your voice.

# Stage 1: Pre-train on Gutenberg corpus (~8M tokens)
cargo run --features cuda --release -- pretrain \
  --data-dir data --artifact-dir pretrain_artifacts \
  --epochs 50 --lr 0.001

# Stage 2: Fine-tune on your blog (~275K tokens)
cargo run --features cuda --release -- finetune \
  --pretrain-dir pretrain_artifacts --data-dir data --artifact-dir artifacts \
  --epochs 100 --lr 0.00001

Backend Selection

We use Candle, Hugging Face's Rust ML framework. Candle supports multiple GPU compute backends. BlogLLM exposes them as Cargo features:

Feature	Hardware	Status	Compile requirement
(none) / cpu	Any CPU	Stable	Pure Rust, no external deps
cuda	NVIDIA (GTX, RTX, A100, H100…)	Stable	nvcc in $PATH
metal	Apple Silicon (M1–M4)	Stable	Xcode + macOS 10.15+
rocm	AMD (RX 7000, Instinct MI…)	Pending upstream	ROCm PR #3424 not merged yet
wgpu	Any GPU via Vulkan/WebGPU	Pending upstream	wgpu PR #3379 not merged yet

# CPU (default, no GPU needed)
cargo run -- pretrain ...

# NVIDIA CUDA
#   nvcc must be in $PATH (e.g. /usr/local/cuda/bin/nvcc)
cargo run --features cuda --release -- pretrain ...

# Apple Metal (macOS)
cargo run --features metal --release -- pretrain ...

Runtime device override (--device)

Even when compiled with a GPU feature, you can force a specific device at runtime:

# Force CPU even though binary was compiled with --features cuda
cargo run --features cuda --release -- generate --device cpu ...

# Force CUDA (same as default when compiled with cuda feature)
cargo run --features cuda --release -- generate --device cuda ...

# Force Metal (same as default when compiled with metal feature)
cargo run --features metal --release -- generate --device metal ...

Auto-detect order (when --device is omitted): CUDA → Metal → CPU.

Requirements for CUDA:

• NVIDIA GPU with CUDA support
• nvcc in $PATH (e.g. /usr/local/cuda-13.2/bin/nvcc)
• On first build, cudarc compiles CUDA kernels (~1-2 minutes)

Running on GPU cloud instances

Auto-detection script — detects NVIDIA / AMD / CPU and picks the right backend:

chmod +x run_on_server.sh
./run_on_server.sh

Makefile

make all                   # prepare + pretrain + finetune (cpu default)
make all FEATURES=cuda     # full pipeline with CUDA GPU

Variables

Variable	Default	Description
FEATURES	cpu	Backend feature (cuda, metal, or cpu)
DEVICE	(auto-detect)	Override runtime device: cpu, cuda, metal
RELEASE	1	Build in release mode (0 for debug)
DB	db.sqlite	Path to Publii database
DATA_DIR	data	Output directory for prepared data
ARTIFACT_DIR	artifacts	Fine-tuned model output
PRETRAIN_DIR	pretrain_artifacts	Pre-train checkpoint output
PROMPT	(required for generate)	Text prompt for generation
MAX_TOKENS	200	Max new tokens for generation
TEMPERATURE	0.8	Sampling temperature
TOP_P	0.9	Top-p (nucleus) sampling

CLI Reference

prepare

Extract text from Publii database and train BPE tokenizer.

cargo run -- prepare --db db.sqlite --output-dir data --vocab-size 32768

Flag	Default	Description
--db	db.sqlite	Path to Publii database
--output-dir	data	Where to save texts.json and tokenizer.json
--vocab-size	32768	BPE vocabulary size

prepare-corpus

Download Gutenberg books for pre-training.

cargo run -- prepare-corpus --output-dir data

pretrain

Train on Gutenberg corpus from scratch.

cargo run --features cuda --release -- pretrain \
  --data-dir data --artifact-dir pretrain_artifacts

Flag	Default	Description
--data-dir	data	Directory containing corpus
--artifact-dir	pretrain_artifacts	Checkpoints saved here
--d-model	768	Transformer hidden dimension
--n-heads	12	Attention heads (must divide d-model)
--n-layers	12	Transformer layers
--d-ff	3072	Feed-forward hidden dim (usually ~4× d-model)
--block-size	256	Max sequence length in tokens
--stride	128	Sliding window stride
--min-seq-len	128	Minimum sequence length to keep
--batch-size	16	Training batch size
--epochs	50	Training epochs
--lr	0.001	Peak learning rate
--weight-decay	0.01	AdamW weight decay
--patience	10	Early stopping patience

finetune

Load pretrain checkpoint and fine-tune on blog.

cargo run --features cuda --release -- finetune \
  --pretrain-dir pretrain_artifacts --data-dir data --artifact-dir artifacts

Same flags as pretrain, plus:

Flag	Default	Description
--pretrain-dir	pretrain_artifacts	Directory with pretrain checkpoint

Fine-tuning defaults differ:

• --lr 0.00001 (1/100th of pretrain — prevents catastrophic forgetting)
• --epochs 100 (more epochs for small dataset)
• --patience 15 (more patience for convergence)
• --min-seq-len 256 (longer sequences for blog posts)

generate

cargo run --features cuda --release -- generate \
  --prompt "What is the future of computing?" \
  --artifact-dir artifacts --data-dir data \
  --max-tokens 200 --temperature 0.8 --top-p 0.9

Flag	Default	Description
--prompt	(required)	Text prompt
--artifact-dir	artifacts	Directory with saved model
--data-dir	data	Directory with tokenizer.json
--max-tokens	200	Maximum new tokens to generate
--temperature	0.8	Sampling temperature
--top-p	0.9	Top-p nucleus sampling (0 = disabled)

rag-generate

Answer a question using TF-IDF retrieval + generation.

cargo run --features cuda --release -- rag-generate \
  --question "What is your opinion on Linux?" \
  --artifact-dir artifacts --data-dir data

Example Workflow

# One-command pipeline (prepare + pretrain + finetune)
make all FEATURES=cuda

# Or step-by-step:

# Prepare: extract text, clean HTML, train BPE tokenizer
cargo run -- prepare --db db.sqlite --output-dir data --vocab-size 32768

# Download Gutenberg corpus for pre-training
cargo run -- prepare-corpus --output-dir data

# Pre-train on Gutenberg (learns English)
cargo run --features cuda --release -- pretrain \
  --data-dir data --artifact-dir pretrain_artifacts

# Fine-tune on blog (learns your voice)
cargo run --features cuda --release -- finetune \
  --pretrain-dir pretrain_artifacts --data-dir data --artifact-dir artifacts

# Generate text
cargo run --features cuda --release -- generate \
  --prompt "The future of computing is" \
  --artifact-dir artifacts --data-dir data

# Answer a question about your blog
cargo run --features cuda --release -- rag-generate \
  --question "What do you think about AMD?" \
  --artifact-dir artifacts --data-dir data

Performance

Hardware: NVIDIA H100 80GB (SXM5) Model: d_model=768, n_heads=12, n_layers=12, d_ff=3072 Dataset: Gutenberg corpus (~8.3M tokens)

Metric	Burn 0.21 (cubecl-cuda)	Candle (raw CUDA)
Step time	~1,200ms	~46ms (26x faster)
GPU utilization	0% (idle)	99% (saturated)
Power draw	71W	340W (max TDP)
Stability	Crashes at batch>16	Stable at batch=256

Burn's cubecl-cuda 0.10 backend had fundamental host-side overhead (~1.2s of pure CPU work between kernel launches). Candle's raw cudarc kernels saturate the GPU with no framework overhead.

Smaller / Faster Training

For quick iteration (CPU or small GPU):

cargo run --features cuda --release -- pretrain \
  --data-dir data --artifact-dir pretrain_artifacts \
  --d-model 384 --n-heads 8 --n-layers 8 --d-ff 1532 \
  --block-size 256 --stride 128 --min-seq-len 128 \
  --batch-size 32 --epochs 50 --lr 0.001 --patience 10

Flag	Default	Fast preset	Effect
--d-model	768	384	Cuts model width by 2× (~6× fewer params)
--n-layers	12	8	Fewer layers = faster forward/backward
--block-size	256	128	Shorter attention matrices = less VRAM
--batch-size	16	32	Fewer iterations per epoch
--epochs	50	50	Early stopping catches plateau

License

BSD 3-Clause. Blog content is CC-BY-4.0.