Add `// SPDX-License-Identifier: BSD-3-Clause` header to all 10 Rust source files and inherit the license from the workspace in each sub-crate Cargo.toml. leaf check-in: f77c251750 user: sourcehut@lewman.us tags: master, trunk

- Replace 19-positional-arg cmd_* wrappers with TrainParams struct - Remove all #[allow(clippy::too_many_arguments)] hack suppressions - Fix pre-existing lints in preprocess.rs and rag.rs - Add pre-commit checklist to AGENTS.md with zero-warnings policy - Explicit rule: no --release on dev machine

Clean build: cargo check + cargo clippy -D warnings + cargo build, all pass with zero warnings, zero errors. check-in: e4dfcceee9 user: sourcehut@lewman.us tags: master, trunk

- Add Tiny CLI subcommand: d_model=128, n_layers=4, n_heads=4, d_ff=512 - Add --lr-min and --warmup-steps to all train commands - Fix stride propagation from CLI into create_batches (was hardcoded) - Add --n-posts flag for training on blog subsets - Rewrite run_on_server.sh for single-stage tiny pipeline: - No Gutenberg corpus download - vocab_size=4096 (was 32768) - Train tiny model only, skip pretrain+finetune two-stage - Test generation at end

Clean build, 0 warnings, 0 errors. check-in: 9655ed6e0f user: sourcehut@lewman.us tags: master, trunk

BlogLLM now supports multiple Candle backends with runtime device selection.

Features added: - : and for runtime backend selection (CUDA -> Metal -> CPU auto-detect) - CLI flag on all training and generation commands - Cargo features: (default), , - auto-detects NVIDIA/AMD/CPU and passes --device - Makefile supports variable for runtime override

ROCm and wgpu are NOT wired yet because Candle 0.10.x stable does not ship them. The plumbing is ready; just uncomment in Cargo.toml and device.rs when Candle merges PR #3424 (ROCm) and PR #3379 (wgpu).

Breaking: Old still works, but / / / from Burn era are removed. See README for migration.

Refs: - https://github.com/huggingface/candle/pull/3424 (ROCm) - https://github.com/huggingface/candle/pull/3379 (wgpu) check-in: c57663fccf user: sourcehut@lewman.us tags: master, trunk

- Remove all Burn/wgpu/ROCm/OpenBLAS references - Document two-stage training (pretrain + finetune) - Update CLI flags: top-p instead of top-k, no grad-accum - Add performance comparison table (Burn vs Candle) - Update AGENTS.md to reflect current architecture and gotchas - Remove duplicate/legacy sections

We no longer use Burn. Candle raw CUDA is 26x faster with 99% GPU util. check-in: 7c9d6c4f57 user: sourcehut@lewman.us tags: master, trunk

Explicitly pass --d-model 768 --n-heads 12 --n-layers 12 --d-ff 3072 for both pretrain and finetune stages. The 128-dim model was too small to hold both Gutenberg English and blog voice, causing overfitting and worse val loss after fine-tuning. check-in: 20a8d2365e user: sourcehut@lewman.us tags: master, trunk

The 128-dim model was overfitting and destroying pre-trained knowledge at 1e-4 on only 275K blog tokens. Lower LR preserves general English learned from Gutenberg while adapting to blog voice. check-in: 558b3b7ab1 user: sourcehut@lewman.us tags: master, trunk

- Replace top_k with top_p throughout CLI (usize -> f64, default 0.9) - Update TrainConfig field names: lr_base/lr_min - Remove unused Tensor import from candle_train.rs - Verified: cargo check + clippy pass cleanly check-in: 886537cb8c user: sourcehut@lewman.us tags: master, trunk

- Replace top-k with top-p (nucleus) sampling in generate() - Use fastrand for lightweight random sampling - Add get_lr() with warmup + cosine decay schedule - Log LR alongside loss during training - Add fastrand dependency to model/Cargo.toml check-in: 64d8d0cea6 user: sourcehut@lewman.us tags: master, trunk

- Make ManualLayerNorm pub to silence private_interface warnings - Remove --grad-accum flag (Burn-only, not used in Candle manual loop) check-in: 3dacb22bea user: sourcehut@lewman.us tags: master, trunk

- Add comprehensive benchmark comparison to AGENTS.md - Create training_stats.txt with detailed before/after metrics - Update run_on_server.sh for cuda backend - Results: 26x faster training, 99% GPU utilization on H100 check-in: d91554937b user: sourcehut@lewman.us tags: master, trunk

Burn's cubecl-cuda had 50% host overhead and allocator crashes. This rewrite uses candle-core/candle-nn directly:

- Custom GPT model with causal attention + weight tying - Manual AdamW training loop (no framework overhead) - Checkpoint save/load via VarMap - Supports both CPU and CUDA backends via feature flags - Updated CLI with device selection logic - Updated run_on_server.sh for new backend check-in: 38941312d9 user: sourcehut@lewman.us tags: master, trunk

- Add bench.rs for isolated benchmark - Fix device type paths for tch feature (incomplete) - Server testing shows cubecl-cuda has 50% host overhead - Server testing shows cubecl-wgpu crashes on allocation - burn-tch blocked by libtorch version mismatch check-in: 44ea0bff5e user: sourcehut@lewman.us tags: master, trunk

Burn's cubecl-cuda backend has extreme per-step host overhead on H100 (~1.2s/iteration, 0% GPU utilization). Switching to burn-tch uses PyTorch's C++ engine which properly saturates the GPU.

- Add 'tch' feature to model and cli Cargo.toml - Wire LibTorch backend into main.rs for both training and inference - Update run_on_server.sh to auto-select 'tch' for NVIDIA GPUs - Update AGENTS.md with backend guidance check-in: 0aa4fbcce9 user: sourcehut@lewman.us tags: master, trunk

- Remove crates/core/src/corpus.rs and ureq dependency - Remove 'prepare-corpus' CLI command (was unreliable) - Add scripts/download_corpus.sh with polite curl delays - Update Makefile to call bash script instead of Rust command - run_on_server.sh already has inline curl downloader, kept as-is check-in: bc4d086e10 user: sourcehut@lewman.us tags: master, trunk

- Cut from 125 to 50 books to avoid rate limits (~5-10M tokens still) - Add 2-second sleep between requests - Add --retry 5 --retry-delay 5 for transient failures - Add descriptive User-Agent header - Fix broken inline comments in bash array check-in: f45bc38ef9 user: sourcehut@lewman.us tags: master, trunk

- Change base URL from aleph.gutenberg.org (cert error) to www.gutenberg.org - Change file path from /ebooks/{id}.txt.utf-8 to /files/{id}/{id}-0.txt - Both changes required for Gutenberg's current mirror layout check-in: 2983dcf485 user: sourcehut@lewman.us tags: master, trunk

- Add Project Gutenberg corpus downloader (prepare-corpus) - Add two-stage training: pretrain on Gutenberg, finetune on blog - Add checkpoint loading for finetune from pretrain artifacts - Add TF-IDF RAG pipeline (rag-build-index, rag-generate) - Implement weight tying: output projection uses transposed embedding weights - Fix: include post titles in training text - Fix: increase html2text width to 9999 (drops artificial line breaks) - Fix: URL filter now catches https:// links - Fix: sliding window off-by-one (use <= to include last window) - Update Makefile with two-stage targets - Rewrite train_loop.jl for two-stage + RAG evaluation - Update AGENTS.md with full documentation check-in: fd2d55091c user: sourcehut@lewman.us tags: master, trunk

Julia backticks treat string interpolation as a single word, so '--features wgpu' was passed as one argument. Use conditional composition instead so each flag is its own command-line argument. check-in: 36ca4aaf40 user: sourcehut@lewman.us tags: master, trunk

run_and_capture now returns (output, exitcode) so cargo compilation errors or CLI argument failures are logged and surfaced properly instead of throwing a Julia stacktrace. check-in: 434ea97522 user: sourcehut@lewman.us tags: master, trunk

The auto-tune loop now exits cleanly when val_ppl < 100. Configurable via SUCCESS_PPL env var (default 100.0). check-in: 8013cfcbed user: sourcehut@lewman.us tags: master, trunk

scripts/train_loop.jl runs prepare -> train -> generate, logs structured metrics to training_stats.csv, and automatically adjusts LR / dropout / model capacity for the next run based on overfit detection.

Features: - Pure stdlib (no dependencies to install) - Env + CLI overrides for any hyperparameter - Auto-detects overfit vs convergence - Appends both CSV (structured) and text (narrative) logs check-in: 62084691ae user: sourcehut@lewman.us tags: master, trunk

scripts/train_loop.fish is a pure fish-shell automation script that runs prepare -> train -> generate and appends results to training_stats.txt. All hyperparameters set via env vars. check-in: 814a4be96e user: sourcehut@lewman.us tags: master, trunk

scripts/train_loop.sh runs prepare → train → generate and automatially appends a formatted summary to training_stats.txt. All hyperparameters are configurable via environment variables. check-in: 8b6aac5a9a user: sourcehut@lewman.us tags: master, trunk

Prevents dimension-mismatch panics when switching model sizes between training runs (e.g. 768-dim → 256-dim). check-in: fbabf8ab85 user: sourcehut@lewman.us tags: master, trunk

- preprocess: detect and extract Publii block-editor JSON before HTML stripping - preprocess: add is_quality_text() filter (min 30 words, drop placeholders/logs) - cli: use new clean_text + is_quality_text pipeline in prepare step - train: add .shuffle() to train and validation dataloaders check-in: ae1a5165e9 user: sourcehut@lewman.us tags: master, trunk