A GAN-inspired adversarial analysis pipeline for any video content — CCTV footage, news broadcasts, home videos, short films, anything. Uses multiple AI models in a Proposer-Critic-Judge architecture to produce detailed forensic reports.

How It Works

Video → ffmpeg → Frames + Audio
                    ↓           ↓
              Proposer      Voxtral
           (vision model)  (audio model)
                    ↓           ↓
               Critic     Translate
           (text model)  (text model)
                    ↓           ↓
                 Judge ←───┘
              (synthesis)
                    ↓
              Final Report (.md)

1. Preprocess: ffmpeg extracts frames at N-second intervals + audio as MP3
2. Proposer: Vision model (Gemma-4-31b) analyzes each frame exhaustively
3. Critic: Text model cross-references frames, flags hallucinations, identifies scene boundaries, builds color timeline, reconstructs narrative
4. Transcribe: Audio model (Voxtral-24b) transcribes speech and notable audio
5. Translate: Text model (DeepSeek-V4-Pro) cleans + translates to result_language
6. Report: Everything assembled into a single markdown report

Requirements

• Python 3.11+ (3.10 works with pip install tomli)
• ffmpeg / ffprobe on PATH
• An OpenAI-compatible API with vision + audio models (tested with Oryxen)

Quick Start

fossil clone https://code.lewman.com/video-forensics/home video-forensics.fossil
mkdir video-forensics && cd video-forensics
fossil open ../video-forensics.fossil

# Copy config and add your API key
cp config.toml.example config.toml
# edit config.toml

# Run
./video-forensics.py cctv_footage.mp4 --context "Front entrance, 2024-03-15"

# Or with a label for the analysis
./video-forensics.py news.mp4 --label "Channel 5 Broadcast" --interval 3

The video-forensics.py script at the repo root adds src/ to sys.path — no installation needed. Zero third-party dependencies on Python 3.11+ — just stdlib.

Configuration

All configuration is via config.toml. Copy config.toml.example to config.toml and adjust.

[api]
base_url = "https://api.oryxen.ai/v1"
api_key = "your-key-here"

[models]
frame = "Oryx_Tertius/gemma-4-31b"
critic = "Oryx_Secundus/glm-5-2"
audio = "Oryx_Secundus/voxtral-small-24b"
translate = "Oryx_Sextus/deepseek-ai/DeepSeek-V4-Pro"

[defaults]
interval = 5
result_language = "English"
critic_chunk_size = 10
timeout_vision = 60
timeout_text = 300
timeout_audio = 180

Config Discovery

The pipeline auto-discovers config.toml in the current working directory. You can also specify a path explicitly:

video-forensics cctv.mp4 --config /path/to/config.toml

Language Handling

• Source language: Auto-detected by the audio model. Optionally override with --language-hint.
• Result language: Configured via result_language in config.toml (default: English).
• Example: A Mandarin speaker analyzing French news footage sets result_language = "Mandarin" — output will be in Mandarin.

Usage

# CCTV footage
./video-forensics.py cctv.mp4 --context "Front entrance, 2024-03-15"

# News broadcast with tighter frame interval
./video-forensics.py news.mp4 --label "Channel 5" --interval 3 --context "Evening broadcast"

# Home video — skip audio (just visual analysis)
./video-forensics.py birthday.mp4 --skip-audio

# Foreign language footage
./video-forensics.py footage.mp4 --language-hint Spanish

# Use cached results (skip re-extraction if already done)
./video-forensics.py cctv.mp4 --skip-frames

# Custom config file
./video-forensics.py cctv.mp4 --config config.toml

If --label is omitted, the video filename is used (e.g. cctv_footage.mp4 → label "cctv_footage").

Output

forensic_<label>/
├── frames/                        # Extracted JPEG frames
│   ├── frame_0001.jpg
│   ├── frame_0002.jpg
│   └── ...
├── audio.mp3                       # Extracted audio
├── proposer_results.json           # Raw frame analyses (cached)
├── critic_report.txt               # Critic analysis (cached)
├── transcription_raw.txt           # Raw audio transcription (cached)
├── transcription_translated.txt    # Cleaned + translated transcription (cached)
└── forensic_report_<label>.md     # Final report

All intermediate results are cached. Re-running the pipeline with the same workdir will skip completed steps.

Model Roles

Role	Default Model	Purpose
Proposer	Oryx_Tertius/gemma-4-31b	Vision: exhaustive frame analysis
Critic	Oryx_Secundus/glm-5-2	Text: cross-reference, find flaws
Audio	Oryx_Secundus/voxtral-small-24b	Audio: speech/audio transcription
Translate	Oryx_Sextus/deepseek-ai/DeepSeek-V4-Pro	Text: clean + translate

Development

# Install dev tooling
pip install pytest ruff mypy

# Run tests
pytest -v

# Lint
ruff check src/ tests/

# Type check
mypy src/video_forensics

CI

SourceHut CI manifest at .build.yml — runs lint, typecheck, and tests on each push.

License

BSD 3-Clause — see LICENSE file.