Verified every claim in the ticket against the current tree. The stale-crate-list item has already been resolved over the past days: Cargo.toml and the AGENTS.md rule-4 inventory now match exactly, dev-only serde_json included. The repo-checkout paragraph no longer assumes any machine-specific path, and a fresh grep of install/ finds only the documented public clone URL, never private paths or hostnames.

Real drift fixed here: the AGENTS.md layout gains the accept_dc.rs harness and the verify-before-done rule now names the gated acceptance command next to the manual openssl loop; README's IMAP subset adds SETMETADATA and notes that Seen persists as the maildir flag suffix; the Milestones list records the live acceptance harness; and the Pending list drops the finished acceptance work in favor of what remains of end-to-end encryption parity. check-in: 24240a8e96 user: andrew tags: trunk

uid_store_cmd tested rest.contains SILENT against the raw command text, so UID STORE 1 plus Flags.Silent Seen emitted an untagged FETCH report while the canonical all-caps spelling stayed quiet. IMAP atoms are case-insensitive (RFC 3501 section 9). The flag-name probes were already uppercased; the SILENT marker now uses the same uppercased copy. No shipped client is affected (Delta Chat core sends the canonical spelling), but mixed-case clients no longer get spurious untagged FETCH traffic. The append round-trip test now also exercises the mixed-case spelling with an exact-equality assert on the quiet reply. check-in: 1a332ebee4 user: andrew tags: trunk

UID STORE plus Seen used to mark a path in a per-session set, so the flag vanished on disconnect and Delta Chat clients re-listed already-read mail as unread. Flag edits now follow the standard maildir encoding: the message moves from new/ to cur/ carrying the :2,S info suffix (atomic rename, crash-safe), and UID FETCH FLAGS and STATUS UNSEEN read the S letter back off the on-disk basename. FlagDeleted stays in-session until EXPUNGE/CLOSE unlinks the file; writing the maildir T letter is deliberately avoided since deletion here is expunge-only.

The rename changes the basename, so the uidlist now keys on the maildir unique name (basename minus the :2, suffix) instead of the full basename, otherwise every Seen STORE would renumber the message and undo the persistent-UID fix. The marking session keeps working across the rename by updating its in-memory path list in place.

New tests cover the store-level rename round trip and idempotency, and the sync-level guarantee that a seen message keeps its uid across a reconnect with the uidlist containing no flag suffix. Live-verified: the Delta Chat acceptance run still passes end to end in about twenty seconds. Also drops the Pending README line whose two halves (persisted Seen here; real IDLE tags in 6b65de1449) have now both shipped. check-in: c2feedac40 user: andrew tags: trunk

New src/accept_dc.rs harness, gated on DC_OXIDE_ACCEPTANCE: boots the real server in-process on ephemeral ports with a temp storage dir (main's listener setup is split into a reusable run function), starts deltachat-rpc-server as a child process, and speaks its JSON-RPC protocol over stdio to create two accounts, run the secure-join handshake both directions, and assert encrypted round-trip delivery. This is the end-to-end check that surfaced the session-local UID renumbering, the swallowed EXISTS announcements, and the IDLE DONE tag defect this week; it completes in about twenty seconds with all three fixed.

New dev-only dependency serde_json for the JSON-RPC framing: the protocol is line-delimited JSON and the std library has no JSON support, and serde is already in the dependency tree so serde_json adds a single small crate. Dev-only, so the shipped binary is unchanged.

Run it with DC_OXIDE_ACCEPTANCE=1 cargo test accept_dc -- --nocapture. check-in: 464c4f037f user: andrew tags: trunk

DONE was answered with the hardcoded label tag_idle, so a client matching on its own command tag never saw the termination reply. RFC 2177 section 3 requires the tagged response to carry the IDLE command's tag. Delta Chat core (async-imap) logs Idle has NewData the instant our EXISTS push lands, sends DONE, and then sits out its full fifteen second IMAP command timeout, reports a failed fetch_idle, and backs off under its rate limiter before reconnecting. Across the eight message hops of the acceptance harness this burned about one hundred and twenty seconds; the harness now completes in about twenty seconds total instead of one hundred and thirty eight.

The tag is captured when IDLE is entered and echoed in the termination reply. The IDLE smoke test now asserts the reply carries the b3 command tag.

Two debug log lines added while attributing this: the imap cmd line now carries the session user, and the IDLE tick logs when it pushes an untagged EXISTS. Neither logs credentials; the user field is the localpart only. check-in: 6b65de1449 user: andrew tags: trunk

Untagged EXISTS was only ever emitted by NOOP and the IDLE tick, by comparing sizes across the tick. Any growth absorbed by the refresh inside UID FETCH, UID STORE, FETCH, STORE, STATUS, or APPEND was silently folded into the session state, so the client was told about new mail only at its next NOOP or keepalive. RFC 3501 section 7.3.1 requires the server to report mailbox size changes whenever it observes them.

Adds an announce_growth helper, gated on a selected mailbox whose count grew since the client was last told, invoked after each of the six command-time refresh sites, plus a unit test that delivers mail mid-session and expects the EXISTS line ahead of the tagged reply. The two APPEND tests that SELECT first now expect the announcement before APPENDUID.

Verified live against the Delta Chat acceptance harness: the server now pushes EXISTS within about a second of every SMTP delivery. check-in: 32b2a01389 user: andrew tags: trunk

Each IMAP connection used to build a private UID map: UIDs 1..N over the files present right then. Delta Chat core expunges consumed messages and reconnects, so a message delivered to the next connection was renumbered UID 1 while the client had UIDNEXT 2 recorded; it concluded nothing new existed and never fetched it. Observed live with deltachat-rpc-server 2.62: the secure-join inviter fetched vg-request as UID 1, expunged it, and the follow-up vg-request-with-auth arrived on a fresh connection again as UID 1, stalling the handshake one hop short of completion.

UIDs now live in a per-user imap-uidlist file beside the maildir (next UID on line 1, then uid-to-basename pairs), reconciled against new/ and cur/ on every refresh: unmapped files get fresh UIDs, vanished files are pruned on an atomic tmp-then-rename rewrite, and next UID only ever increases. With this, the DC acceptance harness completes both SecurejoinProgress directions. Includes a unit test for the expunge-then-reconnect renumbering case. A concurrent-connections race on the listing rewrite is documented in sync_uids; chatmail scale is one or two connections per user. check-in: 6e15baca75 user: andrew tags: trunk

async-imap (Delta Chat core's IMAP client) caches capabilities from the greeting and login response codes and never sends the CAPABILITY command, so our bare-greeting plus CAPABILITY-IMAP4rev1-only login reply made DC core conclude the server does not support IDLE. It fell back to a fake-idle loop that never observed new mail: the acceptance harness delivered a vg-request into the recipient maildir while the client kept reporting an empty INBOX for minutes.

Greeting, LOGIN success, and AUTHENTICATE success now all carry the full list (IMAP4rev1 AUTH=PLAIN IDLE UIDPLUS METADATA), matching Dovecot practice; the CAPABILITY command reply shares the same constant. With this, DC core uses real IDLE and sees EXISTS pushes within a second.

Also logs the IMAP command verb at debug level to make live protocol diagnosis possible without touching credentials. check-in: 485497675f user: andrew tags: trunk

The live-TLS IMAP tests passed on contains-style probes, so a regression that emitted the wrong untagged response (wrong FLAGS shape, wrong UIDNEXT, an extra FETCH line on STORE) would only be caught by a real Delta Chat client. Every server conversation in mail_out.rs tests now asserts the complete response verbatim: greeting, LOGIN, CAPABILITY, the SELECT/EXAMINE untagged block byte for byte, APPEND continuations and APPENDUID codes, FETCH body and bodyless lines, STORE replies, EXPUNGE/CLOSE reports, GETMETADATA pairs, and LOGOUT. Silent STOREs now use the canonical uppercase FLAGS.SILENT spelling DC sends; the SILENT probe happens to be case-sensitive, noted in test comments for a follow-up ticket. IDLE exit stays loosely matched because the fixed tag_idle label is getting its own fix under 2917a4a438.

58 passed, 0 ignored; cargo check, clippy -D warnings, fmt clean. check-in: 0562486335 user: andrew tags: trunk

Two vacuous passes removed. The MX resolver probe was #[ignore]d forever: resolve_mx_hosts_live now runs in the default suite but self-skips with a notice unless DC_OXIDE_LIVE_DNS is set, so the gating is explicit and CI or an operator can run it for real with DC_OXIDE_LIVE_DNS=1 cargo test resolve_mx_hosts_live. SETMETADATA push refusal had no test at all: new live-TLS test imap_setmetadata_refused logs in and pins the exact tagged replies s1/s3 NO SETMETADATA not supported, so a future regression that silently accepts a devicetoken (or refuses with a different shape DC can't parse) fails locally. README 57 to 58 tests and the plus-one-ignored phrasing replaced by the env-var gate. Gates: cargo check, clippy -D warnings, cargo fmt --check clean; 58 passed, 0 ignored. check-in: 63afa8c53f user: andrew tags: trunk

The encrypted-only outbound gate passed any message containing a single 0x02 byte, so plaintext carrying a binary attachment looked like ciphertext, and its From-header term was computed into _from_ok and then discarded. deliver_ok now requires the message to parse with a From header and carry at least one real marker: a Chat-Version, Autocrypt, or Autocrypt-Gossip header (matched at line starts in the header block, case-insensitively), an ASCII-armored OpenPGP block, or an RFC 3156 MIME marker (application/pgp-encrypted as a part type or as the multipart/encrypted protocol param). Chat headers pass by design so secure-join and not-yet-key-exchanged cleartext can still flow, matching production chatmail relays; a From missing or a binary attachment alone no longer unlocks the gate. Tests: BINPGP byte-scan case replaced by RFC 3156 multipart/encrypted, binary-attachment refusal, no-From refusal, plus a marker-matching test pinning case-insensitivity, X- prefixed non-matches, and body-text mentions not counting as headers. mail_in gate comment and relay.rs file header updated to describe the actual policy. README test count 56 to 57. Gates: cargo check, clippy -D warnings, cargo fmt --check clean; 56 passed, 1 ignored. check-in: 9e737c0cb3 user: andrew tags: trunk

hash_password and verify_password_hash in store.rs run ~0.5s of CPU synchronously, and the SMTP AUTH (JIT create plus verify), IMAP LOGIN, IMAP AUTHENTICATE PLAIN, and ACRA invite-redemption paths all awaited them inline on tokio worker threads, so one burst of auth attempts could stall every other session scheduled on the same thread. The expensive calls now go through tokio::task::spawn_blocking at the four async call sites: smtp_auth takes an Arc clone of the store, the IMAP LOGIN and AUTHENTICATE arms wrap verify_password and authenticate_plain, and the ACRA connection wraps its whole sync request handler (account creation on token redemption is the same defect class). The sync store API is unchanged, so main and unit tests are untouched; the existing live-TLS auth roundtrip tests exercise the new wiring. Gates: cargo check, clippy -D warnings, cargo fmt --check all clean; 55 passed, 1 ignored. check-in: cc49c3f3eb user: andrew tags: trunk

The Conn enum with its AsyncRead/AsyncWrite impls and is_loopback_bind were byte-identical copies in mail_in.rs and mail_out.rs, with mx_in.rs already importing Conn from mail_in; they now live once in a new src/common.rs and all three listeners import from there.

smtp_cmd_collect returned the reply code but both EHLO call sites in smtp_send_with_tls bound it to an ignored name, so a 4xx/5xx EHLO reply looked like success and the session marched on to MAIL FROM; both sites now ensure a 2xx class, matching the greeting check. find_ascii_haystack keeps its empty-needle/short-hay guard and lets windows().position() do the scan. The hand-rolled queue JSON (json_str/json_fields/ read_json_string) had no escape-focused tests; json_escapes_roundtrip covers quote, backslash, tab, LF, CR, a sub-0x20 control char round-trip, and the u-escape decode path json_str never emits. README test count 54 to 55; AGENTS.md layout gains src/common.rs. Gates: cargo check, clippy -D warnings, cargo fmt --check clean; 54 passed, 0 failed, 1 ignored (live-DNS probe). check-in: 74882e0dfc user: andrew tags: trunk

unique_name_seed XORed the sub-second filename field with a random u32, so the M-field was neither monotonic nor fixed-width and a lexical filename sort did not match delivery order. The IMAP layer sorts filenames in inbox_files and derives UIDs from delivery order, so two messages delivered within one second could end up with UIDs pointing at each other's files; a UID STORE of \Deleted plus EXPUNGE then unlinks the wrong message (caught by expunge_removes_deleted_messages once the test suite could run). The M-field is now the raw subsec_nanos zero-padded to 9 digits; same-nanosecond collisions stay covered by the existing create_new retry. deliver_writes_readable_file now asserts the ordering invariant directly. Full suite green: 53 passed, 0 failed, 1 ignored (live-DNS probe); clippy -D warnings and rustfmt clean. check-in: eb24fa24d8 user: andrew tags: trunk

The tree committed during the rustc outage never ran a compiler; on a working toolchain cargo check failed in two places: reader.by_ref() on a tokio BufReader in the IMAP APPEND literal path (by_ref is a std/futures-io method, not part of tokio AsyncReadExt) and the server TLS config Arc moved into the MX STARTTLS accept call while the recursive session needed it. The literal path now reborrows the reader before .take() and the MX accept clones the Arc. cargo test then exposed a deadlock in append_oversized_literal_refused: the declared {20} literal was 2 bytes longer than the payload, so the wire terminator was consumed into the message and the server blocked forever on the trailing-CRLF read; the declared size is now {18}. Also resolves the 3 pre-existing clippy -D warnings lints (queue-row type alias in relay, two needless borrows) and applies rustfmt across the tree. check/clippy are clean; the suite runs but one test still fails here (expunge ordering), fixed in the next check-in. check-in: 819d22639d user: andrew tags: trunk

Optional HTTPS listener on new flat config key acra_port (default 0 = disabled) answering the empty POST Delta Chat sends after scanning a DCACCOUNT:https://... QR with 200 {"email","password"} for a freshly minted 9-digit account; failures answer 4xx/5xx with {"reason": ...}. Invites are flat counter files under storage_dir/invites (no DB, per rule 3) minted by the new `dc_oxide invite [--uses N]` subcommand, which prints the DCACCOUNT URL and warns when acra_port is 0. Tokens are restricted to URL-safe ASCII so they double as filenames. HTTP/1.1 is hand-rolled (8 KiB/15 s request bound, body ignored) to keep the crate set unchanged. store::random_local_part is ungated from cfg(test) so acra can draw local parts; Store gains invites_dir(). Docs (README, AGENTS.md, toml example, install-kit firewall notes) updated in step. 4 new tests: token parsing, on-disk use countdown, failure-to-JSON mapping, and a full TLS POST asserting the returned credentials log in and replay gets 403 (54 total). Hand-verified only: local rustc is broken (LLVM_23.1 symbol lookup error), cargo cannot run. check-in: 215e72d3cd user: andrew tags: trunk

UID STORE accepted \Deleted but recorded nothing, and EXPUNGE/CLOSE walked the uid map, unlinked nothing, and reported nothing — every account's maildir grew without bound despite DC's delete_server_after.

STORE now tracks \Deleted in-session alongside \Seen (flags_str helper reports both in FETCH/STORE responses), and EXPUNGE/CLOSE unlink the flagged files, drop them from the uid map, and clear the flag sets. EXPUNGE reports removals with untagged EXPUNGE lines in descending sequence order (per RFC 3501 the numbers reference the pre-removal state); CLOSE performs the same removal silently. New test covers STORE/EXPUNGE unlinking and uid stability plus the silent CLOSE path. Hand-verified only: local rustc is broken (LLVM_23.1 symbol lookup error), cargo cannot run. check-in: c838fe0a08 user: andrew tags: trunk

The no-body FETCH branch emitted FLAGS ("") — a quoted string where RFC 3501 requires a parenthesized atom list — and appended RFC822.SIZE after the closing parenthesis of the response. Both are protocol violations any non-DC client would trip over.

Also in this check-in: the item parser treated RFC822.SIZE as a body request ("RFC822" substring match), so a (FLAGS RFC822.SIZE) fetch pulled the full body; the body probe now strips the SIZE token first. The UID STORE untagged response had the same quoted-FLAGS violation and is fixed too.

New test pins the exact response shape for FETCH with and without \Seen and for the non-silent STORE report. Hand-verified only: local rustc is broken (LLVM_23.1 symbol lookup error), cargo cannot run. check-in: 65c45860c7 user: andrew tags: trunk

The declared {n} size was allocated verbatim before reading the literal, so any authenticated account could force a multi-GB allocation per APPEND command and exhaust server memory. The literal is now refused with a tagged NO before the continuation request when it exceeds the same max_message_size limit SMTP enforces, and accepted literals are streamed with take() reads instead of an upfront allocation. New test covers the refusal path (no continuation, no delivery, session stays usable) and a small successful APPEND. check-in: b4a43228a6 user: andrew tags: trunk

The drain loop built Mail FROM out of the recipient's local part, so every relayed message claimed a sender that never submitted it: remote SPF checks failed on a made-up address and bounces went nowhere. The submission listener now records the accepted envelope MAIL FROM in each queue row and the drain uses it verbatim. Queue rows written before this change have no sender field; those fall back to the old synthesized envelope with a warning instead of being dropped. The test mock captures evelope commands, with new coverage for the recorded-sender path and the legacy-row fallback. check-in: c8c1dcdf7e user: andrew tags: trunk

MAIL FROM accepted any address after AUTH, so one credentialed user could send as any other local user or as an arbitrary third-party domain: SPF and DMARC would fail for the recipient, or worse pass on a forged local sender inside a one-domain trust bubble. The submission listener now requires the envelope sender to be exactly the authenticated account's address; anything else is a permanent 550 5.7.1 and the transaction is reset. Test drive covers a foreign local part, a foreign domain, and the matching account address. check-in: f408ccb3ba user: andrew tags: trunk

The comment claimed a 10 MiB default while the code and example TOML have defaulted to 30 MiB; also notes that the limit is advertised to MX peers. Verified while closing 8b86767bfb: EHLO SIZE, GETMETADATA maxsmtprecipients, and the 30 MiB default were all already in place. check-in: 473c357b7d user: andrew tags: trunk

A chatmail host must receive server-to-server mail, but the only SMTP listener was the TLS-mandatory submission port. Port 25 mail from other servers had nowhere to land, so the domain silently dropped inbound mail. The new mx_in listener accepts plaintext with RFC 3207 STARTTLS (port 25 peers cannot be required to hold certs, a documented exception to the TLS-mandatory rule), refuses AUTH and JIT account creation (existing local accounts only), answers unknown users and foreign domains with one uniform 550 (no address enumeration), pre-rejects oversized messages via the RFC 1870 SIZE parameter, and never relays. Four listener tests drive plaintext, STARTTLS, rejection, and SIZE parsing roundtrips. README, example TOML, and AGENTS.md document the policy. check-in: 3cd74d939a user: andrew tags: trunk

An authenticated user handing a cleartext message to a non-local recipient violates the chatmail encrypted-only policy, yet the submission listener queued it and the relay sent it. DATA now returns a permanent 550 5.7.1 when any non-local recipient would receive an unencrypted body, so the sender sees a hard failure instead of silent data loss; the queue drain also drops any such row at send time (belt: rows queued before this gate existed, or enqueued by a bypass) instead of retrying the send forever. check-in: 452896d8db user: andrew tags: trunk

smtp_send split the raw message on LF and dropped the CR case's missing LF without re-adding it, so a well-formed CRLF source went on the wire with bare CR terminators: headers and body lines ran together for any remote server that honors RFC 5321's CRLF requirement, and a trailing terminator could be doubled. Segments now get their LF back when the source carried CRLF, a full CRLF is synthesized for bare-LF sources, and the final empty segment no longer earns a terminator. Two mock-sink tests assert the exact wire bytes for both CRLF and LF-only sources. check-in: 71a0e4660e user: andrew tags: trunk

The http-01 challenge needs a local web server on port 80: acme-client only writes the token into challengedir, and dc_oxide listens on 25/465/993 only and speaks no HTTP, so first issuance failed on a fresh install. certs.md also claimed OpenBSD renews acme-client domains via a default cron job - that entry does not exist; the @reboot restart line never picked up renewals on a long-running box; and the example put the authority-level contact option inside the domain block. certs.md now configures and starts httpd(8) with the acme-client(1) location block (default /var/www/acme challengedir) and adds the renewal cron "acme-client <domain> && rcctl restart dc_oxide", which restarts the daemon only when the cert actually changed (acme-client exits 2 when still valid). README step 8 pointer updated accordingly.

Add install/linux/README.md, install/freebsd/README.md, install/netbsd/README.md as numbered paste-ready steps matching the OpenBSD README style: regular user with sudo/doas, fossil clone from code.lewman.com, build on target, generic dc.example.com hostname, verify step. Rewrite install/README.md as an index pointing at each per-OS README. check-in: 8aa7239c2c user: andrew tags: trunk

Replace the long commented command-line example with 4 paste-ready steps: challenge dir, acme-client.conf block via tee heredoc, one-shot issuance, renewal note. Generic dc.example.com domain instead of real FQDN; doas everywhere per non-root install flow. check-in: e10854fbc2 user: andrew tags: trunk

Remove internal IP from install/openbsd/README.md, fossil-only workflow (fossil clone/open, no git), fixed repo URL code.lewman.com/dc_oxide, and run as regular user with doas for all privileged steps instead of assuming a root shell. check-in: f22a8f066c user: andrew tags: trunk

rc.dc_oxide: shebang to /bin/ksh (rc.subr requires ksh), start dc_oxide under daemon(8) supervision with flags overridable via rc.conf.local, rc_reload=NO (no SIGHUP support), proper rc_stop that kills the child. README: numbered copy-paste steps 1-12, install(1) commands with modes and ownership inline, netstat instead of Linux ss. check-in: 66691a7cdc user: andrew tags: trunk