docs/data: document Option/variants/match in the LLVM backend
ober
ad4844ed4d829824ea36728fb9f45780209f5d8a
--- a/data/changelog.sexp +++ b/data/changelog.sexp @@ -2,7 +2,19 @@ . "Machine-readable changelog of Jerboa API drift. Consumers (LLM tooling, lints, jerboa_verify) use this to invalidate stale recommendations and to suggest migrations when a symbol is renamed or relocated.") ("entries" - (("added" "typed-modules->llvmir-string" "typed-library-forms->llvmir-string") + (("added") + ("date" . "2026-06-03") + ("modules_added") + ("moved") + ("notes" + . + "LLVM IR backend gained Option, variants, and match. (Option T) lowers to a by-value tagged struct { i1, T } (option-some/option-none). Variants lower to a tagged boxed { i32 tag, ptr box }: the constructor sizes the case's field struct (getelementptr-null + ptrtoint), mallocs the box, and stores each field (null for a fieldless case; never freed; recursive variants supported). match lowers to a switch on the tag, each arm loading its bound fields from the box and joining results through a phi, with an unreachable default when exhaustive or the else arm otherwise. Demonstrated by carving jerboa-secmon's pure psk-hex kernels (constant-time compare + lowercase hex codec) out of psk into their own module, which now compiles whole-program to native code (78 functions) and is asserted against the Rust backend. The crypto halves (sha256/hkdf/aes-gcm/x25519) remain on the RustCrypto FFI path. See docs/llvmir-backend.md.") + ("removed") + ("renamed") + ("tier_changes") + ("tools_added") + ("version" . #f)) + (("added" "typed-modules->llvmir-string" "typed-library-forms->llvmir-string") ("date" . "2026-06-03") ("modules_added") ("moved") --- a/data/cookbooks.sexp +++ b/data/cookbooks.sexp @@ -4954,4 +4954,15 @@ "bytes" "record") ("title" . - "Whole-program Typed Jerboa -> LLVM IR with String/Bytes, records, cross-module calls"))) + "Whole-program Typed Jerboa -> LLVM IR with String/Bytes, records, cross-module calls")) + (("code" + . + ";; Option -> { i1 tag, T payload }; variant -> { i32 tag, ptr box }; match ->\n;; switch on the tag with per-case field loads joined by a phi.\n(import (jerboa typed parser) (jerboa typed llvmir))\n\n(def form\n '(typed-library (demo tok)\n (export size kind)\n (variant Token\n (Some (value : Nat))\n (Pair (a : Nat) (b : Nat))\n (Empty))\n (def (size (t : Token)) : Nat\n (match t\n ((Some value) value)\n ((Pair a b) (+ a b))\n ((Empty) 0)))\n ;; match with an else arm; Option as a value type\n (def (kind (t : Token)) : (Option Nat)\n (match t\n ((Some value) (option-some value))\n (else (option-none Nat))))))\n\n(display (typed-library-form->llvmir-string form))") ("id" . "typed-llvmir-option-variant-match") + ("imports" "(jerboa typed parser)" "(jerboa typed llvmir)") + ("notes" + . + "Variants box their payload: the ctor sizes the case's field struct via getelementptr-null + ptrtoint, mallocs it, and stores each field (null box for a fieldless case); never freed (like bytes-build), and recursive variants work because the box is a pointer. match lowers to `switch i32 %tag`, each arm GEP+loads its bound fields from the box (wildcards skip the load) and brrs to a join phi; an exhaustive match's default is `unreachable`, an `else` arm fills the default. Option<T> = by-value { i1, T } (tag 1=Some/0=None, payload undef when None). variant-pred / record-pred lower to constant i1 true (the value already has that type post-check). Case tag = declared case index. NOT yet: Result, resources, closures. Real use: jerboa-secmon `make llvmir-bin`. See docs/llvmir-backend.md.") + ("tags" "typed" "llvmir" "llvm" "variant" "match" "option") + ("title" + . + "Lower Typed Jerboa Option, variants, and match to LLVM IR"))) --- a/docs/llvmir-backend.md +++ b/docs/llvmir-backend.md @@ -77,13 +77,16 @@ Exactly this; everything else is rejected at lowering with a `R?` predicates; records usable as `for/fold` accumulators. - `(for/fold ((acc init)) ((i (in-range [start] end))) body)` with a scalar, `Float`, or record accumulator. +- `(Option T)`: `option-some`/`option-none`. +- `variant` declarations: case constructors, `V?` predicates, and `match` + (exhaustive or with an `else` arm). - A generated `@main` for smoke tests (below). -Not supported (this pass): variants, `match`, `Option`/`Result`, resources, -closures, GC, the Chez FFI boundary, crypto primitives (`sha256`, -`hmac-sha256`, `hkdf-sha256`, `aes-256-gcm-*`, `x25519-*` — these stay on the -RustCrypto FFI path; reimplementing them in LLVM is out of scope and a security -anti-pattern), and LLVM C API bindings. Modules using those forms are rejected. +Not supported (this pass): `Result`, resources, closures, GC, the Chez FFI +boundary, crypto primitives (`sha256`, `hmac-sha256`, `hkdf-sha256`, +`aes-256-gcm-*`, `x25519-*` — these stay on the RustCrypto FFI path; +reimplementing them in LLVM is out of scope and a security anti-pattern), and +LLVM C API bindings. Modules using those forms are rejected. ## Representation @@ -95,9 +98,19 @@ Float -> double fadd/fsub/fmul/fdiv, ordered fcmp (oeq/olt/...) String -> { ptr, i64 } by-value fat pointer (data, length); UTF-8 bytes Bytes -> { ptr, i64 } identical rep, so string->utf8/utf8->string are no-ops record -> { f0ty, f1ty, ... } by-value struct; ctor=insertvalue, field=extractvalue +Option -> { i1, T } tag (1=Some/0=None) + payload (undef when None) +variant-> { i32, ptr } tag = case index, ptr = malloc'd box of that case's fields Unit -> void (procedure returns only) ``` +Variants box their payload: the constructor sizes the case's field struct with +the `getelementptr null` / `ptrtoint` idiom, `malloc`s it, and stores each +field; the box is null for a fieldless case and is never freed (like +`bytes-build`). This handles cases with different field sets uniformly and +supports recursive variants. `match` lowers to a `switch` on the tag — each arm +loads its bound fields from the box and the results join through a `phi`; an +exhaustive match's `default` is `unreachable`. + Signedness lives in operations, not types. Mixed int/float operands in one operation are rejected at lowering rather than emitted as malformed IR. No `nsw`/`nuw` flags are emitted and shift counts are not masked: Typed Jerboa @@ -187,28 +200,32 @@ directly with `rustc`, no cargo), and both must equal the expected constant. ## Worked Example: secmon Detection Kernels -`~/mine/jerboa-secmon` compiles its six **pure** typed detection kernels — -`analytics`, `strbytes`, `triage`, `lolbin`, `obfuscate`, `dga` — straight to -one native object with no Rust crate in the path: +`~/mine/jerboa-secmon` compiles its **pure** typed detection kernels — +`analytics`, `strbytes`, `triage`, `lolbin`, `obfuscate`, `dga`, and `psk-hex` +(the crypto-free constant-time-compare + hex codec split out of `psk`) — +straight to one native object with no Rust crate in the path: ```bash cd ~/mine/jerboa-secmon make llvmir-bin # whole-program .ll -> verify -> -O2 -> llc -> cc harness -> run ``` -68 functions verify and survive `-O2`; a C harness (`llvmir/harness.c`) calls +78 functions verify and survive `-O2`; a C harness (`llvmir/harness.c`) calls the headline kernels through the `{ ptr, i64 }` buffer ABI and asserts each result against the Rust backend's value (host-risk scoring, DGA entropy/ consonant-run/score, LOLBin command scoring, triage predicates, obfuscation -round-trip). This exercises every expanded feature: String/Bytes, records -(`dga`'s `CrState` fold), `Float`/`log2` entropy, `bytes-build`, and -cross-module calls (`lolbin`/`triage`/`dga` import `strbytes`). - -The crypto trio (`crypto`, `ecies`, `psk`) is **excluded by design**: its -`sha256`/`hmac`/`hkdf`/`aes-256-gcm`/`x25519` kernels are thin wrappers over -vetted RustCrypto crates, which the typed sources themselves say must never be -reimplemented. They stay on the Rust backend; the LLVM path is for the pure -compute kernels. +round-trip, constant-time compare, hex codec). This exercises every expanded +feature: String/Bytes, records (`dga`'s `CrState` fold), `Float`/`log2` +entropy, `bytes-build`, and cross-module calls (`lolbin`/`triage`/`dga` import +`strbytes`). + +The crypto-dependent kernels (`crypto`, `ecies`, and `psk`'s key-derivation / +AEAD half) are **excluded by design**: their `sha256`/`hmac`/`hkdf`/ +`aes-256-gcm`/`x25519` calls are thin wrappers over vetted RustCrypto crates, +which the typed sources themselves say must never be reimplemented. They stay +on the Rust backend; the LLVM path is for the pure compute kernels. +`Option`/variant/`match` *are* lowered (so the crypto halves could compile +against a future C-ABI crypto shim), but no such shim is wired here. ## Files @@ -217,6 +234,6 @@ lib/jerboa/typed/llvmir.ss emitter library (jerboa typed llvmir) support/typed-llvmir.ss CLI: per-module, or --whole-program OUT.ll tests/test-typed-llvmir.ss deterministic emitter unit tests tests/fixtures/typed/llvmir-{basic,if,call,float,bitwise,for-fold,bytes, - record,smoke}.ss + record,option,variant,smoke}.ss build/typed/llvmir/ disposable generated artifacts (.ll/.bc/.o) ```