docs: add LLVM crypto-shim ABI design note (future work)
ober
f374d35f046eaa00800cd39817f95af189dc5837
--- a/docs/index.md +++ b/docs/index.md @@ -39,6 +39,7 @@ New here? Start with [quickstart.md](quickstart.md), then [tutorial.md](tutorial - [typing.md](typing.md) — gradual typing with runtime assertions, zero production overhead - [typed-jerboa.md](typed-jerboa.md) — Typed Jerboa design (vision and non-goals) - [llvmir-backend.md](llvmir-backend.md) — experimental Typed Jerboa → textual LLVM IR backend (scalar subset) +- [llvmir-crypto-shim.md](llvmir-crypto-shim.md) — design note: C-ABI shim to compile the crypto kernels via LLVM (future work) - [jerboa-code-typed-static-plan.md](jerboa-code-typed-static-plan.md) — plan for using Typed Jerboa in the `jerboa-code` project - [contracts.md](contracts.md) — runtime contracts system (engineering plan) - [gerbil-contracts.md](gerbil-contracts.md) — Gerbil's contract system, as a reference for Jerboa's bridge --- a/docs/llvmir-backend.md +++ b/docs/llvmir-backend.md @@ -225,7 +225,9 @@ AEAD half) are **excluded by design**: their `sha256`/`hmac`/`hkdf`/ which the typed sources themselves say must never be reimplemented. They stay on the Rust backend; the LLVM path is for the pure compute kernels. `Option`/variant/`match` *are* lowered (so the crypto halves could compile -against a future C-ABI crypto shim), but no such shim is wired here. +against a future C-ABI crypto shim), but no such shim is wired here. The shim's +proposed ABI is sketched in [llvmir-crypto-shim.md](llvmir-crypto-shim.md) +(design only, not implemented). ## Files new file mode 100644 --- /dev/null +++ b/docs/llvmir-crypto-shim.md @@ -0,0 +1,159 @@ +# LLVM Crypto-Shim ABI (Future Work, Design Only) + +Status: **not implemented.** This is a design note for letting the +crypto-dependent Typed Jerboa kernels (`crypto`, `ecies`, and `psk`'s +key-derivation / AEAD half) compile through the +[LLVM IR backend](llvmir-backend.md) by lowering each crypto primitive to a +`call` into an external C-ABI function, then linking a small shim that forwards +to vetted RustCrypto crates. + +The pure compute kernels already compile with no shim (see +[llvmir-backend.md](llvmir-backend.md)). The crypto primitives are +deliberately *not* lowered to LLVM instructions — SHA-256, HMAC, HKDF, +AES-256-GCM, and X25519 must never be reimplemented by hand, in any language. +A shim keeps the implementation in the audited crates while still letting the +typed kernel bodies (the framing, KDF composition, nonce handling) lower to +native code. + +## Why a shim is the right shape + +- The kernels are thin: each crypto-prim call is one operation; the typed body + around it (`ecies-seal`, `psk-transport-open`, `derive-ecies-key`) is the part + worth compiling. The shim is the FFI seam those bodies already assume. +- The Rust backend already exposes the same primitives via its `jt_*` C ABI; a + shim can reuse that exact code path rather than inventing a second one. +- It composes with the boxing the backend already does for `Option`/variants, + so `aes-256-gcm-open : (Option Bytes)` needs no new representation. + +## Value representation (already what the backend emits) + +```text +Bytes / String { ptr, i64 } by-value fat pointer (data, length) +Nat i64 +(Option Bytes) { i1, { ptr, i64 } } tag (1=Some/0=None) + payload buffer +``` + +Inputs are **borrowed**: the shim reads `ptr[0..len]` and must not free or +retain them past the call. Results are **owned by the result buffer**: the shim +allocates them with the same allocator the backend's `bytes-build` uses +(`malloc`) and the caller never frees them — analysis kernels are short-lived +and the backend has no GC yet (see llvmir-backend.md "buffers are never freed"). +A real long-running embedding would need an ownership story here first. + +## Symbol naming + +Mirror the function-symbol scheme, namespaced under `jt_crypto_`: + +```text +@jt_crypto_sha256 +@jt_crypto_hmac_sha256 +@jt_crypto_hkdf_sha256 +@jt_crypto_x25519_dh +@jt_crypto_x25519_base +@jt_crypto_aes_256_gcm_seal +@jt_crypto_aes_256_gcm_open +``` + +The backend would emit a `declare` for each primitive it uses (once per module, +via the existing intrinsic-dedup path) and lower a `crypto-prim` call to a +`call` of the matching symbol. + +## Primitive signatures + +Written in C with the buffer struct `typedef struct { const uint8_t *ptr; +uint64_t len; } Buf;` and `typedef struct { uint8_t tag; Buf buf; } OptBuf;` +(the `{ i1, {ptr,i64} }` the backend emits for `(Option Bytes)`; `i1` widens to +one byte across the C ABI). + +```c +/* sha256(data) : Bytes -> Bytes (32-byte digest) */ +Buf jt_crypto_sha256(Buf data); + +/* hmac-sha256(key, msg) : (Bytes Bytes) -> Bytes (32-byte tag) */ +Buf jt_crypto_hmac_sha256(Buf key, Buf msg); + +/* hkdf-sha256(salt, ikm, info, length) : (Bytes Bytes Bytes Nat) -> Bytes + empty salt reproduces RFC 5869 "salt not provided" (None) */ +Buf jt_crypto_hkdf_sha256(Buf salt, Buf ikm, Buf info, uint64_t length); + +/* x25519-dh(scalar, point) : (Bytes Bytes) -> Bytes (RFC 7748 ECDH) */ +Buf jt_crypto_x25519_dh(Buf scalar, Buf point); + +/* x25519-base(scalar) : Bytes -> Bytes (scalar * basepoint) */ +Buf jt_crypto_x25519_base(Buf scalar); + +/* aes-256-gcm-seal(key, nonce, plaintext, aad) : (Bytes*4) -> Bytes + returns ciphertext with the 16-byte GCM tag appended */ +Buf jt_crypto_aes_256_gcm_seal(Buf key, Buf nonce, Buf plaintext, Buf aad); + +/* aes-256-gcm-open(key, nonce, ciphertext, aad) : (Bytes*4) -> (Option Bytes) + tag 0 (None) on authentication failure */ +OptBuf jt_crypto_aes_256_gcm_open(Buf key, Buf nonce, Buf ciphertext, Buf aad); +``` + +These match the seven `crypto-prim` operators the Rust emitter already lowers +(`lib/jerboa/typed/rust.ss`, `emit-crypto-prim`) and their typed arities; the +shape parity means a shim can wrap the same crate calls the Rust backend emits. + +## Also needed: pure buffer ops (no shim required) + +`psk`'s `compute-proof`/`psk-transport-*` and `ecies` also use buffer +primitives that are **pure and lowerable directly** — they do *not* belong in +the crypto shim and should be added as native LLVM lowerings first: + +```text +bytevector-append a b -> malloc(len a + len b) + two memcpys +bytevector-copy bv s e -> malloc(e - s) + one memcpy of bv[s..e] +make-bytevector n fill -> malloc(n) + a fill loop (or memset) +integer->le-bytes n -> malloc(8) + store the i64 little-endian +``` + +Lowering these natively (a small follow-up to the existing `bytes-build` work) +removes them as blockers, leaving only the seven genuine crypto primitives for +the shim. + +## Reference shim sketch + +A shim is one Rust `staticlib`/`cdylib` (or C file) exporting the symbols +above. Each result `Buf` is produced by leaking a `Vec<u8>` so its pointer +stays valid (the kernel never frees it): + +```rust +#[repr(C)] pub struct Buf { ptr: *const u8, len: u64 } +fn ret(mut v: Vec<u8>) -> Buf { // leak: caller never frees + v.shrink_to_fit(); + let (ptr, len) = (v.as_ptr(), v.len() as u64); + std::mem::forget(v); + Buf { ptr, len } +} +unsafe fn slice<'a>(b: Buf) -> &'a [u8] { // borrow the input + if b.ptr.is_null() { &[] } else { std::slice::from_raw_parts(b.ptr, b.len as usize) } +} + +#[unsafe(no_mangle)] +pub extern "C" fn jt_crypto_sha256(data: Buf) -> Buf { + use sha2::Digest; + ret(sha2::Sha256::digest(unsafe { slice(data) }).to_vec()) +} +// ... the other six, wrapping hmac / hkdf / aes-gcm / x25519-dalek ... +``` + +Link it alongside the generated object: + +```bash +clang jsecmon.o crypto_shim.a -o jsecmon-llvmir # or link the cdylib +``` + +## Backend changes this would require + +1. Lower the four pure buffer ops above (native; no shim). +2. In `lower-call`, replace the `crypto-prim` rejection with: `declare` the + matching `@jt_crypto_*` symbol (once) and emit a `call`, using the typed + arity for argument order and the call's result type (`Bytes` or + `(Option Bytes)`) for the return. +3. A `make` target / CLI flag that links the shim for the crypto modules, plus + a parity harness asserting the AEAD/KDF outputs against the Rust backend's + crate (the same oracle the pure harness already uses). + +Until those land, `crypto`/`ecies`/`psk` stay on the Rust backend, and the LLVM +path covers the pure compute kernels only.