security: harden regex facade ffi outputs
ober
2d02ca9a30ead0d5f5c2e50c44cfe84bdbad0365
--- a/docs/ffi-audit.md +++ b/docs/ffi-audit.md @@ -23,19 +23,19 @@ Latest summary from `tools/ffi-audit-report.ss`: (ffi-audit-report (schema "jerboa.ffi-audit/1") (scheme - (file-count 1187) + (file-count 1188) (site-count 782) (foreign-procedure 716) (load-shared-object 55) (define-ftype 4) (foreign-callable 7) - (pointer-sites 297) - (width-sensitive-sites 155) + (pointer-sites 295) + (width-sensitive-sites 153) (blocking-candidates 47) (blocking-without-collect-safe 0) - (hazard-site-count 350) + (hazard-site-count 348) (blocking-review-sites 0) - (safety-review-sites 350)) + (safety-review-sites 348)) (native (rust-file-count 29) (c-file-count 5) @@ -71,7 +71,7 @@ caller reconciliation. The Scheme source inventory now ignores generated `.sls` mirrors and string literals in code generators, so the counts reflect source FFI forms instead of -documentation or emitted-code text. All 46 blocking candidate bindings are now +documentation or emitted-code text. All 47 blocking candidate bindings are now either `__collect_safe` or classified as nonblocking by the scanner; the generated report has 0 `blocking-without-collect-safe` sites. @@ -187,7 +187,11 @@ documents the status/count integer-width ABI. The Scheme `(std compress native-rust)` wrapper now declares native result-length out slots as `void*`, allocates/frees scoped `size_t` slots with `dynamic-wind`, checks native output lengths before slicing, and has focused round-trip plus decompression-cap -regressions. `secure_fs.rs` now documents +regressions. The unified Scheme `(std regex)` facade now uses scoped +foreign-allocated `void*` slots for native regex handles and `size_t` match +offsets, checks those slots before loading results, and preserves the facade +regressions for regex search, replacement, folding, and rx integration. +`secure_fs.rs` now documents descriptor-relative `openat`/`mkdirat`, fd duplication/ownership transfer, stat buffers, directory stream lifecycle, caller output slots, atomic output publication, and test-owned fd conversion @@ -197,7 +201,8 @@ helpers, rejects null/nonzero and oversized buffers before slicing, rejects negative accepted fds before adopting them as `TcpStream`, and documents socket shutdown invariants used to break blocked TLS I/O. The generated inventory now reports 390 annotated native unsafe sites, 0 remaining unsafe review sites, and -0 Scheme blocking calls missing `__collect_safe`. +0 Scheme blocking calls missing `__collect_safe`; the clean-tree Scheme FFI +inventory now reports 348 remaining provisional safety-review sites. Remaining work before closing K3-P1-01: --- a/docs/kimi3-security-recommmendations.md +++ b/docs/kimi3-security-recommmendations.md @@ -729,7 +729,10 @@ the current inventory and remediation trail now live in [ffi-audit.md](ffi-audit `(std compress native-rust)` wrapper now uses scoped foreign `void*`/`size_t` result-length slots, validates bytevector slices, checks native output lengths before slicing, and has focused round-trip plus decompression-cap - regressions. + regressions. The unified Scheme `(std regex)` facade now uses scoped + foreign-allocated `void*` slots for native regex handles and `size_t` match + offsets, checks those slots before loading results, and keeps the regex/rx + facade regressions passing. `wasm_sm.rs` now compiles under `unsafe_op_in_unsafe_fn` for the SpiderMonkey feature, bounds C ABI buffers, validates host-memory offsets, converts poisoned handle-store locks into normal FFI errors, and documents @@ -746,7 +749,7 @@ the current inventory and remediation trail now live in [ffi-audit.md](ffi-audit shutdown invariants for blocked TLS I/O. The generated report now shows 390 annotated native unsafe sites, 0 remaining unsafe review sites, and 0 Scheme blocking calls missing `__collect_safe`; the clean-tree Scheme FFI inventory - now reports 350 remaining provisional safety-review sites. + now reports 348 remaining provisional safety-review sites. Remaining work: finish per-binding Scheme FFI review and targeted scanner rules. --- a/docs/security-reference.md +++ b/docs/security-reference.md @@ -718,7 +718,7 @@ Callers should not treat `allow-degraded?` as successful sandbox installation. ## 7. Parser Hardening -Phases 1-4 are implemented and tested (42 tests in `tests/test-security2-parsers.ss`). Phase 5 (FFI audit) has a reproducible inventory in [ffi-audit.md](ffi-audit.md) via `make ffi-audit-report`; native unsafe review and selected Scheme FFI slices such as Seatbelt, sandbox compatibility, `exec-id`, `regex-native`, and `compress native-rust` are remediated, while broad Scheme per-binding remediation remains open. +Phases 1-4 are implemented and tested (42 tests in `tests/test-security2-parsers.ss`). Phase 5 (FFI audit) has a reproducible inventory in [ffi-audit.md](ffi-audit.md) via `make ffi-audit-report`; native unsafe review and selected Scheme FFI slices such as Seatbelt, sandbox compatibility, `exec-id`, `regex-native`, the unified regex facade, and `compress native-rust` are remediated, while broad Scheme per-binding remediation remains open. ### Depth limits --- a/docs/status.md +++ b/docs/status.md @@ -25,7 +25,7 @@ release artifacts are built as Jerboa multicall binaries with `jerboa`, | Area | Current state | Remaining work | |---|---|---| | Kimi security handoff | [kimi3-security-recommmendations.md](kimi3-security-recommmendations.md) is the backlog. Dated evidence and review manifests live under [reviews/](reviews/). | Keep new security evidence in dated review records and summarize the current release state here. | -| FFI audit phase 5 | [ffi-audit.md](ffi-audit.md) records the scanner output, provisional Scheme binding verdicts, and native Rust export inventory. `make native-export-review-check` gates native export decisions. The worker-launch native path in `aproc.rs`/`seccomp.rs`, crypto FFI buffer path in `crypto.rs`, secure-memory region lifecycle in `secure_mem.rs`, secure filesystem capability boundary in `secure_fs.rs`, TLS native ABI in `tls.rs`, Ed25519/X25519 key-agreement buffers, compression buffers, HTTP parse/writev boundary, embed-crypto ABI, integrity ABI, regex-native ABI, process-control ABI, pcap ABI, inotify ABI, epoll/eventfd ABI, antidebug ABI, SOCKS5 server ABI, PostgreSQL native ABI, Landlock native ABI, wasmi sandbox ABI, SpiderMonkey WASM ABI, DuckDB native ABI, X.509 native ABI, SQLite native ABI, the Scheme Seatbelt `sandbox_init` error-buffer paths, the retired `(std os sandbox)` compatibility surface, `(std os exec-id)` realpath/hash helpers, and the Scheme `(std regex-native)` plus `(std compress native-rust)` out-parameter wrappers now have nearby `SAFETY:` comments or equivalent checked FFI/scanner invariants. Generated counts are at 0 unannotated native unsafe sites and 0 Scheme blocking bindings missing `__collect_safe`. | Finish per-binding Scheme FFI review and targeted scanner rules for the remaining provisional verdicts. | +| FFI audit phase 5 | [ffi-audit.md](ffi-audit.md) records the scanner output, provisional Scheme binding verdicts, and native Rust export inventory. `make native-export-review-check` gates native export decisions. The worker-launch native path in `aproc.rs`/`seccomp.rs`, crypto FFI buffer path in `crypto.rs`, secure-memory region lifecycle in `secure_mem.rs`, secure filesystem capability boundary in `secure_fs.rs`, TLS native ABI in `tls.rs`, Ed25519/X25519 key-agreement buffers, compression buffers, HTTP parse/writev boundary, embed-crypto ABI, integrity ABI, regex-native ABI, process-control ABI, pcap ABI, inotify ABI, epoll/eventfd ABI, antidebug ABI, SOCKS5 server ABI, PostgreSQL native ABI, Landlock native ABI, wasmi sandbox ABI, SpiderMonkey WASM ABI, DuckDB native ABI, X.509 native ABI, SQLite native ABI, the Scheme Seatbelt `sandbox_init` error-buffer paths, the retired `(std os sandbox)` compatibility surface, `(std os exec-id)` realpath/hash helpers, and the Scheme `(std regex-native)`, `(std regex)`, and `(std compress native-rust)` out-parameter wrappers now have nearby `SAFETY:` comments or equivalent checked FFI/scanner invariants. Generated counts are at 0 unannotated native unsafe sites, 0 Scheme blocking bindings missing `__collect_safe`, and 348 remaining provisional Scheme safety-review sites. | Finish per-binding Scheme FFI review and targeted scanner rules for the remaining provisional verdicts. | | Native Rust exports | The native export review now has 190 exported functions: 183 tracked Scheme references and 7 retained standalone C/binary helpers. The previous 35 no-Scheme-reference removal candidates no longer have C ABI export markers. | Re-run `make native-export-review-check` whenever adding or removing native exports. | | Confined worker | `(std security worker)` provides the facade, audit lifecycle, output caps, deadlines, process-group kill, memory rlimit pre-exec setup, Linux syscall/ptrace seccomp pre-exec setup, Linux Landlock filesystem/TCP-connect setup for requested axes, macOS Seatbelt deny-default path/exec/no-network setup for supported axes, standard worker-eval Capsicum entry on FreeBSD, explicit sandbox-axis refusal, egress proxy env wiring, and platform CI smoke for Linux/macOS/FreeBSD sandbox paths. | Keep Linux/macOS/FreeBSD parity tests current; finish arbitrary-command/proxy-aware Capsicum worker paths. | | Fuzzing | `tests/fuzz/corpus/` has 15 checked-in seed inputs, `tests/fuzz/regression/` has 11 crash/rejection regressions, `make fuzz-smoke` runs the deterministic regression gate first, and GitHub CI runs smoke fuzzing normally plus deep fuzzing on scheduled daily runs. | Keep adding minimized corpus and regression inputs for every parser/security bug found. | --- a/lib/std/regex.ss +++ b/lib/std/regex.ss @@ -82,29 +82,33 @@ (def _force-native-init-before-ffi native-available?) (def c-native-compile (if native-available? - (foreign-procedure "jerboa_regex_compile" (u8* size_t u8*) int) + (foreign-procedure "jerboa_regex_compile" ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- native handle uses foreign-u64-ref/checked after scoped-slot null guard + (u8* size_t void*) ; jerboa-security: suppress ffi-u8-star-for-foreign-alloc -- u8* args take Scheme bytevectors; foreign-alloc is passed only to the void* handle slot; jerboa-security: suppress u8star-ffi-with-foreign-alloc -- u8* args take Scheme bytevectors; foreign-alloc is passed only to the void* handle slot + int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns status int and stores a u64 handle to the void* out slot (lambda args (error 'c-native-compile "native backend not available")))) (def c-native-find (if native-available? - (foreign-procedure "jerboa_regex_find" (unsigned-64 u8* size_t u8* u8*) int) + (foreign-procedure "jerboa_regex_find" ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- native offsets use foreign-size-t-ref/checked after scoped-slot null guards + (unsigned-64 u8* size_t void* void*) ; jerboa-security: suppress ffi-u8-star-for-foreign-alloc -- u8* args take Scheme bytevectors; foreign-alloc is passed only to void* offset slots + int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns status int and stores size_t match offsets to void* out slots (lambda args (error 'c-native-find "native backend not available")))) (def c-native-free (if native-available? - (foreign-procedure "jerboa_regex_free" (unsigned-64) int) + (foreign-procedure "jerboa_regex_free" (unsigned-64) int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns a status int for an unsigned-64 handle (lambda args (error 'c-native-free "native backend not available")))) (def c-native-find-at (if native-available? - (foreign-procedure "jerboa_regex_find_at" - (unsigned-64 u8* size_t size_t u8* u8*) int) + (foreign-procedure "jerboa_regex_find_at" ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- native offsets use foreign-size-t-ref/checked after scoped-slot null guards + (unsigned-64 u8* size_t size_t void* void*) int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns status int and stores size_t match offsets to void* out slots (lambda args (error 'c-native-find-at "native backend not available")))) (def c-native-group-count (if native-available? - (foreign-procedure "jerboa_regex_group_count" (unsigned-64) int) + (foreign-procedure "jerboa_regex_group_count" (unsigned-64) int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns a bounded capture count as status-like int (lambda args (error 'c-native-group-count "native backend not available")))) (def c-native-captures (if native-available? - (foreign-procedure "jerboa_regex_captures" - (unsigned-64 u8* size_t size_t u8* size_t) int) + (foreign-procedure "jerboa_regex_captures" ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- capture output is a bounded Scheme bytevector; callers load only populated slots + (unsigned-64 u8* size_t size_t u8* size_t) int) ; jerboa-security: suppress ffi-integer-width-ambiguous -- Rust ABI returns status/count int and fills a Scheme bytevector of size_t capture offsets (lambda args (error 'c-native-captures "native backend not available")))) ;; Upper bound on capture groups we are willing to materialize from the @@ -134,6 +138,53 @@ (catch (exn) #f))))) (loop (re-guardian))))) + (def (non-null-ptr? ptr) + (and ptr (not (= ptr 0)))) + + (def (call-with-foreign-slot size init! k) + (let ([slot #f]) + (dynamic-wind + (lambda () + (set! slot (foreign-alloc size)) ; jerboa-security: suppress u8star-ffi-with-foreign-alloc -- this scoped allocator is used only for void* native out slots; u8* inputs receive Scheme bytevectors + (when (non-null-ptr? slot) + (init! slot))) + (lambda () + (and (non-null-ptr? slot) (k slot))) + (lambda () + (when (non-null-ptr? slot) + (foreign-free slot) + (set! slot #f)))))) + + (def (call-with-u64-slot k) + (call-with-foreign-slot + (foreign-sizeof 'unsigned-64) + (lambda (slot) (foreign-set! 'unsigned-64 slot 0 0)) ; jerboa-security: suppress ffi-pointer-arithmetic-without-bounds -- call-with-foreign-slot allocates an unsigned-64-sized slot and writes offset 0 only + k)) + + (def (call-with-size-t-slot k) + (call-with-foreign-slot + (foreign-sizeof 'size_t) + (lambda (slot) (foreign-set! 'size_t slot 0 0)) ; jerboa-security: suppress ffi-pointer-arithmetic-without-bounds -- call-with-foreign-slot allocates a size_t-sized slot and writes offset 0 only + k)) + + (def (call-with-two-size-t-slots k) + (call-with-size-t-slot + (lambda (start-slot) + (call-with-size-t-slot + (lambda (end-slot) + (k start-slot end-slot)))))) + + (def (foreign-u64-ref/checked who ptr) + (unless (non-null-ptr? ptr) + (error who "native u64 pointer is NULL")) + (foreign-ref 'unsigned-64 ptr 0)) ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- helper rejects null pointers and reads offset 0 from a scoped u64 out-parameter buffer + + (def (foreign-size-t-ref/checked who ptr) + (unless (non-null-ptr? ptr) + (error who "native size_t pointer is NULL")) + (foreign-ref 'size_t ptr 0)) ; jerboa-security: suppress ffi-pointer-return-without-null-guard -- helper rejects null pointers and reads offset 0 from a scoped size_t out-parameter buffer + + ;; ========== Internal helpers ========== ;; Scan a pregexp pattern string for (?P<name>...) named groups. @@ -174,17 +225,18 @@ (cond [(string? pat) pat] [(or (list? pat) (symbol? pat)) (sre->pattern-string pat)] - [else (error 're "expected string, SRE list/symbol, or re object" pat)])) + [else (error 're "expected string, SRE list/symbol, or re object" pat)])) ; jerboa-security: suppress bare-error-no-condition -- existing public constructor argument validation uses this module's simple error style ;; Try to compile a native handle. Returns handle (u64) or #f on failure. (def (try-native-compile! pat-str) (and native-available? (drain-re-guardian!) - (try (let* ([bv (string->utf8 pat-str)] - [hbuf (make-bytevector 8)] - [rc (c-native-compile bv (bytevector-length bv) hbuf)]) - (and (>= rc 0) - (bytevector-u64-native-ref hbuf 0))) + (try (let ([bv (string->utf8 pat-str)]) + (call-with-u64-slot + (lambda (handle-slot) + (let ([rc (c-native-compile bv (bytevector-length bv) handle-slot)]) + (and (>= rc 0) + (foreign-u64-ref/checked 'try-native-compile! handle-slot)))))) (catch (exn) #f)))) ;; Construct a re-object, registering with guardian if native handle present. @@ -342,14 +394,14 @@ ;; and the byte-offset result translated back to character offsets. (def (native-find-from r str start om bv bl) (let* ([handle (re-object-native-handle r)] - [start-b (off-map->byte om start)] - [sbuf (make-bytevector 8)] - [ebuf (make-bytevector 8)]) + [start-b (off-map->byte om start)]) (and (<= start-b bl) - (let ([rc (c-native-find-at handle bv bl start-b sbuf ebuf)]) - (and (= rc 1) - (cons (off-map->char om (bytevector-u64-native-ref sbuf 0)) - (off-map->char om (bytevector-u64-native-ref ebuf 0)))))))) + (call-with-two-size-t-slots + (lambda (start-slot end-slot) + (let ([rc (c-native-find-at handle bv bl start-b start-slot end-slot)]) + (and (= rc 1) + (cons (off-map->char om (foreign-size-t-ref/checked 'native-find-from start-slot)) + (off-map->char om (foreign-size-t-ref/checked 'native-find-from end-slot)))))))))) ;; Internal search with explicit offset. Routes through the native linear ;; engine when safe (ReDoS-safe), otherwise the pregexp backtracking engine @@ -469,13 +521,13 @@ [(and native-available? (re-object-native-handle r)) (try (let* ([h (re-object-native-handle r)] [bv (string->utf8 str)] - [blen (bytevector-length bv)] - [sbuf (make-bytevector 8)] - [ebuf (make-bytevector 8)] - [rc (c-native-find h bv blen sbuf ebuf)]) - (and (= rc 1) - (= (bytevector-u64-native-ref sbuf 0) 0) - (= (bytevector-u64-native-ref ebuf 0) blen))) + [blen (bytevector-length bv)]) + (call-with-two-size-t-slots + (lambda (start-slot end-slot) + (let ([rc (c-native-find h bv blen start-slot end-slot)]) + (and (= rc 1) + (= (foreign-size-t-ref/checked 're-match? start-slot) 0) + (= (foreign-size-t-ref/checked 're-match? end-slot) blen)))))) (catch (exn) ; fall through to pregexp (let ([full (string-append "^(?:" pat-str ")$")]) (and (pregexp-match full str) #t))))] @@ -505,26 +557,26 @@ (let* ([handle (re-object-native-handle r)] [bv (string->utf8 str)] [bl (bytevector-length bv)] - [om (string->off-map str bv)] - [sbuf (make-bytevector 8)] - [ebuf (make-bytevector 8)]) - (let loop ([bpos 0] [acc '()]) - (cond - [(> bpos bl) (reverse acc)] - [else - (let ([rc (c-native-find-at handle bv bl bpos sbuf ebuf)]) - (cond - [(= rc 1) - (let* ([ms-b (bytevector-u64-native-ref sbuf 0)] - [me-b (bytevector-u64-native-ref ebuf 0)] - [ms (off-map->char om ms-b)] - [me (off-map->char om me-b)] - [next (if (> me-b ms-b) me-b - (let ([c (off-map->char om ms-b)]) - (if (>= c len) (+ bl 1) - (off-map->byte om (+ c 1)))))]) - (loop next (cons (substring str ms me) acc)))] - [else (reverse acc)]))])))] + [om (string->off-map str bv)]) + (call-with-two-size-t-slots + (lambda (start-slot end-slot) + (let loop ([bpos 0] [acc '()]) + (cond + [(> bpos bl) (reverse acc)] + [else + (let ([rc (c-native-find-at handle bv bl bpos start-slot end-slot)]) + (cond + [(= rc 1) + (let* ([ms-b (foreign-size-t-ref/checked 're-find-all start-slot)] + [me-b (foreign-size-t-ref/checked 're-find-all end-slot)] + [ms (off-map->char om ms-b)] + [me (off-map->char om me-b)] + [next (if (> me-b ms-b) me-b + (let ([c (off-map->char om ms-b)]) + (if (>= c len) (+ bl 1) + (off-map->byte om (+ c 1)))))]) + (loop next (cons (substring str ms me) acc)))] + [else (reverse acc)]))])))))] [else (let loop ([pos 0] [acc '()]) (if (> pos len) @@ -669,22 +721,22 @@ ;; Native fast path: bytes-based linear scan with find_at. ;; ASCII-confined: byte offsets == char offsets, so positions ;; round-trip cleanly back to the caller's coordinate system. - (let* ([bv (string->utf8 str)] - [bl (bytevector-length bv)] - [sbuf (make-bytevector 8)] - [ebuf (make-bytevector 8)]) - (let loop ([pos 0] [i 0] [acc knil]) - (cond - [(> pos bl) acc] - [else - (let ([rc (c-native-find-at handle bv bl pos sbuf ebuf)]) - (cond - [(= rc 1) - (let* ([ms (bytevector-u64-native-ref sbuf 0)] - [me (bytevector-u64-native-ref ebuf 0)] - [next (if (> me ms) me (+ ms 1))]) - (loop next (+ i 1) (kons i ms me str acc)))] - [else acc]))])))] + (let* ([bv (string->utf8 str)] + [bl (bytevector-length bv)]) + (call-with-two-size-t-slots + (lambda (start-slot end-slot) + (let loop ([pos 0] [i 0] [acc knil]) + (cond + [(> pos bl) acc] + [else + (let ([rc (c-native-find-at handle bv bl pos start-slot end-slot)]) + (cond + [(= rc 1) + (let* ([ms (foreign-size-t-ref/checked 're-fold-positions start-slot)] + [me (foreign-size-t-ref/checked 're-fold-positions end-slot)] + [next (if (> me ms) me (+ ms 1))]) + (loop next (+ i 1) (kons i ms me str acc)))] + [else acc]))])))))] [else ;; Fallback: pregexp positions, start/end form (no substring). (let loop ([pos 0] [i 0] [acc knil])