security: add K3 regression suite
ober
c5f1360246b3b71f59c6110d12107b1f758d754a
--- a/Makefile +++ b/Makefile @@ -1601,6 +1601,7 @@ test-security: native @$(NATIVE_TEST_ENV) $(SCHEME) --libdirs $(LIBDIRS) --script tests/test-jerboa-build-argv.ss @$(NATIVE_TEST_ENV) $(SCHEME) --libdirs $(LIBDIRS) --script tests/test-fasl-safety.ss @$(SCHEME) --libdirs $(LIBDIRS) --script tests/test-security-profile.ss + @$(NATIVE_TEST_ENV) $(SCHEME) --libdirs $(LIBDIRS) --script tests/security/test-k3-regressions.ss @SCHEME=$(SCHEME) sh tests/test-native-loader-policy.sh @$(SCHEME) --libdirs $(LIBDIRS) --script tests/test-tls-deadline.ss @$(NATIVE_TEST_ENV) $(SCHEME) --libdirs $(LIBDIRS) --script tests/test-ffi-buffer-capacity.ss --- a/docs/kimi3-security-recommmendations.md +++ b/docs/kimi3-security-recommmendations.md @@ -8,6 +8,9 @@ about code was verified in the tree on the date above. Updated 2026-07-27: the committed `pattern.ss` balance blocker named in K3-P0-01 was repaired, the balance scanner was fixed for escaped identifiers, and `make audit` now includes a tracked-source balance gate. +The first K3 exploit-shaped regression suite was added under +`tests/security/test-k3-regressions.ss` and is run by `make test-security`; +the canonical implemented-security reference is `docs/security-reference.md`. **Audience:** the next model (or human) asked to implement security work in this repo. Each work item has an ID, the threat it closes, exact files to @@ -493,12 +496,21 @@ Current tests verify features work; almost none verify *attacks fail*. - **Where:** new `tests/security/` tree (keeps attack tests separate from feature tests), wired into `make test-security`. +- **Status:** first suite landed 2026-07-27 in + `tests/security/test-k3-regressions.ss`. It currently pins restricted eval + closure, fail-closed sandbox thunk entry, default-deny network hosts, + authenticated actor frame replay/tamper rejection, safe-FASL rejection, + bounded actor deserialization, taint sink enforcement, bounded `read`, + `gensym`/FFI absence, direct Chez import-audit detection, and URL scheme + sanitization. Remaining additions should extend this file or sibling files + under `tests/security/`, not scatter attack-shaped tests through feature + tests. - **Do:** One test per historical finding so regressions are impossible: sandbox default-env closure (#1 from the AI-attack table), unbounded reader budgets (#2/#10), capability intersection attenuation (#3), empty-host default-deny (#4), symlink canonicalization (#5), bounded actor deserialization (#6), taint sink enforcement (#8), `read`/`gensym` - absence (#9), import-audit detection (#11), URL/attribute sanitization + confinement (#9), import-audit detection (#11), URL/attribute sanitization (#12), raw-fork fail-closed (#13/#14), `run-safe-eval` degradation refusal without `allow-degraded?`, plus new P0-02 worker escapes. Each test names the finding it locks. --- a/docs/security-reference.md +++ b/docs/security-reference.md @@ -436,13 +436,23 @@ Phases 1-4 are implemented and tested (42 tests in `tests/test-security2-parsers | 6 | Distributed actors accepted remotely supplied datums without parser resource limits or message authentication | HIGH | `deserialize-message` uses bounded `jerboa-read` with a message-size ceiling; network-facing payloads can use `serialize-authenticated-message` / `deserialize-authenticated-message`, which HMAC the frame and reject tamper, wrong-key, and replay attempts before payload deserialization. | | 7 | Seccomp/Landlock were stubs | HIGH | Both now have real implementations with actual syscalls (BPF bytecode generation, Landlock ABI detection). | | 8 | Taint tracking had no automatic sink enforcement | MEDIUM | Added `safe-open-input-file`, `safe-open-output-file`, `safe-system`, `safe-delete-file` that auto-reject tainted args. | -| 9 | Restricted environment allowlist included `read` and `gensym` | MEDIUM | `read` removed (replaced by `jerboa-read`). `gensym` removed. | +| 9 | Restricted environment allowlist included unsafe reader/state primitives | MEDIUM | `read` is rebound to bounded `jerboa-read` inside the restricted environment. `gensym` is removed. | | 10 | No reader input size limits | MEDIUM | Added `*max-string-length*` (10 MB), `*max-list-length*` (1M), `*max-symbol-length*` (4 KB). | | 11 | AI-generated code could bypass capabilities via direct Chez imports | MEDIUM | New `(std security import-audit)` scans for forbidden imports at build time. | | 12 | HTML sanitization incomplete for attribute/URL contexts | MEDIUM | Added `sanitize-html-attribute` (hex-encodes non-alphanumeric) and `sanitize-url-attribute` (rejects `javascript:`, `data:`, `vbscript:`, `blob:` schemes). | | 13 | Privilege separation ran Scheme callbacks after raw fork | HIGH | `make-privsep` now fails closed; low-level channels remain for separately exec'd workers. | | 14 | `run-safe` used an unsafe raw-fork Scheme callback | HIGH | Thunk execution is retired and fails closed; isolated workloads must use a fresh exec worker. `*sandbox-max-children*` remains only for API compatibility. | +### Security Regression Tests + +The canonical attack-shaped regression suite lives under `tests/security/` and +is run by `make test-security`. `tests/security/test-k3-regressions.ss` names +the K3 finding each check protects, including restricted-eval closure, +fail-closed sandbox thunk execution, default-deny capability behavior, +authenticated distributed actor frames, safe FASL rejection, bounded actor +deserialization, taint sink enforcement, bounded `read`, import auditing, and +URL scheme sanitization. + --- ## 9. Secure Memory --- a/docs/testing-and-infrastructure.md +++ b/docs/testing-and-infrastructure.md @@ -12,6 +12,18 @@ Jerboa standard library modules for testing, profiling, configuration, terminal - [6. (std misc highlight) -- Scheme Syntax Highlighting](#6-std-misc-highlight----scheme-syntax-highlighting) - [7. (std misc guardian-pool) -- Guardian-Based FFI Cleanup](#7-std-misc-guardian-pool----guardian-based-ffi-cleanup) - [8. (std misc memoize) -- Memoization with LRU](#8-std-misc-memoize----memoization-with-lru) +- [Repository Security Test Targets](#repository-security-test-targets) + +--- + +## Repository Security Test Targets + +Use `make test-security` for exploit-shaped security regressions and adjacent +security policy checks. Attack-focused tests live under `tests/security/`; +feature and compatibility tests stay beside the modules they exercise. The K3 +regression suite is `tests/security/test-k3-regressions.ss`, and +`docs/security-reference.md` is the canonical reference for the implemented +security controls it pins. --- --- a/lib/std/security/import-audit.ss +++ b/lib/std/security/import-audit.ss @@ -150,6 +150,4 @@ (catch (exn) '())))) (reverse results))) - (def (min a b) (if (< a b) a b)) - ) ;; end library new file mode 100644 --- /dev/null +++ b/tests/security/test-k3-regressions.ss @@ -0,0 +1,115 @@ +#!chezscheme +;;; K3 exploit-shaped security regression tests. + +(import (scheme) + (std actor distributed) + (std safe-fasl) + (std security capability) + (std security import-audit) + (std security sandbox) + (std security sanitize) + (std security taint) + (std security restrict)) + +(define pass-count 0) +(define fail-count 0) + +(define-syntax check + (syntax-rules (=>) + [(_ name expr => expected) + (let ([result expr] + [exp expected]) + (if (equal? result exp) + (set! pass-count (+ pass-count 1)) + (begin + (set! fail-count (+ fail-count 1)) + (display "FAIL: ") (display name) + (display " => ") (write result) + (display " expected ") (write exp) (newline))))])) + +(define (raises? thunk) + (guard (exn [#t #t]) + (thunk) + #f)) + +(define (raises-pred? pred thunk) + (guard (exn [#t (pred exn)]) + (thunk) + #f)) + +(define (write-text-file path text) + (guard (exn [#t (void)]) (delete-file path)) + (let ([port (open-output-file path)]) + (display text port) + (close-output-port port))) + +(define (import-audit-blocks-direct-chezscheme?) + (let ([path "/tmp/jerboa-k3-import-audit-test.ss"]) + (write-text-file path "(import (chezscheme))\n(display \"bad\")\n") + (let ([violations (parameterize ([*trusted-modules* '()]) + (audit-imports-file path))]) + (guard (exn [#t (void)]) (delete-file path)) + (and (pair? violations) + (import-violation? (car violations)))))) + +(define (empty-host-capability-denies-network?) + (let ([cap (make-net-capability 'connect #t 'listen #f 'hosts '())]) + (net-allowed-host? cap "example.com"))) + +(define (authenticated-frame-replay-rejected?) + (let* ([sender (make-message-auth-state "k3-cookie" #f)] + [receiver (make-message-auth-state "k3-cookie" #f)] + [frame (serialize-authenticated-message sender '(hello k3))]) + (and (equal? (deserialize-authenticated-message receiver frame) '(hello k3)) + (raises? (lambda () + (deserialize-authenticated-message receiver frame)))))) + +(define (authenticated-frame-tamper-rejected?) + (let* ([sender (make-message-auth-state "k3-cookie" #f)] + [receiver (make-message-auth-state "k3-cookie" #f)] + [frame (serialize-authenticated-message sender '(hello k3))] + [last (- (bytevector-length frame) 1)]) + (bytevector-u8-set! frame last (bitwise-xor (bytevector-u8-ref frame last) 1)) + (raises? (lambda () + (deserialize-authenticated-message receiver frame))))) + +(check "K3-AI-01 restricted eval blocks system" + (raises? (lambda () (restricted-eval-string "(system \"true\")"))) => #t) +(check "K3-AI-14 run-safe thunk entry is fail-closed" + (raises-pred? sandbox-error? (lambda () (run-safe (lambda () 1)))) => #t) +(check "K3-AI-01 run-safe-eval blocks system" + (raises-pred? sandbox-error? (lambda () (run-safe-eval "(system \"true\")"))) => #t) +(check "K3-AI-04 empty network host policy denies connects" + (empty-host-capability-denies-network?) => #f) +(check "K3-AI-06 authenticated actor frame rejects replay" + (authenticated-frame-replay-rejected?) => #t) +(check "K3-AI-06 authenticated actor frame rejects tampering" + (authenticated-frame-tamper-rejected?) => #t) +(check "K3-P0-04 safe FASL rejects procedures" + (raises? (lambda () (safe-fasl-write-bytevector (lambda (x) x)))) => #t) +(check "K3-AI-06 distributed message size cap rejects oversize input" + (parameterize ([*max-message-size* 8]) + (raises? (lambda () (deserialize-message (make-bytevector 64 65))))) => #t) +(check "K3-AI-08 tainted command strings are rejected" + (raises-pred? taint-violation? (lambda () (safe-system (taint-http "true")))) => #t) +(check "K3-AI-09 restricted eval blocks reader eval syntax" + (raises? (lambda () (restricted-eval-string "#.(+ 1 2)"))) => #t) +(check "K3-AI-09 restricted read parses ordinary data" + (restricted-eval-string "(let ((p (open-input-string \"(1 2)\"))) (read p))") => '(1 2)) +(check "K3-AI-09 restricted read rejects reader eval syntax" + (raises? (lambda () (restricted-eval-string "(let ((p (open-input-string \"#.(+ 1 2)\"))) (read p))"))) => #t) +(check "K3-AI-09 restricted eval blocks gensym" + (raises? (lambda () (restricted-eval-string "(gensym)"))) => #t) +(check "K3-AI-09 restricted eval blocks foreign-procedure" + (raises? (lambda () (restricted-eval-string "(foreign-procedure \"getpid\" () int)"))) => #t) +(check "K3-AI-11 import audit blocks direct chezscheme imports" + (import-audit-blocks-direct-chezscheme?) => #t) +(check "K3-AI-12 javascript URL attributes are rejected" + (raises-pred? url-scheme-violation? (lambda () (sanitize-url-attribute "javascript:alert(1)"))) => #t) + +(display " k3-security-regressions: ") +(display pass-count) (display " passed") +(when (> fail-count 0) + (display ", ") (display fail-count) (display " failed")) +(newline) +(when (> fail-count 0) (exit 1))