updates
ober
51c9ea2ce0d6cc75f52ddc51bc37ab8eb38a046c
new file mode 100644 --- /dev/null +++ b/docs/jerboa-code-typed-static-plan.md @@ -0,0 +1,207 @@ +# jerboa-code Typed Static Plan + +Date: 2026-05-22 + +This document records the proposed plan for using Typed Jerboa as the source of +truth for generated Rust artifacts in `/Users/user/mine/jerboa-code`, with the +generated Rust linked into the final binary. + +## Goal + +The goal is not to translate arbitrary dynamic Jerboa to Rust. The goal is to +let `jerboa-code` contain selected `(typed-library ...)` source modules that are +checked by Typed Jerboa, compiled to generated Rust, and statically linked into +the `jcode` binary. + +The intended result: + +```text +jerboa-code typed source + -> Typed Jerboa checker + -> generated Rust crate + -> generated Jerboa wrapper + -> static archive linked into jcode + -> wrapper resolves jt_* symbols from the binary +``` + +Generated Rust should remain a disposable build artifact. Handwritten Rust +should remain only for deliberate native boundaries. + +## Current Backend Status + +The Typed Jerboa backend in this repository already has a working vertical +slice: + +- `(typed-library ...)` forms in `.ss` files. +- Parser, checker, typed core IR, and Rust emitter. +- Generated Rust crate under `build/typed/rust`. +- Generated `.ss` wrappers for dynamic Jerboa callers. +- Dynamic FFI mode through `JERBOA_TYPED_RUST_LIB`. +- Static binary mode through `make binary-typed`. + +The static path is important: `make binary-typed` builds the generated Rust +`staticlib`, links it into the binary, extracts `jt_*` exports, generates +symbol-registration code, and registers those symbols with Chez so generated +wrappers work without `JERBOA_TYPED_RUST_LIB`. + +Verified on 2026-05-22: + +```sh +make typed-test +make typed-wrapper-smoke +make typed-split-tree-smoke +make binary-typed-smoke +``` + +All passed. + +## jerboa-code Audit Snapshot + +Read-only audit command: + +```sh +make pure-audit PURE_AUDIT_ARGS="/Users/user/mine/jerboa-code" +``` + +Result on 2026-05-22: + +| Category | Count | +|---|---:| +| `external-process` | 82 | +| `ffi-boundary` | 64 | +| `native-loader` | 14 | +| `rust-wrapper` | 6 | +| `c-native` | 5 | +| `rust-source` | 2 | +| `rust-build` | 1 | + +Conclusion: `jerboa-code` is not mostly a Rust problem. The direct Rust surface +is small. The larger migration problem is native loading, FFI, external process +orchestration, and vendored C/native surfaces. + +## Best First Step + +The best first pilot is a small typed arithmetic kernel behind existing dynamic +Jerboa code. Do not start with TLS, crypto, SQLite, terminal UI, subprocess +orchestration, JSON, or hash-table-heavy modules. + +Recommended first target: + +```text +/Users/user/mine/jerboa-code/src/jcode/core/models.ss +``` + +Specifically, split the pure arithmetic part of `compute-cost` into a new typed +module, for example: + +```text +/Users/user/mine/jerboa-code/src/jcode/typed/pricing.ss +``` + +Dynamic Jerboa should keep responsibility for: + +- model lookup +- pricing cache +- usage hash-table extraction +- JSON and filesystem cache behavior +- missing-field defaults + +Typed Jerboa should own only the normalized scalar calculation: + +- token accounting +- cached-token handling +- cache-write handling +- clamping OpenAI billable input tokens at zero +- final dollar-cost computation + +This is the right first module because it is real application behavior, but it +does not require the typed backend to support hash tables, JSON parsing, file +I/O, process I/O, network I/O, or broad string/list APIs before the static +project attachment path has been proven. + +## Why Not Start Elsewhere + +Avoid these as the first pilot: + +- `src/jcode/core/message.ss`: useful, but JSON/hash/UUID heavy. +- `src/jcode/core/permissions.ss`: attractive, but needs richer string, glob, + and list support before it is a low-risk typed port. +- `src/jcode/provider/provider.ss`: provider I/O and TLS are explicit native and + network boundaries. +- `src/jcode/ui/tui-ffi.ss`: terminal FFI and C shim work should remain a + separate audited boundary. +- `vendor/chez-sqlite`: SQLite is a deliberate native database boundary. +- `vendor/jerboa-websearch/wasm`: handwritten Rust/WASM, but better tackled + after the project attachment path exists. + +## Implementation Plan + +1. Add project attachment support to the typed static build path. + + The Jerboa repo already proves static typed linking for `jerboa-bin`. The + next step is making that mechanism usable by a sibling project binary. + Required inputs: + + - external typed source list + - generated Rust output directory + - generated wrapper output directory + - project libdirs + - project entry script + - extra native archives and link flags needed by `jerboa-code` + +2. Add the typed pricing source to `jerboa-code`. + + Keep the public `compute-cost` API dynamic at first. It should normalize + current hash-table inputs and call the generated typed wrapper for the + arithmetic kernel. + +3. Add tests in `jerboa-code`. + + Cover: + + - OpenAI-style usage fields + - Anthropic-style usage fields + - cached-token read pricing + - cache-write pricing + - unknown pricing returning `0` + - wrapper rejection of wrong dynamic argument shapes + +4. Add a static `jcode` binary smoke. + + Build a `jcode` binary with the generated typed Rust archive linked in, unset + `JERBOA_TYPED_RUST_LIB`, and run a small caller that exercises the typed + pricing path through normal `jcode` imports. + +5. Rerun the pure audit. + + The first pilot will not make `jerboa-code` pure. It should establish the + migration pattern and classify one real slice as Typed Jerboa source plus + generated Rust, not handwritten Rust. + +## Backend Gaps To Track + +These are not blockers for the pricing pilot, but they matter before larger +ports: + +- Project binary integration needs first-class support instead of each project + carrying a custom static linker script. +- Generated wrappers need a project-friendly loading/import convention. +- The backend currently uses Cargo; direct `rustc` mode is still open. +- The typed subset does not yet cover hash tables, rich string/list + transformations, file I/O, process I/O, or network I/O. +- Differential testing against dynamic Jerboa implementations is still limited. +- Some status text in `typed-jerboa.md` is stale and should later be reconciled. + +## Acceptance Criteria + +The pilot is done when: + +- `jerboa-code` has at least one checked `(typed-library ...)` source file. +- Generated Rust and wrappers are build artifacts, not hand-edited source. +- Dynamic `jcode` tests pass through the generated wrapper. +- A static `jcode` binary calls the generated typed function with + `JERBOA_TYPED_RUST_LIB` unset. +- The pure audit reports the new slice as Typed Jerboa/generated Rust rather + than handwritten Rust. +- Remaining native boundaries are documented as future migration work or + deliberate audited boundaries. --- a/docs/limits.md +++ b/docs/limits.md @@ -86,8 +86,9 @@ also accept `deny-read-paths:`, `deny-write-paths:`, and `deny-exec-paths:`; macOS Seatbelt renders those as explicit SBPL denies, while backends without negative path exceptions report degraded so fail-closed callers can refuse. `(std net allow-proxy)` provides `allow-proxy-url` and `allow-proxy-env` helpers -for proxy handoff, and macOS `sandbox-wrap-command` can now wrap `net: -local-only` policies so custom launchers can deny remote egress before exec. +for proxy handoff, including Node's environment proxy opt-in, and macOS +`sandbox-wrap-command` can now wrap `net: local-only` policies so custom +launchers can deny remote egress before exec. ## 2. Process Supervision @@ -297,7 +298,8 @@ Typical flow: 1. parent starts allowlist proxy on loopback or Unix socket 2. parent denies direct child network through sandbox policy -3. parent injects `HTTPS_PROXY`, `HTTP_PROXY`, or tool-specific proxy env +3. parent injects `HTTPS_PROXY`, `HTTP_PROXY`, `NODE_USE_ENV_PROXY`, or + tool-specific proxy env 4. proxy allows only configured destinations The proxy should be a library primitive, not a `jsh`-only feature. --- a/lib/std/net/allow-proxy.ss +++ b/lib/std/net/allow-proxy.ss @@ -115,7 +115,8 @@ (def (allow-proxy-env p) ;; Environment overrides understood by common HTTP clients. The ;; lowercase spellings are intentional: curl, wget, Python requests, - ;; Node, and many package managers check one or both forms. + ;; Node, and many package managers check one or both forms. Modern Node + ;; only wires proxy env into its default agents when explicitly enabled. (let ([url (allow-proxy-url p)] [no-proxy "localhost,127.0.0.1,::1"]) (list (cons "HTTP_PROXY" url) @@ -125,7 +126,8 @@ (cons "https_proxy" url) (cons "all_proxy" url) (cons "NO_PROXY" no-proxy) - (cons "no_proxy" no-proxy)))) + (cons "no_proxy" no-proxy) + (cons "NODE_USE_ENV_PROXY" "1")))) (def (allow-proxy . opts) (let ([host "127.0.0.1"] [port 0] [allow '()] [logger #f]