updates
ober
03037d9fb1637b1e8e406688470dff78f2a08091
--- a/Dockerfile.qt +++ b/Dockerfile.qt @@ -15,9 +15,9 @@ ARG BASE_IMAGE=docker.io/library/ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 FROM ${BASE_IMAGE} AS builder -ARG JERBOA_VERSION=v0.2.7 -ARG JERBOA_SOURCE_COMMIT=73176e945f656334b8b1cf0d6b11c6c41f8ba427 -ARG JERBOA_SOURCE_TREE=5e2ce0ac497d4b93d3a5859a13856fdbb0987ba3 +ARG JERBOA_VERSION=master +ARG JERBOA_SOURCE_COMMIT=70c14e8e0e0d9f194bbd142e7418d2b546c9c468 +ARG JERBOA_SOURCE_TREE=4e5dbcc94684a7e61187897fa484a54f77c78929 ARG JBROWSER_SOURCE_COMMIT ARG JBROWSER_SOURCE_TREE ARG APT_SNAPSHOT=20260701T000000Z --- a/Makefile +++ b/Makefile @@ -1,9 +1,9 @@ # jerbuild bundles Chez Scheme + the jerboa stdlib, so building the browser # entry needs only `jerbuild` + a C compiler. CI uses a pinned project-local # release toolchain; developer machines can still override JERBUILD explicitly. -JERBOA_VERSION ?= v0.2.7 -JERBOA_COMMIT ?= 73176e945f656334b8b1cf0d6b11c6c41f8ba427 -JERBOA_TREE ?= 5e2ce0ac497d4b93d3a5859a13856fdbb0987ba3 +JERBOA_VERSION ?= master +JERBOA_COMMIT ?= 70c14e8e0e0d9f194bbd142e7418d2b546c9c468 +JERBOA_TREE ?= 4e5dbcc94684a7e61187897fa484a54f77c78929 JERBOA_BUNDLE_SHA256 ?= ac8587f3b466f7d6d3047b4cafefd9985a08983503a5ad0764d4947cef25e828 JERBOA_TOOL_DIR ?= $(CURDIR)/.jerboa/bin JERBUILD ?= $(JERBOA_TOOL_DIR)/jerbuild new file mode 100644 --- /dev/null +++ b/docs/browser-core-rewrite.md @@ -0,0 +1,475 @@ +# Browser Core Rewrite — Sizing a Port of the C++ Backend to Jerboa + +**Status:** assessment / planning document. No code in this document has been +implemented. Written 2026-07-30 against repo state `jerboa-browser` (Qt +6.11.1, jerboa 0.2.6/0.2.7). + +**Audience:** maintainers deciding whether/how to reduce the C++ footprint, and +the implementing agent (a lower-tier model) executing the ticket plan in §9. + +**Scope:** the first-party C++ web-toolkit dependency rooted at +`qt-webengine/` plus `include/jerboa_browser.h` and the Rust C-FFI wrapper at +`vendor/adblock-rust-ffi/`. The pure-Scheme browser engine already living in +`scheme/browser/*.ss` (keymap, minibuffer, commands, buffers, passwords, +bookmarks) is **not** in scope — it is already Jerboa and needs no port. + +--- + +## 1. Executive summary + +- The "web toolkit written in C++" is **Qt WebEngine 6.11.1 (Chromium)**, + reached through a first-party shim, `libjerboa_browser`, of **~2,800 lines of + C++** (plus a 315-line C ABI header, ~420 lines of C++ tests, and a 248-line + Rust C-FFI wrapper for Brave's adblock engine). +- **A literal 1:1 port of that shim to Jerboa is impossible while Qt WebEngine + remains the engine.** Qt exposes only a C++ ABI; Jerboa's FFI + (`(jerboa ffi)` → Chez `foreign-procedure`) can only call **C** functions. + Three of the shim's integration points are C++ **virtual overrides** + (request interceptor, navigation-policy page, key-event filter) and several + more are **synchronous, on the Qt UI thread** (intercept decisions, TLS and + permission decisions), which a polling Scheme run-loop cannot service. +- What *is* portable today: everything Qt-free — the handle registry, UTF-8 + validation, URL-policy decision logic, download bookkeeping, TLS/permission + grant sets, the secure-memory allocator (pure POSIX `mmap`/`mlock`), and the + adblock engine (already a C ABI — Jerboa can call `engine_create` / + `engine_match` directly). That is roughly **half the shim by line count** and + nearly all of its *auditable security logic*. +- **Recommended path: Option A ("logic-up port")** — move all Qt-free logic + into Jerboa in seven small tickets, shrinking the C++ to a ~900–1,200-line + event-tap. Estimated **13–21 engineer-days**. The engine, build pipeline, + sandbox posture, and 145–149 MB bundled tarballs are unchanged. +- **Option B (full port onto a C-ABI engine — CEF or WebKit2GTK, later Servo)** + is the only route that eliminates C++ entirely. Estimated **36–63 + engineer-days** plus sandbox/packaging re-proof. It is **gated**: for Servo, + the six conditions in `notes/servo-watchpoint.md` must hold; they do not + today. +- **Option C (write a browser engine in Jerboa)** is sized and rejected: + NetSurf-class capability is ≥5 engineer-years; modern-web parity is a + 10⁶-LOC, decade-scale effort (Servo, Ladybird). Not viable for this project. + +--- + +## 2. What the C++ dependency actually is + +### 2.1 Inventory (first-party code only; build artifacts excluded) + +| File | LOC | Role | Portability class | +|---|---:|---|---| +| `include/jerboa_browser.h` | 315 | Backend-neutral `jwb_*` C ABI (~60 entry points) | Contract — unchanged by any option | +| `qt-webengine/src/browser_handles.cpp` | 150 | Tagged handle registry (context/view/window), thread-local last-error. **Qt-free by design** | **Port now** | +| `qt-webengine/src/browser_utf8.cpp` | 42 | Strict UTF-8 validation (overlong/surrogate/range) | **Port now** | +| `qt-webengine/src/browser_url_policy.cpp` (+`.h` 17) | 112 | Deny-by-default scheme/capability matrix, file-root canonicalization | **Port now** (logic); hot-path matcher stays native (see §4) | +| `qt-webengine/src/browser_policy.cpp` | 470 | Capability interceptor, download tracking, adblock engine glue + ABI | Mixed: ~300 LOC portable bookkeeping/glue, ~170 LOC Qt seam | +| `qt-webengine/src/browser_ffi.cpp` | 251 | Jerboa-friendly facade (thread-local string buffers) + `jwb_secure_*` POSIX secure memory | **Port now** (secure-mem); facade evaporates (§4.1) | +| `qt-webengine/src/browser_abi.cpp` | 53 | `QApplication` singleton, version string | Tap (stays C++) | +| `qt-webengine/src/browser_view.cpp` | 480 | Context/view lifecycle, `PolicyPage` override, TLS + permission decisions, navigation verbs | Mixed: ~130 LOC portable grant-sets; rest is Qt seam | +| `qt-webengine/src/browser_page.cpp` | 224 | JS eval (async + sync pump), event pumping, find-in-page | Tap (stays C++) | +| `qt-webengine/src/browser_window.cpp` | 546 | `QMainWindow` chrome (stack/status/minibuffer), key-capture filter + queue | Tap (stays C++); queue consumer already Scheme | +| `qt-webengine/src/browser_automation.cpp` | 129 | Offscreen test hooks: resize, PNG grab, synthetic keys, zoom, clipboard | Tap (stays C++) | +| `qt-webengine/src/browser_host.cpp` | 73 | Pane focus/title, `jwb_exec`/`jwb_quit` | Tap (stays C++) | +| `qt-webengine/src/main_demo.cpp` | 101 | Smoke-test driver | Disposable | +| `qt-webengine/host/host_demo.cpp` | 106 | Ticket 2.2 host-integration demo | Disposable | +| `qt-webengine/probe/main.cpp` | 50 | Ticket 0.1 proof-of-host | Disposable | +| `qt-webengine/tests/*.cpp` | 419 | ctest: ABI/handles, URL policy, secure-fs, adblock | Port expectations to golden tests (§9 T0) | +| `vendor/adblock-rust-ffi/src/lib.rs` | 248 | C ABI over Brave `adblock-rust` (`engine_create/_match/_destroy`, domain resolver) | **Already C** — callable from Jerboa today | +| `qt-webengine/CMakeLists.txt` (+exports) | ~120 | Build: Qt6 WebEngineWidgets, cargo staticlib, symbol export lists | Shrinks under A; replaced under B | + +**Totals:** ~3,670 LOC first-party C++ (of which the shipped shim is ~2,800), +248 LOC Rust wrapper, ~120 LOC CMake. Consumed from Scheme by **67 +`define-c-lambda` bindings** in `scheme/browser.ss` (845 LOC). + +### 2.2 What the shipped artifact costs today + +- `libjerboa_browser.dylib`: **4.7 MB** (macOS arm64 build). +- Bundled Linux artifacts (`make static-qt`, Podman): **145–149 MB tarballs** + containing the browser binary, the shim, Qt libs/plugins, + `QtWebEngineProcess`, and WebEngine resources. **This size is ~100% Qt + WebEngine; no port of the shim changes it. Only an engine swap (Option B) + moves it.** +- Toolchain: Apple clang 17 / C++17 / CMake 4.3.1 / Qt 6.11.1 (Homebrew) on + macOS; Ubuntu 24.04 container + cargo (for the vendored adblock staticlib) + for Linux bundles. Supply-chain tracking in `supply-chain.lock`. + +--- + +## 3. Why the C++ exists at all (the three hard constraints) + +1. **Qt has no C ABI.** Every `QWebEngine*`/`QWidget` call must originate in + C++. Jerboa's `(jerboa ffi)` (`c-lambda`, `define-c-lambda`) expands to Chez + `foreign-procedure`, which binds **C symbols only**. There is no path from + Scheme to a C++ class method. +2. **Three integration points are C++ virtual overrides**, which cannot be + implemented as C callbacks at all: + - `CapabilityInterceptor::interceptRequest` (`QWebEngineUrlRequestInterceptor`) + — every network/subresource request is gated here, synchronously. + - `PolicyPage::acceptNavigationRequest` (`QWebEnginePage`) — navigation + preflight, synchronously. + - `KeyFilter::eventFilter` (`QObject`) — application-wide key capture. +3. **Signal/slot delivery and synchronous decisions.** Qt signals + (`loadFinished`, `certificateError`, `permissionRequested`, + `downloadRequested`, `runJavaScript`/`findText` result callbacks) are + delivered to C++ callables on the Qt UI thread. `interceptRequest` and the + TLS/permission handlers must answer **in the call** — they cannot suspend + and ask a polling Scheme loop. Chez `foreign-callable` exists but is not + wrapped by `(jerboa ffi)`, carries thread/collection restrictions, and + calling into the Scheme runtime re-entrantly from the UI thread mid-FFI is + an unjustified risk for this codebase's assurance posture + (`docs/threat-model.md`). + +These constraints define the **irreducible native core**: QApplication +lifecycle, the three override classes, signal→data forwarding, JS-eval/find +plumbing, widget chrome, clipboard, PNG grab, synthetic keys, event pumping. +Everything else in the shim is glue that history placed in C++, not necessity. + +**The pattern that already works:** the key-capture queue +(`jwb_window_poll_key`). C++ *pushes* events into a queue; Scheme *polls* and +owns all decisions downstream of the queue. Every portable subsystem below +moves to this push/poll shape, with synchronous hot-path decisions fed by +**immutable data snapshots pushed down from Scheme** (never by calling up into +Scheme). + +--- + +## 4. Portability classification + +### 4.1 Port to Jerboa now (no engine change; ~1,100–1,400 LOC of C++ deleted) + +| Component | Current C++ | Jerboa form | Notes | +|---|---|---|---| +| Handle registry | 150 LOC `std::unordered_map` + mutex | Hash table + Chez mutex; or stay thin (see T6) | Already Qt-free; has Qt-free tests today | +| UTF-8 validation | 42 LOC table walk | Bytevector walk; Chez decoders exist | Golden-testable against the C++ truth | +| URL-policy matrix | 112 LOC (QUrl/QFileInfo) | Pure Scheme predicate over parsed URLs + `realpath(3)` via libc FFI | The *authoring* moves; the hot-path matcher stays native, fed by a data snapshot (§4.2) | +| Secure memory (`jwb_secure_*`) | ~120 LOC in `browser_ffi.cpp`: `mmap` guard pages, `mlock`, `madvise(DONTDUMP/DONTFORK)`, `/dev/urandom` fill, volatile wipe | Direct libc FFI from `scheme/std/crypto/secure-mem.ss` (already the sole consumer; binds `jwb_secure_*` today) | **Zero Qt.** Cleanest deletion in the project | +| Facade string buffers | ~60 LOC thread-local `std::string` | Nothing — Chez `char-string` returns already copy | The facade exists *for* the FFI; a Jerboa-native binding needs no facade | +| Downloads bookkeeping | ~150 LOC records + snapshot formatter | Scheme records fed by download **events** from the tap | Decision (accept/cancel) is capability data — pushed down, applied natively | +| TLS / permission grant sets | ~130 LOC `std::set` + last-error strings | Scheme sets; serialized snapshots pushed to the tap on every change | Tap consults the snapshot synchronously; never calls up | +| Adblock lists + counters | ~120 LOC `QFile`/`QDir`/env + seed list | Scheme: list discovery, 16 MiB cap, seed rules, byte counters | Rules blob is passed down at engine build; `engine_match` stays in the native hot path | +| Adblock engine itself | Rust C ABI (`vendor/adblock-rust-ffi`) | Optionally bound directly from Scheme for offline tooling/tests (`engine_create`, `engine_match`, `engine_destroy`, `c_char_buffer_destroy`) | The request-path match remains native (synchronous, UI thread) | + +### 4.2 Stays native under Option A (the "tap", ~900–1,200 LOC after thinning) + +`QApplication` lifecycle · `PolicyPage` (navigation preflight — applies the +pushed-down policy snapshot) · `CapabilityInterceptor` (applies snapshot + +`engine_match`) · `KeyFilter` (already minimal: push to queue) · signal→event +queue forwards (load-finished, download, TLS-denied, permission-denied, +JS/find results) · window chrome widgets (QMainWindow/QStackedWidget/QLabel/ +QLineEdit — see §6 risk R4) · clipboard, PNG grab, synthetic keys, zoom · +`jwb_pump_events`/`jwb_pump_wait` (the Scheme run-loop's heartbeat). + +### 4.3 Never portable (any option short of a new engine) + +The rendering engine: HTML/CSS/JS execution, layout, compositing, the Chromium +sandbox, `QtWebEngineProcess`. This is the dependency that dominates binary +size, build time, and attack surface — and it is untouched by every realistic +option except B. + +--- + +## 5. Options + +### Option 0 — Status quo (keep the shim as-is) + +**Cost of keeping:** ~3,670 LOC C++ + 248 LOC Rust to audit each release; +C++/CMake/cargo toolchain required for every build; the FFI facade duplicates +string handling that Chez already does; security logic (policy matrix, grant +sets, adblock lists) lives in the least-reviewable language in the repo. +**Benefit:** zero engineering spend; the ABI is already stable and tested. + +### Option A — Logic-up port (thin C++ tap) — **recommended** + +Move §4.1 to Jerboa in seven tickets (§9). C++ shrinks from ~2,800 to +~900–1,200 LOC of near-declarative tap code. + +- **Gain:** all deny-by-default policy logic, grant management, list handling, + secure memory, and bookkeeping become Scheme — dogfoodable, REPL-inspectable, + covered by the existing Scheme test harness; the C++ audit surface roughly + halves; the `jwb_*` ABI and every consumer in `scheme/browser.ss` remain + valid throughout (the port is *behind* the ABI). +- **Cost/risk:** 13–21 engineer-days; new failure mode class at the + snapshot-push boundary (stale policy snapshot vs. live decision — mitigated + by versioning snapshots, §9 T4); Qt, build, packaging, tarball size + **unchanged**. + +### Option B — Full port onto a C-ABI engine — the only literal "no C++ left" + +Replace Qt WebEngine with an engine exposing a C ABI, then bind it directly +from Jerboa; `libjerboa_browser` and every C++ file in §2.1 are deleted. + +| Candidate engine | C ABI | Chromium parity | Verdict | +|---|---|---|---| +| **CEF** (`cef_capi.h`) | Yes, comprehensive | Yes (is Chromium) | Reference candidate. Callback-heavy (structs of function pointers → needs Chez `foreign-callable`; spike first). Distribution/packaging roughly as heavy as Qt WebEngine. | +| **WebKit2GTK** | Yes (GObject C API) | No (WebKit) | Good Linux fit; chrome could be Scheme-driven GTK (C API). macOS story is poor (WKWebView is Objective-C; binding via `libobjc` FFI is possible but fragile). Page-compat risk is real. | +| **Servo** | Embedding API not yet stable | No | **Gated** — all six conditions in `notes/servo-watchpoint.md` must hold; none of the six is fully met today. When they are, Option B *becomes* "the Servo backend" behind the existing ABI, exactly as that note designs. | +| **Ultralight** | Yes | No | Proprietary license — incompatible with this repo's supply-chain/release-evidence posture. Rejected. | + +Effort (CEF reference path): minimal-embed spike 3–5 d · Jerboa bindings for +the ~40–60-function capi subset + `foreign-callable` spike 8–12 d · chrome +reimplementation (no widgets ship with CEF; GTK on Linux / ObjC bridge on +macOS — the point where Option B quietly reintroduces native code) 8–15 d · +policy/adblock re-plumb 4–6 d · **sandbox + threat-model re-evidence 5–10 d** · +packaging pipeline replacement 5–10 d · offscreen test harness rework 3–5 d. +**Total: 36–63 engineer-days (7–13 weeks)**, ending with a *larger* Jerboa +codebase but zero C++, a re-proven sandbox story, and a rebuilt ~150 MB +packaging pipeline. + +### Option C — Pure-Jerboa engine — sized and rejected + +Reference points: Servo and Ladybird are 10⁶-LOC, decade-scale, many-person +efforts; NetSurf (~1.5×10⁵ LOC C + Duktape/QuickJS) still fails much of the +modern web; litehtml (~5×10⁴ LOC) has no JS at all. A Jerboa engine reaching +even NetSurf-class (limited JS, no modern sandbox) is **≥5 engineer-years**; +reaching this project's stated needs (real-world pages, `jwb_eval` +JavaScript, sandbox parity per `docs/webengine-sandbox.md`) is not achievable +on any project-relevant horizon. **Rejected**; recorded so the question stays +answered. + +--- + +## 6. Risk register + +- **R1 — Snapshot staleness (Option A):** the tap decides synchronously from + data Scheme pushed earlier. Mitigation: every snapshot carries a generation + counter; the tap exposes it; Scheme asserts the generation after each push; + tests cover push-during-flight ordering. +- **R2 — `foreign-callable` reliance (Option B only):** CEF's capi is + callback-structs; Chez `foreign-callable` is unwrapped by `(jerboa ffi)` and + has threading/GC restrictions. Mitigation: a ≤2-day spike *before* any Option + B commitment; if the spike fails, Option B is dead on this runtime. +- **R3 — Security regressions in transit:** the ported logic *is* the security + model (deny-by-default, TLS fail-closed, permission denial). Mitigation: T0 + golden tests freeze behavior *before* any deletion; no ticket merges with a + golden-test hole; `make verify` + the four ctest suites stay green until the + suites themselves are ported (then their Scheme successors must assert the + same matrix, reproduced in §8). +- **R4 — Chrome widgets:** QLabel/QLineEdit status/minibuffer stay Qt under + Option A (reimplementing them buys nothing — they are 200 LOC of + declarations). Under Option B they become a real cost (GTK/ObjC). Noted, not + hidden. +- **R5 — Scope creep into the Scheme engine:** `scheme/browser/*.ss` is done; + the port must not "improve" it. Any diff there beyond binding-name updates + fails review. + +--- + +## 7. Effort summary + +| Option | Engineer-days (1 experienced, Jerboa-fluent) | Lower-tier-model multiplier | C++ remaining | Engine/build/tarball | +|---|---:|---:|---:|---| +| 0 — status quo | 0 | — | ~3,670 LOC | unchanged | +| **A — logic-up port** | **13–21 d** | ×2–3 wall-clock (ticket structure contains failure modes) | ~900–1,200 LOC tap | unchanged | +| B — C-ABI engine swap | 36–63 d | not recommended as a first delegation | 0 (plus GTK/ObjC chrome risk) | replaced; re-proven | +| C — pure Jerboa engine | ≥5 engineer-years | n/a | 0 | replaced | + +**Recommendation:** execute **Option A** now (it is dogfooding with a real +payoff and zero engine risk); keep **Option B** as the documented contingency +whose Servo trigger conditions are already codified in +`notes/servo-watchpoint.md`; keep **Option C** answered-and-closed. + +--- + +## 8. Golden specification (frozen from the C++; the port's contract) + +These tables transcribe the current native behavior. Golden tests in T0 assert +them; every ported subsystem must reproduce them exactly. + +### 8.1 URL-policy matrix (`browser_url_policy.cpp`, `tests/url_policy_test.cpp`) + +| Input | Caps | Result | +|---|---|---| +| `http:`, `https:`, `ws:`, `wss:` | needs `NETWORK`; allow iff present | else deny "network URL requires the NETWORK capability" | +| `file:` | needs `FILESYSTEM` **and** empty host (`file://server/...` always denied) **and** non-empty canonical root **and** target exists **and** `canonicalPath(target)` beneath root (root itself allowed; prefix is `root + separator`, except filesystem root `/`) | symlink escapes and `%2e%2e` traversal resolve-then-deny | +| `data:`, `blob:`, `qrc:` | needs `LOCAL_CONTENT` | else deny | +| `about:` | needs `LOCAL_CONTENT`; only literal `about:blank` | `about:config` denied even with the cap | +| `devtools:` | needs `DEVTOOLS` | | +| anything else (`javascript:`, `custom:`, …) | always deny "URL scheme is not authorized" | | +| invalid/relative/scheme-less | always deny | | + +### 8.2 Capability bits (`jerboa_browser.h`) + +`NETWORK=1, CLIPBOARD=2, FILESYSTEM=4, DOWNLOADS=8, PERSISTENT_STORAGE=16, +DEVTOOLS=32, POPUPS=64, LOCAL_CONTENT=128, MEDIA=256`. FILESYSTEM additionally +requires `JERBOA_BROWSER_FILESYSTEM_ROOT` to be an existing, absolute, +non-symlink directory, canonicalized at context creation. + +### 8.3 Other frozen behaviors + +- **Handles:** 0 never valid; kinds tagged (context/view/window); monotonic, + never reused; context release rejected while any child view lives; + double-free rejected as `JWB_ERR_INVALID_HANDLE`. +- **TLS:** fail-closed; per-host grants RAM-only; `once` grants consumed on + use; last-denied record format `HOST\tDESCRIPTION\tOVERRIDABLE\tURL\tDER_B64,…`. +- **Permissions:** deny by default; one-shot grants keyed `ORIGIN\tTYPE`; + honored only if the context's caps also allow (MEDIA for capture types, + CLIPBOARD for clipboard-read-write). Type names: `media-audio`, + `media-video`, `media-audio-video`, `desktop-video`, + `desktop-audio-video`, `mouse-lock`, `notifications`, `geolocation`, + `clipboard-read-write`, `local-fonts`. +- **Downloads:** denied (recorded as cancelled) without DOWNLOADS; snapshot + rows `ID\tSTATE\tRECEIVED\tTOTAL\tFILENAME\tURL[\tINTERRUPT_REASON]` with + TAB/CR/LF flattened to spaces; states `requested|in-progress|completed| + cancelled|interrupted|unknown`. +- **Adblock:** process-global, default ON; seed list = the 33 `||host^` rules + in `browser_policy.cpp`; external lists `easylist.txt` + `easyprivacy.txt` + from `JERBOA_ADBLOCK_LIST_DIR` or `$XDG_DATA_HOME/jerboa-browser/lists` + (fallback `$HOME/.local/share/...`), each capped at 16 MiB; naive resolver + (whole host = domain, start=0); third-party = last-two-labels mismatch. +- **Secure memory:** page-rounded; one `PROT_NONE` guard page each side; + `mlock`; `MADV_DONTDUMP`/`MADV_DONTFORK` when available; volatile wipe on + free; random fill from `/dev/urandom` with `O_CLOEXEC`. +- **Key capture:** every KeyPress queued except bare modifiers + (Control/Shift/Alt/Meta/AltGr/key==0); poll returns `KEY\tMODS\tTEXT` or `""`; + insert-mode probe + Escape/`C-g` blur-and-quit chord behavior as in + `browser_window.cpp` (`insert_escape_key`, `kEditableProbeJs`). + +--- + +## 9. Handoff implementation plan — Option A (for a lower-tier model) + +**Read first, in this order:** `AGENTS.md` (repo root — the `.ss` editing rules +are mandatory), `docs/ffi-boundary.md`, this document, `scheme/browser.ss` +header comment. Keep `include/jerboa_browser.h` and the `jwb_*` ABI **byte +compatible** throughout: the Scheme wrapper and all tests depend on it. + +**Mandatory working rules (from AGENTS.md, restated):** +1. NEVER edit `*.ss`/`*.sls` with `edit`/`write`/`sed`/`python`. Use + `jerboa_balanced_insert`, `jerboa_balanced_replace` (`dry_run:false` to + write), or `jerboa_write_file` (`verify:true`) for new files. +2. After every `.ss` change run `jerboa_check_balance`, then `jerboa_verify` + before building. If a file becomes unbalanced: STOP; `git checkout -- <file>` + or `jerboa_repair_balance`; never count parens by hand. +3. Before writing any Jerboa code: `jerboa_howto` search → + `jerboa_module_exports` / `jerboa_function_signature` to confirm APIs → + write → `jerboa_verify` → `jerboa_security_scan` for FFI/file-I/O code. +4. Keep closer-runs ≤4; flatten with helpers and `let*`. Internal `def`s must + precede expressions in a body. +5. Per repo pre-commit policy on macOS: `make binary` must succeed. Run + `make test` plus the affected suites (`make test-securestore`, + `make test-gui`, `make test-adblock`) before claiming any ticket done. +6. Do not touch sibling repos, `scheme/browser/*.ss` beyond binding updates + (R5), or the Qt build pipeline. +7. Save non-trivial discoveries with `jerboa_howto_add` / + `jerboa_error_fix_add` when you hit them (AGENTS.md "Save What You Learn"). + +**Stop conditions (any ticket):** a golden test fails and you cannot explain +why within one iteration → revert the ticket's changes and report; a `jwb_*` +signature change seems required → STOP and report (the ABI is frozen); a +callback-from-native-into-Scheme design appears tempting → STOP (R2: it is out +of scope for Option A; use push/poll + snapshots). + +### T0 — Golden-test freeze (1–2 d) + +Port the four ctest suites' *expectations* into Scheme-runnable golden tests +**before deleting anything**; the C++ stays as the reference oracle. +- Create `test/golden/url-policy.ss` asserting §8.1 row-by-row (drive the + *existing native* `jwb_load` against a hermetic context — see + `scheme/browser-test.ss` for the harness pattern; `JWB_TEST_NO_NETWORK=1`). +- Create `test/golden/handles.ss` for §8.3 handle rules (double-free, + kind-mixing, context-with-live-view). +- Transcribe §8.3 download/TLS/permission record formats into test constants. +- **Done when:** new suites pass against the unmodified native backend; + `make test` green. + +### T1 — Secure memory to libc FFI (1–2 d) — *first deletion* + +- Rewrite `scheme/std/crypto/secure-mem.ss` to bind `mmap`, `mprotect`, + `mlock`, `munlock`, `madvise`, `munmap`, `open`, `read`, `close` directly + from libc via `define-c-lambda` (load the C library with + `load-shared-object*` on the platform's libc path — check + `jerboa_howto` for an existing libc-load recipe first). +- Reproduce §8.3 secure-memory semantics exactly, including page rounding and + the two guard pages. `sysconf(_SC_PAGESIZE)` via FFI, fallback 4096. +- Remove the six `jwb_secure_*` functions from + `qt-webengine/src/browser_ffi.cpp` and their declarations from + `include/jerboa_browser.h`. Bump nothing — the symbols were internal to the + shim's consumer, which is the file you just rewrote. +- **Done when:** `make test-securestore` and `make test` pass; `nm`/`otool` + shows no `jwb_secure_*` exports; `jerboa_security_scan` clean on the new FFI. + +### T2 — UTF-8 validation + handle registry in Scheme (1–2 d) + +- Add `scheme/browser/native/utf8.ss`: strict validator reproducing + `browser_utf8.cpp` (overlong/surrogate/>U+10FFFF/truncated rejection), with + property tests comparing against the native validator *before* T3 removes + reliance on it. +- Add `scheme/browser/native/handles.ss`: kind-tagged, monotonic, + parent-tracking registry mirroring §8.3 (wrap the still-native handles; + the C++ registry stays as the second line of defense — do **not** delete + `browser_handles.cpp` in this ticket). +- **Done when:** property tests pass (≥10k random inputs, including crafted + invalid sequences); `make test` green. + +### T3 — Adblock configuration in Scheme (2–3 d) + +- Move seed list, list-directory discovery (`JERBOA_ADBLOCK_LIST_DIR` → XDG → + HOME fallback), 16 MiB caps, and byte counters into + `scheme/browser/adblock-config.ss`. The assembled rules blob is passed to + the native side through one new *additive* ABI call + (`jwb_adblock_set_rules(const char*)`) — additive change to the header is + permitted here; flag it in the ticket report. +- Optionally bind `engine_create`/`engine_match`/`engine_destroy`/ + `c_char_buffer_destroy` directly for an offline `adblock-check` test tool + (never in the request hot path — R2 does not apply; this is Scheme→C calls + only, no callbacks into Scheme; the native resolver stays). +- Delete the corresponding `QFile`/`QDir`/env logic from `browser_policy.cpp`. +- **Done when:** `make test-adblock` green (updated to Scheme-driven config); + external-rule-bytes counter matches golden value. + +### T4 — Policy snapshot push-down (3–4 d) — *hardest ticket; read R1 first* + +- Define a versioned, serialized policy snapshot (caps, scheme allowlist, + canonical fs root, TLS host grants, permission one-shot grants) authored in + Scheme (`scheme/browser/policy.ss` implementing §8.1/§8.3 as pure + predicates *for authoring and preflight*), pushed to the tap on every + change via one additive ABI call; tap applies it mechanically in + `interceptRequest` / `acceptNavigationRequest` / TLS / permission handlers. +- Tap exposes the applied generation; Scheme asserts it post-push. Tests must + include push-during-load ordering (R1). +- Delete the `QString` decision logic from `browser_url_policy.cpp` / + `browser_view.cpp`, leaving the mechanical matcher. +- **Done when:** T0 golden suites pass unchanged; the four ctest suites still + pass (they exercise the tap through the same ABI); `make verify` green. + +### T5 — Downloads + TLS/permission bookkeeping as events (2–3 d) + +- Generalize the key-queue pattern: tap emits `DOWNLOAD`, `TLS_DENIED`, + `PERM_DENIED` events onto one queue; `jwb_window_poll_key`-style drain from + the existing Scheme run loop; records/formatters in Scheme per §8.3. +- **Done when:** `browser-downloads-snapshot`, `browser-tls-last-error`, and + permission surfaces return byte-identical golden records; `make test-gui` + green. + +### T6 — Registry sole-ownership + facade removal (1 d) + +- Retire the C++ facade string buffers (T1–T5 leave no consumers); make the + Scheme registry from T2 the primary owner and reduce `browser_handles.cpp` + to the tap's internal pointer table (or delete it if nothing native remains + a client). +- **Done when:** `browser_ffi.cpp` is empty or deleted from CMake; ABI surface + unchanged; full `make test` + `make verify` green. + +### T7 — Documentation + evidence (1 d) + +- Update `docs/ffi-boundary.md`, `README.md` (Layout section), `AGENTS.md` if + any referenced paths moved, and regenerate release evidence + (`make release-evidence`) noting the reduced C++ surface. +- **Done when:** docs build references match reality; `make binary` clean from + a fresh clone-equivalent tree. + +--- + +## 10. Appendix — current `jwb_*` ABI surface (frozen reference) + +~60 entry points in five groups (see `include/jerboa_browser.h`): +runtime/version (`jwb_runtime_init`, `jwb_version`, `jwb_last_error`, +`jwb_last_status`); contexts/views (`jwb_context_{new,free,open}`, +`jwb_view_{new,free,open}`); navigation + JS (`jwb_load_url[_sync]`, +`jwb_{back,forward,reload,stop}`, `jwb_eval_js[_sync]`, `jwb_eval`, +`jwb_load`, observers `jwb_{title,current_url}[_sync]`, getters +`jwb_get_{title,url}`); event loop (`jwb_pump_events`, `jwb_pump_wait`, +`jwb_exec`, `jwb_quit`); pane/window chrome (`jwb_view_{show,hide,focus, +set_title,resize,grab_png,set_zoom,get_zoom,find}`, `jwb_window_*` 19 calls); +clipboard/zoom/find; TLS (`jwb_tls_*`); permissions (`jwb_permission_*`); +secure memory (`jwb_secure_*` — deleted by T1); adblock (`jwb_adblock_*`); +downloads (`jwb_downloads_*`). Permitted additive calls during the port: +`jwb_adblock_set_rules` (T3), one policy-snapshot push + one generation query +(T4), one event-queue drain (T5). Nothing else; no signature may change.