Harden remote tools and native dependencies
ober
1672a4207c0471407de9d740053334f6b04901d5
--- a/.build.yml +++ b/.build.yml @@ -13,7 +13,6 @@ packages: - zip sources: - https://git.sr.ht/~lisp/jerboa-code - - https://git.sr.ht/~lisp/jerboa artifacts: - jerboa-code/jerboa-code-0.1.0.jpkg tasks: @@ -23,8 +22,16 @@ tasks: # (std regex), (std net request) etc. binds the lib's symbols the moment # the library is visited. debian/stable ships no Rust toolchain, so # install one via rustup and persist it onto PATH for every later task. - curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain stable + curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --profile minimal --default-toolchain 1.88.0 echo 'source "$HOME/.cargo/env"' >> ~/.buildenv + - fetch-jerboa: | + . "$HOME/jerboa-code/vendor-lock.env" + git init -q "$HOME/jerboa" + git -C "$HOME/jerboa" remote add origin "$JERBOA_REPOSITORY" + git -C "$HOME/jerboa" fetch -q --depth 1 --filter=blob:none origin "$JERBOA_COMMIT" + git -C "$HOME/jerboa" checkout -q --detach FETCH_HEAD + test "$(git -C "$HOME/jerboa" rev-parse HEAD)" = "$JERBOA_COMMIT" + test "$(git -C "$HOME/jerboa" rev-parse 'HEAD^{tree}')" = "$JERBOA_TREE" - build-jerboa-tools: | # Build the current multicall tool so CI exercises the same # jerboa/jerbuild/jpkg image users install. @@ -42,14 +49,10 @@ tasks: "$J" pkg verify --reproduce - vendor-code: | cd jerboa-code - mkdir -p vendor - [ -d vendor/jerboa-sqlite ] || git clone --depth 1 https://git.sr.ht/~lisp/jerboa-sqlite vendor/jerboa-sqlite - [ -d vendor/jsqlite ] || git clone --depth 1 https://git.sr.ht/~lisp/jsqlite vendor/jsqlite - [ -d vendor/termbox2 ] || git clone --depth 1 https://github.com/termbox/termbox2.git vendor/termbox2 - [ -d vendor/jerboa-websearch ] || git clone --depth 1 https://git.sr.ht/~lisp/jerboa-websearch vendor/jerboa-websearch + make vendor-deps - build-code: | cd jerboa-code - env -u JERBOA_HOME JSQLITE_DIR="$PWD/vendor/jsqlite" make build + env -u JERBOA_HOME make build - test-code: | cd jerboa-code - env -u JERBOA_HOME JSQLITE_DIR="$PWD/vendor/jsqlite" make test + env -u JERBOA_HOME make test --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -14,7 +14,7 @@ jobs: runs-on: ubuntu-latest timeout-minutes: 60 steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - name: Install system dependencies run: | @@ -23,10 +23,11 @@ jobs: sudo apt-get install -y build-essential git curl ca-certificates ripgrep pkg-config libncurses-dev uuid-dev - name: Install Rust - uses: dtolnay/rust-toolchain@stable + # Pinned action commit for the Rust 1.88.0 branch. + uses: dtolnay/rust-toolchain@4e529fb27e59237866a6523e61ab248308c068b4 # gitsafe:ignore - name: Install cargo-audit - run: cargo install cargo-audit --locked + run: cargo install cargo-audit --version 0.22.2 --locked - name: Build and test run: make test --- a/.github/workflows/security-baseline.yml +++ b/.github/workflows/security-baseline.yml @@ -13,7 +13,7 @@ jobs: baseline: runs-on: ubuntu-latest steps: - - uses: actions/checkout@v4 + - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 - name: Required release files run: | @@ -33,3 +33,10 @@ jobs: echo "High-confidence secret pattern found." exit 1 fi + + - name: Application credential gate + run: | + set -eu + test ! -e .repl-token + ! git ls-files --error-unmatch .repl-token >/dev/null 2>&1 + ! git grep -n -I -E '(compile-time-token|JCODE_REPL_TOKEN|REPL_TOKEN_FILE)' -- Makefile src support scripts '.github/workflows/ci.yml' --- a/.gitignore +++ b/.gitignore @@ -15,6 +15,7 @@ jcode.json /jcode_tui_shim.dylib /jcode_tui_shim.so /dist/ +/.test-runtime/ # Android build outputs /android/android.jar @@ -41,4 +42,3 @@ jcode.json /jcode-linux-amd64-main.c /jcode-linux-arm64 /jcode-linux-arm64-main.c -.repl-token --- a/.jerbuild +++ b/.jerbuild @@ -3,18 +3,19 @@ ;; jerbuild bundles Chez Scheme + the jerboa stdlib, so NO jerboa source ;; checkout or external Chez is required. The only external input is the native ;; Rust lib (libjerboa_native), built from vendor/jerboa-native-rs with the -;; features jcode needs (tls + crypto). SQLite session storage comes from the -;; pure Scheme jsqlite library in ../jsqlite/src. +;; features jcode needs (tls + crypto + wasm). SQLite session storage comes from the +;; pure Scheme jsqlite library fetched at its locked commit under vendor/. (entry "main-binary.ss") (output "jcode") ;; do-binary-build auto-appends the bundled stdlib; list only project + vendor. (libdirs "lib" - "../jsqlite/src" + "vendor/jsqlite/src" + "vendor-overrides/jerboa-websearch/src" "vendor/jerboa-websearch/src") ;; Symbols statically registered via Sforeign_symbol (termbox TUI shim and -;; jerboa-native tls+crypto). The custom main.c #includes + calls them. +;; jerboa-native tls+crypto+wasm). The custom main.c #includes + calls them. (ffi-symbols "support/ffi-symbols.list") ;; Stock jerbuild main.c + setenv(JERBOA_STATIC=1); see the file's header. @@ -30,5 +31,5 @@ ;; runs cargo and links the resulting libjerboa_native.a. (rust-crates ("vendor/jerboa-native-rs/Cargo.toml" - features: "tls,crypto" + features: "tls,crypto,wasm" no-default-features: #t)) --- a/.jerbuild.freebsd-amd64 +++ b/.jerbuild.freebsd-amd64 @@ -16,6 +16,7 @@ (libdirs "lib" "vendor/jerboa-sqlite/lib" + "vendor-overrides/jerboa-websearch/src" "vendor/jerboa-websearch/src") (ffi-symbols "support/ffi-symbols.list") @@ -32,5 +33,5 @@ (rust-crates ("vendor/jerboa-native-rs/Cargo.toml" - features: "tls,crypto" + features: "tls,crypto,wasm" no-default-features: #t)) --- a/.jerbuild.linux-amd64 +++ b/.jerbuild.linux-amd64 @@ -14,6 +14,7 @@ (libdirs "lib" "vendor/jerboa-sqlite/lib" + "vendor-overrides/jerboa-websearch/src" "vendor/jerboa-websearch/src") (ffi-symbols "support/ffi-symbols.list") @@ -30,5 +31,5 @@ (rust-crates ("vendor/jerboa-native-rs/Cargo.toml" - features: "tls,crypto" + features: "tls,crypto,wasm" no-default-features: #t)) --- a/.jerbuild.linux-arm64 +++ b/.jerbuild.linux-arm64 @@ -14,6 +14,7 @@ (libdirs "lib" "vendor/jerboa-sqlite/lib" + "vendor-overrides/jerboa-websearch/src" "vendor/jerboa-websearch/src") (ffi-symbols "support/ffi-symbols.list") @@ -30,5 +31,5 @@ (rust-crates ("vendor/jerboa-native-rs/Cargo.toml" - features: "tls,crypto" + features: "tls,crypto,wasm" no-default-features: #t)) --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ # jerbuild is the self-contained build tool: it bundles Chez Scheme + the # jerboa stdlib under ~/.cache/jerbuild/<sha>/, so the host dev loop and the -# native `binary` need NO jerboa source checkout and NO separately-built Chez. +# native `binary` need NO mutable dependency checkout and NO separately-built Chez. JERBOA_VERSION ?= v0.2.3 JERBOA_TOOL_DIR ?= $(CURDIR)/.jerboa/bin JERBUILD ?= $(shell if [ -x ./jerbuild ]; then echo ./jerbuild; \ @@ -14,27 +14,28 @@ JERBUILD ?= $(shell if [ -x ./jerbuild ]; then echo ./jerbuild; \ # asking the jerbuild binary where it unpacked its bundle. JH = $(shell "$(JERBUILD)" --jerboa-home 2>/dev/null) -JSQLITE_DIR ?= $(HOME)/mine/jsqlite +JSQLITE_DIR ?= $(CURDIR)/vendor/jsqlite JSQLITE_LIBDIR ?= $(JSQLITE_DIR)/src -LIBDIRS = --libdirs ./lib:$(JSQLITE_LIBDIR):vendor/jerboa-websearch/src:$(JH)/lib +WEBSEARCH_OVERLAY := $(CURDIR)/vendor-overrides/jerboa-websearch +WEBSEARCH_WORKER := $(WEBSEARCH_OVERLAY)/bin/jerbsearch-engine-worker +LIBDIRS = --libdirs ./lib:$(JSQLITE_LIBDIR):$(WEBSEARCH_OVERLAY)/src:vendor/jerboa-websearch/src:$(JH)/lib JEXEC = $(JERBUILD) exec $(LIBDIRS) SBOM_DIR ?= dist/sbom REPRO_DIR ?= dist/reproducibility TARGET_EVIDENCE_DIR ?= dist/target-evidence RELEASE_EVIDENCE_DIR ?= dist/release-evidence -# Library paths for FFI shared objects (macOS: dylib, Linux: so). The native -# Rust lib now lives in ./lib (dropped by the native-rs target), matching the -# (std crypto native-rust) loader's CWD-relative "lib/" fallback. +# Exact development inputs for FFI shared objects. Source-mode runs opt into +# each canonical file explicitly; ambient dyld/ld paths and CWD searches are +# intentionally not part of the loader contract. TUI_SHIM_DIR := $(CURDIR)/vendor/termbox2 NATIVE_LIB_DIR := $(CURDIR)/lib -LDPATH := $(TUI_SHIM_DIR):$(NATIVE_LIB_DIR) # Native Rust lib (vendored crate, built with cargo). Only the features jcode -# uses — tls + crypto; no duckdb/pcap/postgres. +# uses — tls + crypto + wasm; no duckdb/pcap/postgres. NATIVE_DIR := vendor/jerboa-native-rs -NATIVE_FEATURES ?= tls,crypto +NATIVE_FEATURES ?= tls,crypto,wasm comma := , NATIVE_FEATURES_TAG := $(subst $(comma),-,$(NATIVE_FEATURES)) NATIVE_A := $(NATIVE_DIR)/target/release/libjerboa_native.a @@ -47,13 +48,22 @@ NATIVE_SENTINEL := $(NATIVE_DIR)/target/release/.built-with-$(NATIVE_FEATURES_TA UNAME_S := $(shell uname -s) ifeq ($(UNAME_S),Darwin) JCODE_OS_LIBS := -lm -lpthread -lncurses -liconv -lc++ -framework Security -framework CoreFoundation +JCODE_DEV_NATIVE_LIB := $(NATIVE_LIB_DIR)/libjerboa_native.dylib +JCODE_DEV_TUI_LIB := $(TUI_SHIM_DIR)/jcode_tui_shim.dylib else ifeq ($(UNAME_S),FreeBSD) JCODE_OS_LIBS := -lm -lpthread -lutil -lncurses -L/usr/local/lib -liconv +JCODE_DEV_NATIVE_LIB := $(NATIVE_LIB_DIR)/libjerboa_native.so +JCODE_DEV_TUI_LIB := $(TUI_SHIM_DIR)/jcode_tui_shim.so else JCODE_OS_LIBS := -lm -ldl -lpthread -luuid -lncurses -lstdc++ +JCODE_DEV_NATIVE_LIB := $(NATIVE_LIB_DIR)/libjerboa_native.so +JCODE_DEV_TUI_LIB := $(TUI_SHIM_DIR)/jcode_tui_shim.so endif +JCODE_DEV_NATIVE_ENV = JERBOA_DEV_NATIVE=1 \ + JERBOA_NATIVE_LIB="$(JCODE_DEV_NATIVE_LIB)" \ + JCODE_TUI_DEV_NATIVE=1 JCODE_TUI_LIB="$(JCODE_DEV_TUI_LIB)" -.PHONY: all help ensure-jerboa-tools build gen run test security audit security-audit verify sbom target-evidence reproducibility-report release-evidence test-providers local-eval clean repl binary install tui-shim run-tui native-rs linux linux-check linux-amd64 linux-arm64 jcode-linux-amd64 jcode-linux-arm64 test-linux test-linux-amd64 freebsd freebsd-amd64 jcode-freebsd-amd64 purge-stale android android-clean vendor-deps vendor-clean +.PHONY: all help ensure-jerboa-tools build gen run test test-websearch-worker test-websearch-packaged test-tui-native-loader-security security audit security-audit verify sbom target-evidence reproducibility-report release-evidence test-providers local-eval clean repl binary install tui-shim run-tui native-rs linux linux-check linux-amd64 linux-arm64 jcode-linux-amd64 jcode-linux-arm64 freebsd freebsd-amd64 jcode-freebsd-amd64 purge-stale android android-clean vendor-deps vendor-provenance-check vendor-clean all: help @@ -73,7 +83,7 @@ help: @echo " run-tui Start TUI mode" @echo " repl Open a bare Scheme REPL with project libdirs" @echo " gen Transpile src/ → lib/ (jerbuild)" - @echo " native-rs Build vendored libjerboa_native (cargo, tls+crypto)" + @echo " native-rs Build vendored libjerboa_native (cargo, tls+crypto+wasm)" @echo " binary Build native binary via 'jerbuild build' (.jerbuild)" @echo " install Install binary to ~/.local/bin" @echo " binary Build jcode for THIS host's os/arch (jerbuild, native)" @@ -131,6 +141,10 @@ purge-stale: break; \ fi; \ done + @find "$(WEBSEARCH_OVERLAY)/src" \( -name '*.so' -o -name '*.wpo' \) 2>/dev/null | while read f; do \ + src=$${f%.*}.ss; \ + if [ ! -f "$$src" ] || [ "$$src" -nt "$$f" ]; then rm -f "$$f"; fi; \ + done # ── libjerboa_native (Rust FFI) ───────────────────────────────────────────── # secrets.ss + std/net/request + std/regex etc. all bind FFI symbols from @@ -140,17 +154,17 @@ purge-stale: # (std crypto native-rust) is visited. # # We vendor the crate (sparse-clone from the jerboa monorepo) and build it -# with cargo using ONLY the features jcode needs (tls + crypto) — no +# with cargo using ONLY the features jcode needs (tls + crypto + wasm) — no # dependency on a ~/mine/jerboa checkout. The same crate + features feed the # static `binary` build via (rust-crates ...) in .jerbuild. -native-rs: $(NATIVE_DIR) +native-rs: $(NATIVE_DIR) vendor-provenance-check @command -v cargo >/dev/null 2>&1 || { \ echo "ERROR: cargo not found on PATH. Install rustup from rustup.rs"; exit 1; } @if [ ! -f "$(NATIVE_A)" ] || \ [ ! -f "$(NATIVE_SENTINEL)" ] || \ [ -n "$$(find $(NATIVE_DIR)/src -name '*.rs' -newer $(NATIVE_A) 2>/dev/null)" ]; then \ echo "=== Building libjerboa_native (features: $(NATIVE_FEATURES)) ==="; \ - ( cd $(NATIVE_DIR) && cargo build --release --no-default-features --features $(NATIVE_FEATURES) ); \ + ( cd $(NATIVE_DIR) && cargo build --locked --release --no-default-features --features $(NATIVE_FEATURES) ); \ rm -f $(NATIVE_DIR)/target/release/.built-with-*; \ touch "$(NATIVE_SENTINEL)"; \ fi @@ -166,76 +180,89 @@ native-rs: $(NATIVE_DIR) fi # ─── Vendor Dependencies ───────────────────────────────────────────────────── -# Clone dependency repos into vendor/ at build time. vendor/ is .gitignored; -# we do NOT use git submodules — every target that needs vendor/<x> depends on -# vendor-deps so a fresh checkout populates itself. +# Populate dependency repos only from the immutable commit/tree identities in +# vendor-lock.env. No build target resolves or checks out a moving branch. + +vendor-deps: vendor/termbox2 vendor/jerboa-websearch vendor/jsqlite + @JCODE_VENDOR_COMPONENT=source sh scripts/vendor-provenance-check.sh >/dev/null -vendor-deps: vendor/termbox2 vendor/jerboa-websearch +vendor/termbox2: vendor-lock.env scripts/fetch-vendor.sh + @echo "=== Fetching locked termbox2 source ===" + @sh scripts/fetch-vendor.sh termbox2 -vendor/termbox2: - @mkdir -p vendor - @echo "=== Cloning termbox2 (upstream) into vendor/ ===" - @git clone --depth 1 https://github.com/termbox/termbox2.git vendor/termbox2 +vendor/jerboa-websearch: vendor-lock.env scripts/fetch-vendor.sh + @echo "=== Fetching locked jerboa-websearch source ===" + @sh scripts/fetch-vendor.sh websearch -vendor/jerboa-websearch: - @mkdir -p vendor - @echo "=== Cloning jerboa-websearch into vendor/ ===" - @git clone --depth 1 https://git.sr.ht/~lisp/jerboa-websearch vendor/jerboa-websearch +vendor/jsqlite: vendor-lock.env scripts/fetch-vendor.sh + @echo "=== Fetching locked jsqlite source ===" + @sh scripts/fetch-vendor.sh jsqlite # Sparse-clone just the jerboa-native-rs crate out of the jerboa monorepo. # Depended on only by native-rs (not vendor-deps), so cross targets — which # build their own target-triple native lib — don't pull it. -$(NATIVE_DIR): - @mkdir -p vendor - @echo "=== Sparse-cloning jerboa-native-rs from the jerboa monorepo ===" - @git clone --depth 1 --filter=blob:none --sparse https://git.sr.ht/~lisp/jerboa vendor/.jerboa-monorepo - @git -C vendor/.jerboa-monorepo sparse-checkout set jerboa-native-rs - @mv vendor/.jerboa-monorepo/jerboa-native-rs $(NATIVE_DIR) - @rm -rf vendor/.jerboa-monorepo +$(NATIVE_DIR): vendor-lock.env scripts/fetch-vendor.sh + @echo "=== Fetching locked jerboa-native-rs source ===" + @sh scripts/fetch-vendor.sh native + +vendor-provenance-check: vendor-deps $(NATIVE_DIR) + @sh scripts/vendor-provenance-check.sh vendor-clean: rm -rf vendor -# ── Debug-REPL auth token ──────────────────────────────────────────────────── -# Baked into the binary at compile time (compile-time getenv in debug-repl.ss); -# required to connect when --repl-port binds a non-loopback IP. Generated once -# per checkout; delete .repl-token (+ make clean) to rotate. -REPL_TOKEN_FILE ?= .repl-token -$(REPL_TOKEN_FILE): - @head -c 16 /dev/urandom | od -An -tx1 | tr -d ' \n' > $@ - @echo "Generated debug-REPL auth token in $(REPL_TOKEN_FILE)" - -gen: ensure-jerboa-tools vendor-deps purge-stale $(REPL_TOKEN_FILE) - JCODE_REPL_TOKEN=$$(cat $(REPL_TOKEN_FILE)) $(JERBUILD) transpile src lib +gen: ensure-jerboa-tools vendor-deps purge-stale + $(JERBUILD) transpile src lib build: ensure-jerboa-tools gen native-rs - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + $(JCODE_DEV_NATIVE_ENV) \ $(JERBUILD) compile $(LIBDIRS) support/build-check.ss < /dev/null run: ensure-jerboa-tools native-rs - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) main.ss tui-shim: vendor/termbox2 - @test -f $(TUI_SHIM_DIR)/jcode_tui_shim.dylib \ - -o -f $(TUI_SHIM_DIR)/jcode_tui_shim.so || \ + @if [ ! -f "$(JCODE_DEV_TUI_LIB)" ] || \ + [ src/jcode/ui/jcode_tui_shim.c -nt "$(JCODE_DEV_TUI_LIB)" ]; then \ cc -shared -fPIC -DTB_OPT_ATTR_W=32 \ -I$(TUI_SHIM_DIR) \ - -o $(TUI_SHIM_DIR)/jcode_tui_shim.dylib \ - src/jcode/ui/jcode_tui_shim.c + -o "$(JCODE_DEV_TUI_LIB)" \ + src/jcode/ui/jcode_tui_shim.c; \ + fi run-tui: ensure-jerboa-tools build tui-shim native-rs - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) main.ss --tui repl: ensure-jerboa-tools - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) support/repl.ss -test: build - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ +test: build test-websearch-worker test-tui-native-loader-security + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ + sh test/security-regression.sh $(JEXEC) test/security-regression.ss + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) test/run.ss +test-websearch-worker: build + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ + $(JEXEC) test/websearch-worker-failclosed.ss + JERBSEARCH_ENGINE_WORKER="$(CURDIR)/test/fixtures/websearch-engine-worker" \ + $(JCODE_DEV_NATIVE_ENV) \ + $(JEXEC) test/websearch-search-limits.ss + +test-websearch-packaged: binary + sh test/websearch-worker-binary.sh ./jcode + +test-tui-native-loader-security: ensure-jerboa-tools gen tui-shim + JERBUILD="$(JERBUILD)" sh test/tui-native-loader-security.sh + security: security-audit audit: security-audit @@ -284,9 +311,10 @@ release-evidence: (rustc --version && cargo --version) > "$(RELEASE_EVIDENCE_DIR)/rust-version.txt" 2>&1 || true ./jcode --version > "$(RELEASE_EVIDENCE_DIR)/binary-version.txt" 2>&1 shasum -a 256 jcode > "$(RELEASE_EVIDENCE_DIR)/binary-sha256.txt" - find Makefile .jerbuild .jerboa/security.json main.ss main-binary.ss src support test docs tools .github \ + find Makefile .build.yml .jerbuild .jerboa/security.json main.ss main-binary.ss src support test docs tools .github \ -type f \( -name 'Makefile' -o -name '*.ss' -o -name '*.sls' -o -name '*.c' -o -name '*.h' -o -name '*.list' -o -name '*.sh' -o -name '*.json' -o -name '*.md' -o -name '*.yml' -o -name '*.yaml' \) \ -print 2>/dev/null | LC_ALL=C sort | xargs shasum -a 256 > "$(RELEASE_EVIDENCE_DIR)/release-inputs.sha256" + shasum -a 256 vendor-lock.env rust-toolchain.toml >> "$(RELEASE_EVIDENCE_DIR)/release-inputs.sha256" grep -q '^sbom_status=present$$' "$(RELEASE_EVIDENCE_DIR)/sbom/manifest.txt" grep -q '^status=match$$' "$(RELEASE_EVIDENCE_DIR)/reproducibility/result.txt" grep -Eq '^(status=local-status-recorded|status=target-proof-recorded-review-required)$$' "$(RELEASE_EVIDENCE_DIR)/target-evidence/status.txt" @@ -296,7 +324,8 @@ release-evidence: # Providers without a key are SKIPped, not failed. Excluded from `test` # because it makes real network calls. test-providers: build - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) test/run-providers.ss PROVIDER ?= mlx2 @@ -304,7 +333,8 @@ MODEL ?= /Users/user/models/qwen3-coder-next-mlx LOCAL_EVAL_JCODE ?= $(HOME)/.local/bin/jcode local-eval: build - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + JERBSEARCH_ENGINE_WORKER="$(WEBSEARCH_WORKER)" \ + $(JCODE_DEV_NATIVE_ENV) \ $(JEXEC) eval/local-model/run-local-eval.ss \ --provider "$(PROVIDER)" \ --model "$(MODEL)" \ @@ -316,8 +346,7 @@ local-eval: build # main-c, C shims, rust-crate), bundles Chez + stdlib, runs cargo for the native # .a, and links it all. Per-OS link flags come from --os-libs. binary: ensure-jerboa-tools gen native-rs - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ - JCODE_REPL_TOKEN=$$(cat $(REPL_TOKEN_FILE)) \ + $(JCODE_DEV_NATIVE_ENV) \ $(JERBUILD) build --config .jerbuild --os-libs "$(JCODE_OS_LIBS)" cp vendor/termbox2/jcode_tui_shim.dylib ./jcode_tui_shim.dylib 2>/dev/null || true cp vendor/termbox2/jcode_tui_shim.so ./jcode_tui_shim.so 2>/dev/null || true @@ -340,7 +369,7 @@ install: binary JERBOA_HOME ?= $(HOME)/mine/jerboa SCHEME ?= $(JERBOA_HOME)/.chez/bin/scheme FREEBSD_AMD64_CC ?= $(JERBOA_HOME)/support/cross-cc-freebsd-amd64 -XC_LIBDIRS = ./lib:$(JSQLITE_LIBDIR):vendor/jerboa-websearch/src:$(JH)/lib:$(JERBOA_HOME)/lib +XC_LIBDIRS = ./lib:$(JSQLITE_LIBDIR):$(WEBSEARCH_OVERLAY)/src:vendor/jerboa-websearch/src:$(JH)/lib:$(JERBOA_HOME)/lib # `make linux` defaults to the linux amd64 cross-build. linux: linux-amd64 @@ -351,7 +380,7 @@ linux: linux-amd64 # stdlib exports) in ~10s before burning minutes in the full cross build. linux-check: ensure-jerboa-tools gen @echo "=== Running linux-check (fast cross-build drift check) ===" - DYLD_LIBRARY_PATH=$(LDPATH) LD_LIBRARY_PATH=$(LDPATH) \ + $(JCODE_DEV_NATIVE_ENV) \ $(SCHEME) -q --libdirs "$(XC_LIBDIRS)" --script linux-check.ss # ─── Cross-compile: host → Linux x86_64 / arm64 (musl static) ─────────────── @@ -371,7 +400,7 @@ linux-amd64: ensure-jerboa-tools gen @command -v cargo >/dev/null 2>&1 || { \ echo "ERROR: cargo not found on PATH. Install rustup from rustup.rs"; \ exit 1; } - JERBOA_HOME=$(JERBOA_HOME) TARGET_ARCH=amd64 JCODE_REPL_TOKEN=$$(cat $(REPL_TOKEN_FILE)) $(SCHEME) -q --libdirs "$(XC_LIBDIRS)" --script build-jcode-cross.ss + JERBOA_HOME=$(JERBOA_HOME) TARGET_ARCH=amd64 $(SCHEME) -q --libdirs "$(XC_LIBDIRS)" --script build-jcode-cross.ss @ls -lh jcode-linux-amd64 @file jcode-linux-amd64 @@ -387,7 +416,7 @@ linux-arm64: ensure-jerboa-tools gen @command -v cargo >/dev/null 2>&1 || { \ echo "ERROR: cargo not found on PATH. Install rustup from rustup.rs"; \ exit 1; } - JERBOA_HOME=$(JERBOA_HOME) TARGET_ARCH=arm64 JCODE_REPL_TOKEN=$$(cat $(REPL_TOKEN_FILE)) $(SCHEME) -q --libdirs "$(XC_LIBDIRS)" --script build-jcode-cross.ss + JERBOA_HOME=$(JERBOA_HOME) TARGET_ARCH=arm64 $(SCHEME) -q --libdirs "$(XC_LIBDIRS)" --script build-jcode-cross.ss @ls -lh jcode-linux-arm64 @file jcode-linux-arm64 --- a/SECURITY.md +++ b/SECURITY.md @@ -37,8 +37,18 @@ must be cut from a clean checkout after: sensitive project content. Normal logs redact known secret shapes and no longer record full prompt or shell command bodies, but explicit trace output is still a sensitive diagnostic artifact. -- Remote `serve`, `relay`, and debug REPL endpoints require token handling and - bind-address review before use off-host. +- Remote `serve`, `relay`, `connect`, and debug-control transports are + loopback-only. Off-host use requires an authenticated SSH or mTLS tunnel; + tokens remain mandatory defense in depth. +- `web_search` never runs engine callbacks in the threaded agent process or a + raw-fork Scheme child. Source builds pin the reviewed worker launcher; + packaged binaries re-exec their own canonical image in a fixed hidden mode. + Requests and both output streams are bounded, deadlines terminate and reap + the worker process group, and HTML is parsed only by the embedded wasm + sandbox. Its TLS transport connects to the exact policy-approved numeric + address while retaining SNI/certificate verification. A missing or invalid + worker, parser, or reviewed native TLS overlay fails closed. Packaged mode + clears external parser/fallback overrides and requires its embedded sandbox. ## Release Gates @@ -49,7 +59,8 @@ must be cut from a clean checkout after: - Secret scanning: high-confidence working-tree scan and history review before first public release. - SBOM/provenance: release evidence records the Jerboa toolchain, generated - binary inputs, vendored native Rust crate, termbox2, websearch, jsqlite, and + binary inputs, vendored native Rust crate, termbox2, locked websearch source, + the reviewed Websearch worker overlay, jsqlite, and native shim artifacts. - Reproducibility: release evidence compares two clean project binary builds and requires a no-network `./jcode --version` smoke to pass. @@ -59,8 +70,10 @@ must be cut from a clean checkout after: incomplete. - Native code: review the Rust `jerboa-native-rs` dependency set, C shims, FFI lifetimes, allocation/freeing rules, and cross-build generated artifacts. -- Daemon-like features: remote server, relay, and debug REPL must retain token - auth, restrictive file permissions, and documented bind-address behavior. +- Daemon-like features: remote server, relay, and debug control must retain + loopback enforcement, token auth, restrictive file permissions, and + documented tunnel behavior. Debug control must also retain its line, time, + and concurrent-client limits. ## Reporting --- a/build-jcode-cross.ss +++ b/build-jcode-cross.ss @@ -72,7 +72,7 @@ ;; crypto — secrets.ss, hashing, hmac, aead, scrypt, argon2id ;; Session storage uses pure Scheme jsqlite from the libdirs; no sqlite C shim ;; or jerboa-native sqlite feature is linked here. -(define cargo-features "tls,crypto") +(define cargo-features "tls,crypto,wasm") (define (write-text-file! path proc) (call-with-port @@ -341,9 +341,10 @@ "jerboa_tls_server_new_mtls" "jerboa_tls_server_free" "jerboa_tls_accept" "jerboa_tls_connect" "jerboa_tls_connect_pinned" + "jerboa_tls_connect_timeout" "jerboa_tls_connect_addr_timeout" "jerboa_tls_connect_mtls" "jerboa_tls_close" "jerboa_tls_read" "jerboa_tls_write" "jerboa_tls_flush" - "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" + "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" "jerboa_tls_set_timeout" "jerboa_last_error" ;; jerboa-native (x509) — self-signed cert for the debug-REPL TLS listener "jerboa_x509_generate_self_signed_mem" @@ -356,6 +357,14 @@ "jerboa_scrypt" "jerboa_pbkdf2_derive" "jerboa_pbkdf2_verify" "jerboa_argon2id_hash" "jerboa_argon2id_verify" + ;; jerboa-native (wasmi) — embedded Websearch parser worker + "jerboa_wasm_module_new" "jerboa_wasm_module_free" + "jerboa_wasm_instance_new" "jerboa_wasm_instance_new_hosted" + "jerboa_wasm_instance_free" "jerboa_wasm_call" + "jerboa_wasm_memory_read" "jerboa_wasm_memory_write" + "jerboa_wasm_memory_size" "jerboa_wasm_add_fuel" + "jerboa_wasm_fuel_remaining" "jerboa_wasm_get_log" + "jerboa_wasm_allow_cdb_dir" "jerboa_wasm_set_socket" ;; jerboa-native (regex) — visit-time even if jcode never matches "jerboa_regex_compile" "jerboa_regex_is_match" "jerboa_regex_find" "jerboa_regex_replace_all" "jerboa_regex_free" @@ -448,7 +457,7 @@ (display "/* Shim function decls — jcode_tui_shim.c + landlock-shim.c */\n" out) (for-each (lambda (n) (fprintf out "extern void ~a();\n" n)) ffi-shim-symbols) ;; extern decls for libjerboa_native.a symbols - (display "\n/* libjerboa_native.a — features=tls,crypto */\n" out) + (display "\n/* libjerboa_native.a — features=tls,crypto,wasm */\n" out) (for-each (lambda (n) (fprintf out "extern void ~a();\n" n)) jerboa-native-symbols) ;; POSIX libc symbols already declared by system headers above. (display "\nextern int *__errno_location(void);\n" out) @@ -490,8 +499,7 @@ (display " Sscheme_init(NULL);\n" out) (display " Sregister_boot_file_bytes(\"petite\", (void *)petite_boot, petite_boot_size);\n" out) (display " Sregister_boot_file_bytes(\"scheme\", (void *)scheme_boot, scheme_boot_size);\n" out) - (display " Sbuild_heap(NULL, NULL);\n" out) - (display " register_ffi_symbols();\n\n" out) + (display " Sbuild_heap(NULL, register_ffi_symbols);\n\n" out) ;; argv is passed through directly to Sscheme_program — main-binary.ss ;; reads (command-line-arguments) for --tui detection. (display " int status = Sscheme_program(prog_path, argc, (const char **)argv);\n\n" out) --- a/build-jcode-freebsd-cross.ss +++ b/build-jcode-freebsd-cross.ss @@ -51,7 +51,7 @@ ;; crypto — secrets.ss, hashing, hmac, aead, scrypt, argon2id ;; Session storage uses pure Scheme jsqlite from the libdirs; no sqlite C shim ;; or jerboa-native sqlite feature is linked here. -(define cargo-features "tls,crypto") +(define cargo-features "tls,crypto,wasm") (define (write-text-file! path proc) (call-with-port @@ -328,9 +328,10 @@ "jerboa_tls_server_new_mtls" "jerboa_tls_server_free" "jerboa_tls_accept" "jerboa_tls_connect" "jerboa_tls_connect_pinned" + "jerboa_tls_connect_timeout" "jerboa_tls_connect_addr_timeout" "jerboa_tls_connect_mtls" "jerboa_tls_close" "jerboa_tls_read" "jerboa_tls_write" "jerboa_tls_flush" - "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" + "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" "jerboa_tls_set_timeout" "jerboa_last_error" ;; jerboa-native (x509) — self-signed cert for the debug-REPL TLS listener "jerboa_x509_generate_self_signed_mem" @@ -343,6 +344,14 @@ "jerboa_scrypt" "jerboa_pbkdf2_derive" "jerboa_pbkdf2_verify" "jerboa_argon2id_hash" "jerboa_argon2id_verify" + ;; jerboa-native (wasmi) — embedded Websearch parser worker + "jerboa_wasm_module_new" "jerboa_wasm_module_free" + "jerboa_wasm_instance_new" "jerboa_wasm_instance_new_hosted" + "jerboa_wasm_instance_free" "jerboa_wasm_call" + "jerboa_wasm_memory_read" "jerboa_wasm_memory_write" + "jerboa_wasm_memory_size" "jerboa_wasm_add_fuel" + "jerboa_wasm_fuel_remaining" "jerboa_wasm_get_log" + "jerboa_wasm_allow_cdb_dir" "jerboa_wasm_set_socket" ;; jerboa-native (regex) — visit-time even if jcode never matches "jerboa_regex_compile" "jerboa_regex_is_match" "jerboa_regex_find" "jerboa_regex_replace_all" "jerboa_regex_free" @@ -427,7 +436,7 @@ (display "\n/* Shim function decls — jcode_tui_shim.c */\n" out) (for-each (lambda (n) (fprintf out "extern void ~a();\n" n)) ffi-shim-symbols) ;; extern decls for libjerboa_native.a symbols - (display "\n/* libjerboa_native.a — features=tls,crypto */\n" out) + (display "\n/* libjerboa_native.a — features=tls,crypto,wasm */\n" out) (for-each (lambda (n) (fprintf out "extern void ~a();\n" n)) jerboa-native-symbols) ;; register all symbols at startup (newline out) @@ -470,8 +479,7 @@ (display " Sscheme_init(NULL);\n" out) (display " Sregister_boot_file_bytes(\"petite\", (void *)petite_boot, petite_boot_size);\n" out) (display " Sregister_boot_file_bytes(\"scheme\", (void *)scheme_boot, scheme_boot_size);\n" out) - (display " Sbuild_heap(NULL, NULL);\n" out) - (display " register_ffi_symbols();\n\n" out) + (display " Sbuild_heap(NULL, register_ffi_symbols);\n\n" out) ;; argv is passed through directly to Sscheme_program — main-binary.ss ;; reads (command-line-arguments) for --tui detection. (display " int status = Sscheme_program(prog_path, argc, (const char **)argv);\n\n" out) --- a/build-jcode-musl.sh +++ b/build-jcode-musl.sh @@ -20,6 +20,17 @@ JERBOA_LIB="${JERBOA_HOME}/lib" JSQLITE_DIR="${JSQLITE_DIR:-${HOME_DIR}/mine/jsqlite}" JSQLITE_LIBDIR="${JSQLITE_LIBDIR:-${JSQLITE_DIR}/src}" SCHEME="${SCHEME:-${JERBOA_HOME}/.chez/bin/scheme}" +HOST_NATIVE_LIB="${JERBOA_NATIVE_LIB:-${JERBOA_LIB}/libjerboa_native.so}" +TUI_NATIVE_LIB="${JCODE_TUI_LIB:-${SCRIPT_DIR}/vendor/termbox2/jcode_tui_shim.so}" + +case "${HOST_NATIVE_LIB}" in + /*) ;; + *) echo "ERROR: JERBOA_NATIVE_LIB must be an absolute path" >&2; exit 1 ;; +esac +case "${TUI_NATIVE_LIB}" in + /*) ;; + *) echo "ERROR: JCODE_TUI_LIB must be an absolute path" >&2; exit 1 ;; +esac if [ ! -x "${SCHEME}" ]; then echo "ERROR: Chez Scheme not found at ${SCHEME}" >&2 @@ -88,10 +99,13 @@ VALIDATE echo "" echo "[2/2] Running musl build..." -LD_LIBRARY_PATH="${SCRIPT_DIR}/vendor/termbox2:${JERBOA_LIB}:." \ JERBOA_HOME="${JERBOA_HOME}" \ +JERBOA_DEV_NATIVE=1 \ +JERBOA_NATIVE_LIB="${HOST_NATIVE_LIB}" \ +JCODE_TUI_DEV_NATIVE=1 \ +JCODE_TUI_LIB="${TUI_NATIVE_LIB}" \ JERBOA_MUSL_CHEZ_PREFIX="${MUSL_CHEZ_PREFIX}" \ -"${SCHEME}" -q --libdirs "${JERBOA_LIB}:./lib:${JSQLITE_LIBDIR}:vendor/jerboa-websearch/src" \ +"${SCHEME}" -q --libdirs "${JERBOA_LIB}:./lib:${JSQLITE_LIBDIR}:vendor-overrides/jerboa-websearch/src:vendor/jerboa-websearch/src" \ <build-jcode-musl.ss # Verify --- a/build-jcode-musl.ss +++ b/build-jcode-musl.ss @@ -1,7 +1,7 @@ #!chezscheme ;;; build-jcode-musl.ss — Build a fully static jcode binary using musl libc ;;; -;;; Usage: scheme -q --libdirs <jerboa-lib>:./lib:$HOME/mine/jsqlite/src:vendor/jerboa-websearch/src < build-jcode-musl.ss +;;; Usage: scheme -q --libdirs <jerboa-lib>:./lib:$HOME/mine/jsqlite/src:vendor-overrides/jerboa-websearch/src:vendor/jerboa-websearch/src < build-jcode-musl.ss ;;; ;;; This script: ;;; 1. Compiles jcode modules (using stock scheme with glibc) @@ -295,9 +295,10 @@ "jerboa_tls_server_new" "jerboa_tls_server_new_mtls" "jerboa_tls_server_free" "jerboa_tls_accept" "jerboa_tls_connect" "jerboa_tls_connect_pinned" + "jerboa_tls_connect_timeout" "jerboa_tls_connect_addr_timeout" "jerboa_tls_connect_mtls" "jerboa_tls_close" "jerboa_tls_read" "jerboa_tls_write" "jerboa_tls_flush" - "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" + "jerboa_tls_get_fd" "jerboa_tls_set_nonblock" "jerboa_tls_set_timeout" "jerboa_last_error" ;; jerboa-native (crypto/ring) — used by secrets.ss via (std crypto native-rust) "jerboa_sha1" "jerboa_sha256" "jerboa_sha384" "jerboa_sha512" @@ -308,6 +309,14 @@ "jerboa_scrypt" "jerboa_pbkdf2_derive" "jerboa_pbkdf2_verify" "jerboa_argon2id_hash" "jerboa_argon2id_verify" + ;; jerboa-native (wasmi) — embedded Websearch parser worker + "jerboa_wasm_module_new" "jerboa_wasm_module_free" + "jerboa_wasm_instance_new" "jerboa_wasm_instance_new_hosted" + "jerboa_wasm_instance_free" "jerboa_wasm_call" + "jerboa_wasm_memory_read" "jerboa_wasm_memory_write" + "jerboa_wasm_memory_size" "jerboa_wasm_add_fuel" + "jerboa_wasm_fuel_remaining" "jerboa_wasm_get_log" + "jerboa_wasm_allow_cdb_dir" "jerboa_wasm_set_socket" ;; jerboa-native (regex) — pulled in transitively via (jerboa prelude) ;; which imports (std regex). foreign-procedure forms run at library ;; visit time, so these have to exist even if jcode never matches a regex. @@ -434,8 +443,9 @@ ;; the cargo features jcode actually uses. jcode's FFI surface is: ;; - jerboa_tls_* (provider/provider.ss) => feature `tls` ;; - jerboa_crypto_* / hashing / secret-store => feature `crypto` -;; That's it. `duckdb_feat`, `pcap`, `spidermonkey`, `wasm`, and -;; `postgres_feat` are not referenced from any jcode source. +;; - jerboa_wasm_* (sandboxed Websearch HTML parser) => feature `wasm` +;; `duckdb_feat`, `pcap`, `spidermonkey`, and `postgres_feat` are not +;; referenced from any jcode source. ;; ;; The old fall-back-to-glibc behaviour was the recurring source of "works on ;; Mac, breaks on Linux": cargo build --release during normal jerboa dev @@ -445,14 +455,14 @@ ;; We force cargo to use the rustup-managed rustc — some PATH setups (e.g. ;; Homebrew rust ahead of the rustup proxies) resolve `rustc` to a binary ;; that has no musl target installed. Doing it via env (not `rustup run`) is -;; critical, because `rustup run stable cargo build` only configures cargo's +;; critical, because `rustup run 1.88.0 cargo build` only configures cargo's ;; environment — cargo still launches whichever `rustc` appears first on ;; PATH for the child compilations. ;; ;; A sentinel file under target/ records the exact feature set that produced ;; the .a, so future builds rebuild iff features changed or any .rs source ;; is newer than the .a (jerboa-shell uses the same pattern). -(define jcode-cargo-features "tls,crypto") +(define jcode-cargo-features "tls,crypto,wasm") (define native-lib-path (format "~a/jerboa-native-rs/target/x86_64-unknown-linux-musl/release/libjerboa_native.a" jerboa-dir-base)) @@ -492,8 +502,8 @@ (define (rebuild-native-lib!) (let* ([nrs-dir (format "~a/jerboa-native-rs" jerboa-dir-base)] [rustup-active (string-append (getenv "HOME") - "/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin")] - [cargo-args (format "build --release --no-default-features --features ~a --target x86_64-unknown-linux-musl" + "/.rustup/toolchains/1.88.0-x86_64-unknown-linux-gnu/bin")] + [cargo-args (format "build --locked --release --no-default-features --features ~a --target x86_64-unknown-linux-musl" jcode-cargo-features)] [via-rustup (format "cd '~a' && env PATH='~a:'$PATH RUSTC='~a/rustc' cargo ~a" --- a/docs/architecture.md +++ b/docs/architecture.md @@ -89,6 +89,7 @@ Everything lives under `~/.jcode/` (created on first run): | `last-session` | The session to resume on reconnect. | | `models-cache.json`, `pricing-cache.json` | Cached model + pricing metadata. | | `server-token`, `relay-token` | Auth tokens for [remote control](remote.md). | +| `repl-token`, `repl-port` | Per-start debug-control credential and endpoint (mode 0600; removed on shutdown). | | `themes/` | Custom TUI themes. | ## Build pipeline --- a/docs/cli.md +++ b/docs/cli.md @@ -85,6 +85,10 @@ jcode relay --port N [--bind ADDR] # rendezvous on a VPS jcode connect HOST:PORT --host NAME [--token T] # controller side ``` +All credential-bearing TCP addresses must be loopback (`127.0.0.1` or `::1`). +Use an authenticated SSH or mTLS local forward for every off-host connection; +see [remote.md](remote.md) for complete examples. + ### `verified` ``` --- a/docs/credential-handling.md +++ b/docs/credential-handling.md @@ -20,8 +20,11 @@ provider-native login state over plaintext project configuration. - The encrypted store is passphrase-protected and written with owner-only file permissions. - Remote server and relay tokens live under `~/.jcode/` and are written with - owner-only permissions. -- Debug REPL tokens are generated per checkout and are ignored by git. + owner-only permissions. Their TCP transports are loopback-only; remote use + requires an authenticated SSH or mTLS tunnel. +- The debug control token is generated with the CSPRNG on every explicit + debug-socket start, atomically published as `~/.jcode/repl-token` mode 0600, + and removed on shutdown. It is never an input to compilation. - Runtime caches, logs, and traces are excluded from the repository by `.gitignore`. @@ -38,7 +41,11 @@ provider-native login state over plaintext project configuration. ## Release Checks - Run `make audit` before release. -- Review `jcode.json`, `.jcode/`, `.repl-token`, logs, traces, and provider auth +- Review `jcode.json`, `.jcode/`, logs, traces, and provider auth examples before packaging. - Run a high-confidence secret scan over the working tree and review git history before first public release. +- The scanner explicitly rejects the legacy `.repl-token`, build-time REPL + embedding hooks, and raw application-token assignment shapes. This is useful + coverage, not proof that a tree or artifact contains no secret; manual review, + history review, and rotation remain required. --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -40,24 +40,21 @@ prerequisites, building on each platform, configuration, and your first session. ## Vendored dependencies -Two repos are cloned into `vendor/` at build time (the directory is git-ignored -— these are **not** submodules). Session storage uses the sibling `jsqlite` -checkout at `~/mine/jsqlite` by default; set `JSQLITE_DIR=/path/to/jsqlite` if -it lives elsewhere. +Three repos are populated under `vendor/` at build time (the directory is +git-ignored — these are **not** submodules). Every checkout is fetched by exact +commit and verified by Git tree ID from `vendor-lock.env`. | Dependency | Source | |---|---| -| `jsqlite` | `~/mine/jsqlite` — pure Scheme SQLite-compatible session storage | +| `jsqlite` | `git.sr.ht/~lisp/jsqlite` — pure Scheme SQLite-compatible session storage | | `termbox2` | `github.com/termbox/termbox2` — the TUI's terminal backend | -| `jerboa-websearch` | `git.sr.ht/~lisp/jerboa-websearch` — in-process web search | +| `jerboa-websearch` | `git.sr.ht/~lisp/jerboa-websearch` — Websearch libraries, with the bounded worker overlay under `vendor-overrides/` | -`make build` runs `make vendor-deps` for you. The Makefile clones the sr.ht repos -over HTTPS: +`make build` runs `make vendor-deps` for you. To verify already populated +sources without changing them: ```bash -mkdir -p vendor -git clone --depth 1 https://github.com/termbox/termbox2.git vendor/termbox2 -git clone --depth 1 https://git.sr.ht/~lisp/jerboa-websearch vendor/jerboa-websearch +make vendor-provenance-check ``` ## Building --- a/docs/remote.md +++ b/docs/remote.md @@ -6,8 +6,10 @@ Three subcommands cooperate: **`serve`** (the agent), **`relay`** (a rendezvous point), and **`connect`** (a controller). All three share one auth scheme: a 256-bit hex token in `~/.jcode/` (`0600`), -auto-generated on first run and printed to stderr. `--show-token` prints it; -`--rotate-token` regenerates it. +auto-generated on first run. `--show-token` prints it; `--rotate-token` +regenerates it. Token authentication is defense in depth, not transport +security: credential-bearing TCP listeners and dialers are loopback-only and +must travel through an authenticated SSH or mutually authenticated TLS tunnel. ## `jcode serve` — the agent server @@ -17,7 +19,6 @@ Speaks a line-delimited JSON (JSONL) protocol. By default it runs over **stdio** ```bash jcode serve # stdio jcode serve --port 8321 # TCP on 127.0.0.1:8321 -jcode serve --port 8321 --bind 0.0.0.0 # reachable off-host (use a token!) jcode serve --show-token | --rotate-token ``` @@ -53,40 +54,55 @@ Token: `~/.jcode/server-token`. ## `jcode relay` — rendezvous (VPS side) -When the agent host is behind NAT, run a `relay` on a public VPS. It pairs an +When the agent host is behind NAT, run a loopback relay on a VPS. It pairs an outbound host with an inbound controller and then pumps bytes between them. +The relay never binds a public interface itself. ```bash -jcode relay --port 9001 # binds 0.0.0.0:9001 by default +jcode relay --port 9001 # binds 127.0.0.1:9001 ``` Handshake (newline-delimited JSON, then raw bytes): ``` -host → relay: {"role":"host","name":"NAME","token":"T"} → {"ok":true} -controller → relay: {"role":"controller","host":"NAME","token":"T"} → {"ok":true} +relay → peer : {"challenge":"fresh-256-bit-nonce"} +host → relay: {"role":"host","name":"NAME","proof":"HMAC"} → {"ok":true} +controller → relay: {"role":"controller","host":"NAME","proof":"HMAC"} → {"ok":true} relay → host : {"controller_connected":true} … transparent byte pump in both directions … ``` -One host per name, one controller per host; both must present the matching -token. Token: `~/.jcode/relay-token`. +One host per name, one controller per host. Each peer proves possession of the +matching token using HMAC-SHA-256 over the fresh challenge, role, and host name; +the reusable token is never transmitted and captured proofs cannot be replayed. +Token: `~/.jcode/relay-token`. ## `jcode connect` — controller side -Dials a relay and becomes a transparent stdin/stdout proxy to the remote -`jcode serve`. Pair it with the host registering via `jcode serve --connect`: +Dials a loopback tunnel endpoint and becomes a transparent stdin/stdout proxy +to the remote `jcode serve`. Both the host and controller first authenticate +the VPS with SSH, then point jcode at the resulting local port: ```bash -# On the workstation (behind NAT): -jcode serve --connect vps.example.com:9001 --register myhost +# On the VPS: +jcode relay --port 9001 -# On the laptop: -jcode connect vps.example.com:9001 --host myhost --token a1b2c3… +# On the workstation (behind NAT), keep this tunnel running: +ssh -N -L 19001:127.0.0.1:9001 vps.example.com +jcode serve --connect 127.0.0.1:19001 --register myhost + +# On the controller laptop, use a separate local forward: +ssh -N -L 19001:127.0.0.1:9001 vps.example.com +jcode connect 127.0.0.1:19001 --host myhost --token a1b2c3… ``` `--token` defaults to `~/.jcode/relay-token` if omitted. +Non-loopback values for relay `--bind`, serve `--bind`, serve `--connect`, or +the `connect` endpoint fail closed before any token is transmitted. An mTLS +tunnel may be used instead of SSH when it authenticates the server before +opening the local forwarding socket. + ## Android The Android app is a **thin GUI client**, not an embedded agent. It connects to --- a/docs/threat-model.md +++ b/docs/threat-model.md @@ -9,7 +9,7 @@ production use. provider CLIs, and project configuration. - Project source, git history, private notes, terminal scrollback, and generated traces. -- Remote-control tokens under `~/.jcode/` and the compile-time debug REPL token. +- Remote-control tokens under `~/.jcode/` and the per-start debug control token. - MCP server configuration, tool arguments, and tool results. - Native libraries, C shims, generated build artifacts, and cross-build outputs. @@ -22,10 +22,15 @@ production use. account privileges. - Provider APIs receive prompts and selected repository content. Local models avoid cloud transit but still process sensitive content in local runtimes. -- `serve`, `relay`, `connect`, and the debug REPL cross a network boundary. - Non-loopback binds require explicit token and transport review. +- `serve`, `relay`, `connect`, and the debug control socket cross a network + boundary only through an authenticated tunnel; their own credential-bearing + listeners and dialers fail closed outside loopback. - FFI calls cross from Scheme into Rust and C. Pointers, allocated memory, file descriptors, and socket lifetimes must be explicit and reviewed. +- Search queries cross an exec boundary into a fresh worker. The source layout + pins an absolute reviewed launcher; a packaged `jcode` resolves and re-execs + its own image with a fixed worker flag. Query text is sent only through the + bounded JSON stdin protocol, never argv or Scheme evaluation. ## Controls @@ -36,12 +41,18 @@ production use. - The key store encrypts stored provider keys and writes with owner-only file permissions. - Remote server and relay tokens are generated locally, stored outside the - repository, and written with owner-only permissions. + repository, and written with owner-only permissions. The debug token rotates + on every explicit start and is never embedded in an artifact. - External CLI providers run with a reduced environment and provider-specific allow lists so one provider cannot read another provider's auth state through the wrapper. - Build, test, audit, baseline secret scan, and Jerboa high-severity scan are release gates. +- The search parent caps request/stdout/stderr bytes, owns the absolute + deadline, terminates the worker process group, and reaps it. Worker HTML is + parsed with the compile-time-embedded wasmi payload; unavailable worker, + network policy, reviewed numeric-address TLS API, or parser sandbox states + fail closed. ## Residual Risks @@ -53,9 +64,11 @@ production use. terms and configuration. - The local OS sandbox and user permissions remain part of the security model; `jcode` does not contain untrusted code execution by itself. -- Remote endpoints are inappropriate for public internet exposure without a - deployment-specific network policy, TLS/token review, rate limiting, and log