docs: add Chez-changes.md — plain-English list of fork divergences
ober
a3a41a156509814d8910d11e1bf018db5614eed8
new file mode 100644 --- /dev/null +++ b/docs/Chez-changes.md @@ -0,0 +1,96 @@ +# What we change vs. stock Chez Scheme + +The Chez Scheme bundled at `vendor/ChezScheme/` is **not stock**. It is a soft +fork of `github.com/cisco/ChezScheme` (hosted at `git.sr.ht/~lisp/ChezScheme`), +branched at upstream commit `4bde3b35` — just before `10.5.0-pre-release.1`. + +This file is the plain-English list of everything that differs. For the +maintainer narrative (themes, motivations, provenance) see +[`chez-fork.md`](chez-fork.md); for the exact line-level diffs see the one +`.patch`-per-commit series in [`../vendor/ChezScheme-patches/`](../vendor/ChezScheme-patches/). + +**Scope:** 24 source-affecting fork commits (a further 28 commits only touch the +internal logs below). Net diff vs. upstream: **63 files, +5498 / −243**. + +--- + +## 1. New primitives + +Callable from Scheme, registered in `s/primdata.ss`, and visible to the +optimizer (no C FFI round-trip): + +| Primitive | Notes | +| --- | --- | +| `bytevector-slice`, `bytevector-append` | bytevector building blocks (Phase 65) | +| `base64-encode`, `base64-decode` | RFC 4648 base64 over bytevectors (Phase 66) | +| `sha1-bytevector`, `sha256-bytevector` | one-shot digests over bytevectors (Phase 67) | +| `make-ordered-hashtable` + `ordered-hashtable-*` family | insertion-ordered hashtable: `-ref` `-set` `-delete` `-contains` `-keys` `-values` `-entries` `-cells` `-copy` `-clear` `-update` `-walk` `-size` `-hash-function` `-equivalence-function` (Phase 68) | +| `record-walk` | traverse record fields generically (Phase 72) | + +## 2. Security hardening + +Jerboa ships as a single statically-linked binary that parses untrusted +FASL/data; stock Chez assumes a trusted build env. + +- **FASL/vfasl hardening** — bounds checks in vfasl deserialization, `path_append` + hardening, FFI and build-script hardening against malformed input + (`c/fasl.c`, `c/vfasl.c`, `c/ffi.c`, `c/foreign.c`, `c/main.c`). +- **`--static` / `--foreign-libs`** configure options — hermetic static binaries + (the musl-static pipeline depends on these). +- **`--enable-harden`** configure flag — opt-in hardening switch. +- **Intel CET/IBT** — `ENDBR64` emission at function entries (`s/x86_64.ss`). +- **ARM64 BTI** — branch-target-identification landing pads (`s/arm64.ss`). + +## 3. Optimizer specializations + +Performance only — semantics unchanged, each guarded by a regression mat. + +- **`newhash`** — dispatch generic + bulk hashtable ops via sealed-record + predicates (`s/newhash.ss`). +- **`cptypes`** — specialize `hashtable-cell` / `hashtable-ref-cell`, + `hashtable-clear!` / `hashtable-copy`, and 2-arg eq-hashtable bulk ops + (`s/cptypes.ss`, `s/cptypes-lattice.ss`). +- **`cp0`** — fold runs of adjacent literal `string-append` arguments + (`s/cp0.ss`). +- **Regression mats** — `mats/cptypes.ms`, `mats/hash.ms`, `mats/record.ms`, + `mats/bytevector.ms`. + +## 4. Build & packaging + +- **Android/Termux** build script (`make-android`). +- **`tools/gen-static-ffi.sh`** (new) — generates static FFI tables for + `--static` builds. +- **`install.zuo`** copies instead of hard-linking (`makefiles/install.zuo`). +- **WiX installer** updates for `--static` (`wininstall/*.wxs`). +- **`sync-upstream` / `sync-upstream-status`** make targets — fork-maintenance + helpers (`makefiles/Makefile.in`). + +## 5. Routine fix not yet upstreamed + +- **ppc32 + logtest repairs** (cisco PR #1045) — carried as a fork commit until + it lands upstream (`s/ppc32.ss`). + +## 6. Internal docs riding inside the fork — NOT Chez changes + +These are *jerboa* files that live in the Chez tree for editing convenience. +**Never** include them in a patch sent to cisco/ChezScheme: + +- `plan.md` — running log of Phase N investigations +- `findings.md` — optimizer research notes +- `CLAUDE.md` — Claude instructions for work inside the Chez tree +- `bench/jerboa-bench.ss` — jerboa-specific benchmarks + +--- + +## Re-deriving / keeping current + +``` +cd ~/mine/ChezScheme # the fork working tree (has full git history) +git fetch upstream +git log --no-merges --oneline upstream/main..HEAD # the divergence +git diff --stat upstream/main..HEAD # 63 files, +5498/-243 +``` + +Regenerate the patch series after the fork advances: +`./tools/regen-chez-patches.sh`. Vendored snapshot currently at `a4be658a` +(see [`../vendor/ChezScheme/UPSTREAM.md`](../vendor/ChezScheme/UPSTREAM.md)).