Document Chez disassembly investigation
ober
d83b3e747cd9191220f35d636c37818bbcaed577
new file mode 100644 --- /dev/null +++ b/docs/disassembly.md @@ -0,0 +1,363 @@ +# Disassembly Support + +This note captures the May 2026 investigation into whether Jerboa can expose a +Common Lisp style `disassemble` facility on top of Chez Scheme. + +## Summary + +Jerboa can support this, but there is not a hidden public Chez +`disassemble` binding to expose directly. + +The practical path is: + +1. Add a safe metadata layer first, for example `(procedure-code-info proc)`. +2. Add a debug-only native or Chez-internal bridge that can read exact code + object byte ranges. +3. Feed those bytes to a platform disassembler, or add a real instruction + decoder later. + +The implementation should live outside vendored Chez initially. It should be +treated as a debug facility, not as a stable portable API. + +## Verified Environment + +The checks below were run against Jerboa's bundled Chez: + +```text +.chez/lib/csv10.4.0-pre-release.4/tarm64osx/scheme +``` + +The binary reports: + +```text +10.4.0-pre-release.4 +``` + +The runtime probe reported: + +```text +chez-version="Chez Scheme Version 10.4.0" +machine-type=tarm64osx +disassemble-bound=#f +inspect-bound=#t +foreign-callable-entry-point-bound=#t +object-type=procedure +code-type=code +code-name="p" +code-free-count=0 +code-arity-mask=2 +source-path=("/dev/stdin" 1 127) +entry=<native address> +``` + +The important facts are: + +- `disassemble` is not bound. +- `inspect` and `inspect/object` are available. +- `foreign-callable-entry-point` is available. +- A procedure can be inspected to obtain a code object and useful metadata. +- The code object can provide a native entry address. + +## Chez Source Findings + +Chez's inspector has a TODO for a disassembler: + +```text +vendor/ChezScheme/s/inspect.ss:27 +; ---disassembler +``` + +That is a strong signal that Chez's inspector layer knows about code objects, +but does not currently ship a public disassembler interface. + +Procedure inspection is implemented in `inspect.ss`. A procedure object's +`code` method wraps the internal closure code object: + +```text +vendor/ChezScheme/s/inspect.ss:2149 +vendor/ChezScheme/s/inspect.ss:2153 +vendor/ChezScheme/s/inspect.ss:2162 +``` + +Code object inspection exposes metadata that is useful for a first pass: + +```text +vendor/ChezScheme/s/inspect.ss:2227 +vendor/ChezScheme/s/inspect.ss:2229-2243 +``` + +The available code object methods include: + +- `value` +- `name` +- `info` +- `free-count` +- `arity-mask` +- `source` +- `source-path` +- `source-object` +- `realm` +- `reloc` +- `reloc+offset` +- `size` + +Chez also has private helpers for the data needed by a real disassembler: + +```text +vendor/ChezScheme/s/inspect.ss:2495-2503 +``` + +Those helpers include `$code-reloc-size` and `$code-length`, but they depend on +private runtime accessors like `$object-ref` and constants such as +`code-length-disp`. + +The size accounting code confirms that a code object's real byte length is +computed from `header-size-code` plus `$code-length`: + +```text +vendor/ChezScheme/s/inspect.ss:2699-2705 +vendor/ChezScheme/s/inspect.ss:2881-2884 +``` + +The public entry-point hook is in `prims.ss`: + +```text +vendor/ChezScheme/s/prims.ss:682-693 +``` + +On non-portable-bytecode targets it checks that the value is a code object and +returns: + +```scheme +($object-address x (constant code-data-disp)) +``` + +The reverse mapping is also present: + +```text +vendor/ChezScheme/s/prims.ss:695-707 +``` + +That is useful for validation, but it does not solve byte-length extraction. + +## What Is Public Today + +This works in ordinary Chez Scheme code: + +```scheme +(import (chezscheme)) + +(define p (lambda (x) (+ x 1))) +(define obj (inspect/object p)) +(define code (obj 'code)) + +(code 'name) +(code 'free-count) +(code 'arity-mask) +(call-with-values (lambda () (code 'source-path)) list) +(foreign-callable-entry-point (code 'value)) +``` + +This is enough for a metadata API: + +```scheme +(procedure-code-info proc) +``` + +A useful result shape would include: + +- procedure name +- source path and position, when available +- realm +- arity mask +- free variable count +- relocation table summary +- native entry address, when available +- machine type +- Chez version + +## What Is Not Public Today + +The following are not bound in a normal `(chezscheme)` interaction environment: + +```text +$object-address +$address->object +$object-ref +$closure-code +$code-length +$code-name +$code-info +$code-free-count +$code-arity-mask +$code? +``` + +Constants used by the inspector are also not directly available: + +```text +code-data-disp +code-length-disp +header-size-code +ptr-bytes +architecture +``` + +That means a pure Jerboa implementation can get an entry address, but cannot +safely know how many instruction bytes to read. + +Guessing a byte count is not acceptable. It could read past the code object, +cross into relocation data or unrelated heap data, and produce misleading output +or crash the process. + +## Why `compile-file` Is Not Enough + +Chez `compile-file` writes files with a `.so` suffix, but those files are Chez +compiled/fasl data, not normal Mach-O or ELF shared libraries. + +A test compile produced: + +```text +compiling /tmp/.../t.ss with output to /tmp/.../t.so +/tmp/.../t.so: data +``` + +The header starts with Chez fasl data: + +```text +00 00 00 00 63 68 65 7a ... +``` + +Because of this, platform tools such as `otool`, `objdump`, or +`llvm-objdump` cannot directly disassemble normal Chez `.so` compile output. + +## Implementation Plan + +### Phase 1: Metadata + +Add a module such as: + +```scheme +(std debug disassemble) +``` + +Initial exports: + +```scheme +(procedure-code-info proc) +(write-procedure-code-info proc port) +``` + +This phase should be implemented using only public Chez hooks: + +- `inspect/object` +- code object methods from the inspector +- `foreign-callable-entry-point` +- `machine-type` +- `scheme-version` + +This gives immediate value without depending on private object layout. + +### Phase 2: Byte Extraction + +Add a debug-only bridge that can return the exact native instruction bytevector +for a code object. + +The bridge needs: + +- code object validation +- code entry address +- exact code length +- byte copy from the code data region +- clear failure on portable-bytecode targets or unsupported machine types + +Possible approaches: + +1. Expose a narrow Chez-side wrapper around the same internals used by + `inspect.ss`. +2. Add a small C helper built with access to Chez internal headers/macros. +3. Keep the bridge behind a debug feature flag so release builds do not depend + on private runtime layout. + +The helper should return data, not formatted assembly: + +```scheme +(procedure-code-bytes proc) ; => bytevector plus metadata +``` + +Formatting should stay separate from extraction. + +### Phase 3: Disassembly Formatting + +Once bytes are available, Jerboa can format them in one of three ways: + +1. Shell out to `llvm-objdump` or another platform disassembler after wrapping + bytes in a minimal object container. +2. Use a native disassembly library through FFI. +3. Implement a small decoder for selected architectures later. + +The first version should target the machines Jerboa already builds: + +- macOS arm64: `tarm64osx`, ARM64 assembly +- Linux amd64: `ta6le`, x86-64 assembly +- Linux arm64: `tarm64le`, ARM64 assembly + +Portable bytecode targets need a different story, because they do not expose a +normal native instruction stream. + +## Proposed User API + +The Common Lisp inspired API can be: + +```scheme +(disassemble proc) +(disassemble proc port) +``` + +It should print a human-readable report: + +```text +procedure: f +source: path.ss:line:column +machine: tarm64osx +entry: 0x... +arity-mask: ... +free-count: ... + +<assembly> +``` + +For tooling and tests, expose structured variants: + +```scheme +(procedure-code-info proc) +(procedure-code-bytes proc) +(procedure-disassembly proc) +``` + +The string-returning or data-returning APIs are easier to test than a pure +printer. + +## Risks + +- Chez code object layout is internal and version-specific. +- Native output is architecture-specific. +- Optimizer settings affect what code exists and how recognizable it is. +- WPO, cross compilation, and stripped inspector information may reduce source + metadata. +- Reading raw heap bytes must be exact. Do not implement this by guessing a + fixed byte count from the entry address. +- Relocations and embedded constants should be displayed carefully; raw + instruction disassembly alone may not explain all call targets. + +## Recommendation + +Do not patch vendored Chez first. + +Start with `(std debug disassemble)` as a Jerboa debug module that exposes +metadata. Then add a narrow, well-tested debug bridge for byte extraction. Once +the bridge can return exact bytes and metadata, build the user-facing +`disassemble` printer on top of that. + +This keeps the risky part small and makes the first useful version available +without committing Jerboa to Chez private layout as a public API.