Add comprehensive documentation for all 13 implementation phases
ober
bd8b2aee556f92f1f5dac8b409ea1050a815cf8b
new file mode 100644 --- /dev/null +++ b/docs/async.md @@ -0,0 +1,446 @@ +# `(std async)` — Async I/O Runtime + +## Overview + +`(std async)` provides a structured-concurrency runtime for Jerboa (a Chez Scheme dialect) built on top of the algebraic effects system in `(std effect)`. It gives you cooperative, task-based async programming without abandoning native threads — instead it uses threads as the underlying suspension mechanism while exposing a clean effect-based API. + +### Thread-per-task model + +Each async task runs in its own OS thread. Suspension is real thread blocking: + +- `(Async await promise)` — blocks the current thread via a mutex/condition variable until the promise resolves +- `(Async spawn thunk)` — forks a new OS thread with Async handlers pre-installed, then returns immediately to the caller +- `(Async sleep ms)` — calls Chez Scheme's `sleep` on the current thread + +This is distinct from a cooperative event loop (like Node.js or Tokio): there is no central scheduler and no `poll`-based reactor. Concurrency comes from OS-level thread scheduling. The benefit is that truly blocking C calls (filesystem, sockets via FFI) work without any special adaptation. + +### Built on algebraic effects + +The entire suspension protocol is expressed as the `Async` algebraic effect (defined with `defeffect` from `(std effect)`). Effect handlers installed by `run-async` intercept each `Async` operation and implement it with mutex/condition-variable primitives from Chez Scheme's built-in thread library. You can write your own handlers on top of `Async::descriptor` to intercept, log, or mock async operations in tests. + +### Dependencies + +``` +(std async) imports: + (chezscheme) — fork-thread, mutex, condition, sleep + (std effect) — defeffect, with-handler, resume + (std misc channel) — channel-get, channel-put, channel-try-get +``` + +--- + +## Core API + +### Effect descriptor + +```scheme +Async::descriptor +``` + +The unique `effect-descriptor` record identifying the `Async` effect. You rarely reference this directly, but it is exported so that external code can install custom handlers with `with-handler` using the `Async` name, or introspect the handler stack via `*effect-handlers*`. + +--- + +### `Async` — effect operations + +`Async` is a macro generated by `defeffect`. It has three operations: + +#### `(Async await promise)` + +Suspend the current task until `promise` is resolved, then return its value. Internally calls `promise-wait!` which performs `condition-wait` in a loop — a true blocking wait on the OS thread. + +Returns the value passed to `async-promise-resolve!`. + +```scheme +(run-async + (lambda () + (let ([p (make-async-promise)]) + ;; Resolve the promise from a spawned task + (Async spawn (lambda () (async-promise-resolve! p 42))) + ;; Suspend until resolved + (Async await p))) ;=> 42 +``` + +#### `(Async spawn thunk)` + +Launch `thunk` as a concurrent task in a new OS thread. Returns `(void)` immediately; the caller is not blocked. The new thread runs with the same Async handler infrastructure installed. + +```scheme +(Async spawn (lambda () + (display "running concurrently\n"))) +``` + +#### `(Async sleep ms)` + +Sleep the current task thread for `ms` milliseconds. Uses Chez Scheme's `sleep` with a `time-duration` record constructed from the millisecond count. + +```scheme +(Async sleep 500) ; sleep 500 ms +``` + +--- + +### `run-async` + +```scheme +(run-async thunk) => value +``` + +Entry point for async code. Runs `thunk` in a new thread with Async effect handlers installed, blocks the calling thread until the thunk completes, and returns the thunk's result. + +- Installs the three Async handlers (`await`, `spawn`, `sleep`) via `install-async-handlers!`. +- Wraps the thunk in a `guard` that catches any exception, prints it to `current-error-port`, and re-raises it so the main thread sees the failure (the promise is resolved with `raise-continuable`). +- The main thread blocks on `promise-wait!` — a `condition-wait` loop — until the root task finishes. + +```scheme +(import (chezscheme) (std effect) (std async)) + +(run-async + (lambda () + (display "hello from async\n") + 42)) +;=> 42 +``` + +--- + +### `run-async/workers` + +```scheme +(run-async/workers thunk n-workers) => value +``` + +Identical to `run-async` in behavior. The `n-workers` argument is accepted for API compatibility but has no effect: since each `(Async spawn ...)` call creates a dedicated OS thread, there is no fixed worker pool to size. Future versions may use `n-workers` to cap parallelism via a semaphore. + +```scheme +(run-async/workers my-thunk 4) ; n-workers is currently ignored +``` + +--- + +### Promises + +Promises are the fundamental coordination primitive. A promise is a write-once, broadcast-notify cell backed by a mutex and condition variable. + +#### `(make-async-promise)` => promise + +Create a new, unresolved promise. + +```scheme +(define p (make-async-promise)) +``` + +#### `(async-promise? x)` => boolean + +Return `#t` if `x` is an async promise. + +#### `(async-promise-resolve! p val)` + +Resolve promise `p` with value `val`. Thread-safe. If `p` is already resolved, the call is a no-op (the first write wins). All threads waiting on `p` via `(Async await p)` are unblocked via `condition-broadcast`. + +```scheme +(async-promise-resolve! p 'done) +``` + +#### `(async-promise-resolved? p)` => boolean + +Return `#t` if the promise has been resolved. This is a non-blocking poll; it does not synchronize with the condition variable beyond the mutex acquisition implied by the record field read. + +#### `(async-promise-value p)` => value + +Return the resolved value. Behavior is unspecified if called before the promise is resolved; use `(async-promise-resolved? p)` or `(Async await p)` to ensure resolution first. + +--- + +### Task management + +#### `(async-task thunk)` => promise + +Spawn a concurrent task that runs `thunk` and captures its return value in a new promise. Returns the promise immediately. The task is launched via `(Async spawn ...)`, so this must be called inside an `run-async` context. + +```scheme +(run-async + (lambda () + (let ([p (async-task (lambda () (expensive-computation)))]) + ;; ... do other work ... + (Async await p)))) ; block until computation finishes +``` + +#### `(async-task? x)` => boolean + +Alias for `async-promise?`. An `async-task` handle is just an `async-promise`. + +--- + +### Async channels + +Async channels bridge `(std misc channel)` (which uses OS-level blocking internally) with the `Async` effect so that waiting for channel data suspends via `(Async await ...)` rather than blocking the thread with a raw `channel-get`. + +#### `(async-channel-get ch)` => value + +Get the next value from channel `ch`. If the channel already has data, it is returned immediately via `channel-try-get`. If the channel is empty, a helper thread is spawned that blocks on `channel-get`, and the current task suspends via `(Async await promise)` until the helper thread delivers a value. + +#### `(async-channel-put ch val)` + +Put `val` into channel `ch`. Always asynchronous: a helper thread is forked to call `channel-put` (which may block if `ch` is bounded and full), and the current task suspends via `(Async await promise)` until the put completes. + +> **Note:** Both channel operations spawn a helper OS thread per call. For high-throughput channels, prefer using `channel-get`/`channel-put` directly from dedicated producer/consumer threads rather than through the Async channel wrappers. + +--- + +### `async-sleep` + +```scheme +(async-sleep ms) +``` + +Sleep the current async task for `ms` milliseconds. Thin wrapper around `(Async sleep ms)`. Must be called inside a `run-async` context. + +```scheme +(run-async + (lambda () + (display "before\n") + (async-sleep 1000) + (display "after 1 second\n"))) +``` + +--- + +## Working examples + +### 1. Basic async task with promise + +Spawn a computation, await its result. + +```scheme +(import (chezscheme) (std effect) (std async)) + +(run-async + (lambda () + (let ([p (async-task (lambda () + ;; Simulate work + (async-sleep 100) + (* 6 7)))]) + (let ([result (Async await p)]) + (printf "Result: ~a~%" result))))) +; prints: Result: 42 +``` + +### 2. Parallel tasks with join + +Run two independent computations in parallel and join their results. + +```scheme +(import (chezscheme) (std effect) (std async)) + +(define (parallel-add a b) + (run-async + (lambda () + (let ([pa (async-task (lambda () (begin (async-sleep 50) a)))) + [pb (async-task (lambda () (begin (async-sleep 80) b)))]) + ;; Both tasks run concurrently; total wall time ~80 ms, not 130 ms + (+ (Async await pa) (Async await pb)))))) + +(printf "~a~%" (parallel-add 10 32)) ; => 42 +``` + +### 3. Producer/consumer with async channels + +```scheme +(import (chezscheme) (std effect) (std async) (std misc channel)) + +(run-async + (lambda () + (let ([ch (make-channel 4)]) ; bounded channel, capacity 4 + + ;; Producer: send 5 items then a sentinel + (Async spawn + (lambda () + (let loop ([i 0]) + (when (< i 5) + (async-channel-put ch i) + (async-sleep 10) + (loop (+ i 1)))) + (async-channel-put ch 'done))) + + ;; Consumer: drain until sentinel + (let consume () + (let ([msg (async-channel-get ch)]) + (unless (eq? msg 'done) + (printf "got: ~a~%" msg) + (consume)))) + + (display "consumer finished\n")))) +``` + +### 4. HTTP-like request with timeout + +Pattern for racing an async operation against a deadline. + +```scheme +(import (chezscheme) (std effect) (std async)) + +(define (with-timeout ms thunk) + ;; Returns (values result #t) on success or (values #f #f) on timeout + (run-async + (lambda () + (let ([result-p (async-task thunk)] + [timeout-p (make-async-promise)]) + ;; Timeout task + (Async spawn + (lambda () + (async-sleep ms) + (async-promise-resolve! timeout-p 'timeout))) + ;; Race: whichever promise resolves first wins + ;; We poll both — a proper select would use channel-select + (let poll () + (cond + [(async-promise-resolved? result-p) + (values (async-promise-value result-p) #t)] + [(async-promise-resolved? timeout-p) + (values #f #f)] + [else + (async-sleep 5) + (poll)])))))) + +(define-values (val ok) + (with-timeout 200 + (lambda () + (async-sleep 100) ; fast enough + "response body"))) + +(if ok + (printf "Got: ~a~%" val) + (printf "Timed out~%")) +``` + +### 5. Fan-out / gather pattern + +Dispatch N tasks in parallel and collect all results. + +```scheme +(import (chezscheme) (std effect) (std async)) + +(define (gather thunks) + ;; Returns a list of results in the same order as thunks + (run-async + (lambda () + (let ([promises (map async-task thunks)]) + (map (lambda (p) (Async await p)) promises))))) + +(define results + (gather + (list + (lambda () (begin (async-sleep 30) 'a)) + (lambda () (begin (async-sleep 10) 'b)) + (lambda () (begin (async-sleep 20) 'c))))) + +(display results) ; => (a b c) +; Total wall time ~30 ms, not 60 ms +``` + +--- + +## Error handling + +When a task spawned by `async-task` or `(Async spawn ...)` throws an unhandled exception: + +- **`async-task`**: the exception propagates inside the spawned thread. Because `async-task` does not wrap the thunk in a `guard`, the exception will be printed by Chez Scheme's default thread-error handler and the task's promise will **never resolve**. Any `(Async await p)` on that promise will block forever. + + Defensive pattern — wrap the thunk yourself: + + ```scheme + (define (safe-task thunk) + (let ([p (make-async-promise)]) + (Async spawn + (lambda () + (guard (exn [#t (async-promise-resolve! p (cons 'error exn))]) + (async-promise-resolve! p (cons 'ok (thunk)))))) + p)) + ``` + +- **`run-async` root task**: the `guard` in `run-async` catches any exception thrown by the root thunk, prints it to `current-error-port`, and resolves the result promise with `raise-continuable`. This means the exception is re-raised in the calling thread (the thread that called `run-async`), so the caller sees a normal Chez Scheme condition. + + ```scheme + (guard (exn [#t (printf "caught: ~a~%" (condition-message exn))]) + (run-async (lambda () (error "boom" "test")))) + ; stderr: run-async error: boom + ; caught: boom + ``` + +--- + +## Performance + +| Scenario | Recommendation | +|---|---| +| CPU-bound parallel work | Use `async-task` / `gather` — each gets a real OS thread | +| Many short-lived tasks | Overhead per task is `fork-thread` cost; avoid spawning thousands | +| Blocking FFI/IO | Works naturally — threads block independently | +| High-throughput channels | Use `channel-get`/`channel-put` directly from dedicated threads; avoid `async-channel-get`/`async-channel-put` (each call spawns a helper thread) | +| Sequential async code | `run-async` with `async-sleep` / `Async await` is fine; no event-loop overhead | + +`run-async/workers n` currently does not limit parallelism. If you need to cap OS thread count (e.g., for a bounded thread pool), implement a semaphore-based wrapper around `async-task`. + +--- + +## Integration with `(std effect)` + +`Async` is defined with `defeffect`, which generates: + +1. `Async::descriptor` — a unique `effect-descriptor` record used as the key in the handler hashtable. +2. A macro `Async` that expands `(Async op arg ...)` into `(effect-perform Async::descriptor 'op (list arg ...))`. + +`effect-perform` captures the current one-shot continuation (`call/1cc`) and calls the matching handler procedure with `(k . args)`. The handler can then: + +- Call `(resume k val)` to continue the task with `val` as the result of the `Async` expression. +- Block the current thread before resuming (as the `await` and `sleep` handlers do). +- Fork a new thread and resume immediately (as the `spawn` handler does). + +The handler stack is stored in a thread-local parameter `*effect-handlers*` (a list of `eq-hashtable` frames). Each `with-handler` call pushes a new frame; `run-with-handler` installs it via `parameterize`. Because each spawned task runs in its own thread, `fork-thread` inherits the parent's thread-parameter values, so child tasks start with the same handler stack depth — but `install-async-handlers!` then installs a fresh set of Async handlers on top for the new thread. + +### Replacing handlers for testing + +You can intercept `Async` operations in unit tests by wrapping code in `with-handler` before it reaches the default handlers: + +```scheme +(import (chezscheme) (std effect) (std async)) + +;; Mock: record all sleep durations, never actually sleep +(define recorded-sleeps '()) + +(with-handler + ([Async + (sleep (k ms) + (set! recorded-sleeps (cons ms recorded-sleeps)) + (resume k (void)))]) ; override: skip the real sleep + (run-async + (lambda () + (async-sleep 100) + (async-sleep 200)))) + +(display recorded-sleeps) ; => (200 100) +``` + +Note that `with-handler` pushes a frame on top of the stack, so the inner `with-handler` in the example above takes precedence over the Async handlers installed by `run-async`, because `effect-perform` searches from the innermost frame outward. + +--- + +## Summary of exported symbols + +| Symbol | Kind | Purpose | +|---|---|---| +| `Async` | macro | Perform `await`, `spawn`, or `sleep` operations | +| `Async::descriptor` | value | Effect descriptor for the `Async` effect | +| `run-async` | procedure | Run a thunk in async context, block until done | +| `run-async/workers` | procedure | Like `run-async`; `n-workers` arg is a no-op | +| `make-async-promise` | procedure | Create an unresolved promise | +| `async-promise?` | procedure | Type predicate | +| `async-promise-resolve!` | procedure | Resolve a promise (write-once, thread-safe) | +| `async-promise-resolved?` | procedure | Non-blocking poll for resolution | +| `async-promise-value` | procedure | Read the resolved value | +| `async-task` | procedure | Spawn a task, return its result promise | +| `async-task?` | procedure | Alias for `async-promise?` | +| `async-channel-get` | procedure | Get from channel, suspending via `Async await` | +| `async-channel-put` | procedure | Put to channel, suspending via `Async await` | +| `async-sleep` | procedure | Sleep current task for N milliseconds | new file mode 100644 --- /dev/null +++ b/docs/build.md @@ -0,0 +1,715 @@ +# `(jerboa build)` and `(jerboa cache)` — Native Binary Toolchain + +Jerboa's build system compiles Chez Scheme source files into standalone native +binaries. The pipeline embeds Chez boot files and compiled code into a C +translation unit, links it against the Chez runtime, and optionally strips all +dynamic-library dependencies with musl libc. + +Two libraries cooperate: + +| Library | Role | +|---------|------| +| `(jerboa build)` | Compilation pipeline, cross-compilation, static linking | +| `(jerboa cache)` | Content-addressed `.so` cache keyed on source + deps + Chez version | + +--- + +## Part 1: `(jerboa build)` + +### Imports + +```scheme +(import (jerboa build)) +``` + +--- + +### Step 41 — Incremental Parallel Build Pipeline + +#### `trace-imports` + +```scheme +(trace-imports source-path) → list-of-import-specs +``` + +Reads `source-path` and returns every `(import ...)` spec found in the file as +a list of S-expressions. The file is read form-by-form; any read error causes +an early return of the forms collected so far. This is used to determine which +modules a source file depends on before compilation begins. + +```scheme +(trace-imports "myapp/main.sls") +;; => ((chezscheme) (myapp util) (myapp model)) +``` + +--- + +#### `compute-file-hash` + +```scheme +(compute-file-hash path) → string or #f +``` + +Computes a 64-bit FNV-1a hash of the entire file contents and returns it as a +lowercase hex string. Returns `"empty"` for zero-length files and `#f` if the +file cannot be read. The hash is used as a cheap change-detection signal — it +is **not** a cryptographic hash. + +```scheme +(compute-file-hash "lib/myapp/util.sls") +;; => "3b0c2d4f9a1e6c8d" +``` + +--- + +#### `module-changed?` + +```scheme +(module-changed? path hash-table) → boolean +``` + +Returns `#t` if the current hash of `path` differs from the value stored in +`hash-table` under the same key, or if no entry exists yet. Used by +`build-project` to skip recompiling files whose content has not changed. + +```scheme +(define ht (make-hashtable equal-hash equal?)) +(module-changed? "lib/myapp/util.sls" ht) ;; => #t (not seen before) +;; ... after recording the hash ... +(module-changed? "lib/myapp/util.sls" ht) ;; => #f (unchanged) +``` + +--- + +#### `compile-modules-parallel` + +```scheme +(compile-modules-parallel paths compile-fn) → alist of (path . result) +``` + +Spawns one Chez thread per path using `fork-thread`. Each thread calls +`(compile-fn path)` and stores its result (or exception) in a shared vector. +After all threads finish (signalled via a mutex-guarded condition variable), any +captured error is re-raised in the calling thread. Returns an alist mapping each +path to its compile result. + +```scheme +(compile-modules-parallel + '("lib/a.sls" "lib/b.sls" "lib/c.sls") + (lambda (path) + (compile-library path))) +;; => (("lib/a.sls" . #t) ("lib/b.sls" . #t) ("lib/c.sls" . #t)) +``` + +--- + +#### `build-project` + +```scheme +(build-project source-paths output-path [parallel: boolean]) → output-path +``` + +Incremental build for a list of source files. Checks each file against the +module-level content hash table. Only files that have changed since last build +are recompiled. When `parallel:` is `#t` (the default), changed files are +compiled concurrently via `compile-modules-parallel`. + +```scheme +(build-project + '("lib/myapp/util.sls" + "lib/myapp/model.sls" + "lib/myapp/main.sls") + "build/myapp" + 'parallel: #t) +;; Prints: +;; Recompiling 2 module(s)... +;; [compile] lib/myapp/model.sls +;; [compile] lib/myapp/main.sls +;; => "build/myapp" +``` + +If all files are up to date, prints `[up to date] build/myapp` and returns +immediately. + +--- + +#### `file->c-array` + +```scheme +(file->c-array file-path var-name) → string +``` + +Reads a binary file and returns a C source string containing a +`static const unsigned char var-name[]` array initialiser, with the bytes laid +out 16 per line in `0xNN` hex notation, followed by a +`static const unsigned int var-name_len` constant. Used internally by +`build-binary` to embed boot files. + +```scheme +(display (file->c-array "/usr/lib/csv10.0.0/ta6le/petite.boot" "petite_boot")) +;; static const unsigned char petite_boot[] = { +;; 0x7f,0x45,0x4c,0x46,0x02,0x01,0x01,0x00,... +;; }; +;; static const unsigned int petite_boot_len = 1234567; +``` + +--- + +#### `generate-main-c` + +```scheme +(generate-main-c boot-arrays program-array link-libs) → string +``` + +Generates a complete C `main()` that initialises Chez Scheme, registers each +boot file from its embedded byte array via `Sregister_boot_file_bytes`, builds +the heap with `Sbuild_heap`, and tears down cleanly. On Linux it includes the +`memfd_create` helper for in-memory boot loading. + +Parameters: +- `boot-arrays` — list of C array strings (output of `file->c-array`) for + `petite.boot`, `scheme.boot`, and `app.boot` in that order. +- `program-array` — optional C array string for a compiled `.so` embedded as + data, or `#f`. +- `link-libs` — reserved list (currently unused in output). + +--- + +#### `build-boot-file` + +```scheme +(build-boot-file output-path deps so-path) → void +``` + +Thin wrapper around Chez's `make-boot-file`. Creates a boot file at +`output-path` that chains the listed `deps` (e.g. `'("petite" "scheme")`) and +embeds the compiled `.so` at `so-path`. + +```scheme +(build-boot-file "/tmp/myapp.boot" '("petite" "scheme") "/tmp/myapp.so") +``` + +--- + +#### `build-binary` + +```scheme +(build-binary source-path output-path + [optimize-level: integer] + [release: boolean] + [static: boolean] + [target: cross-target-or-#f]) +→ output-path +``` + +The main entry point for producing a single native binary. Executes the full +five-stage pipeline: + +1. **Compile** — calls `compile-program` on `source-path`, producing a `.so` + in a temporary build directory. Uses `optimize-level:` (default `2`); in + release mode, `optimize-level` is forced to `3` and inspector information is + suppressed. +2. **Boot** — locates `petite.boot` and `scheme.boot` under standard Chez + install paths (respects `$SCHEMEHEAPDIRS`) and creates `app.boot` from the + compiled `.so`. +3. **C generation** — calls `generate-main-c` with all three boot files + embedded as C arrays. +4. **Link** — compiles the generated `main.c` with GCC (or the + `cross-target-cc` when `target:` is given). Uses `musl-link-flags` when + `static:` is `#t`, otherwise links with `-lm -ldl -lpthread`. +5. **Report** — prints `Built: <output-path>` on success. + +```scheme +;; Simple debug build +(build-binary "src/hello.sls" "bin/hello") + +;; Optimised release build +(build-binary "src/hello.sls" "bin/hello" + 'release: #t) + +;; Static zero-dependency binary +(build-binary "src/hello.sls" "bin/hello" + 'static: #t) + +;; Cross-compile for aarch64 +(build-binary "src/hello.sls" "bin/hello-arm" + 'target: target-linux-aarch64) +``` + +--- + +### Step 42 — Release Builds with Tree Shaking (WPO) + +Chez Scheme's whole-program optimiser (`compile-whole-program`) performs +inter-procedural analysis and dead-code elimination. `(jerboa build)` exposes +this as the "release" build mode. + +#### `wpo-compile` + +```scheme +(wpo-compile source-path output-path) → void +``` + +Direct interface to `compile-whole-program` with maximum settings: +`optimize-level` 3, `cp0-effort-limit` 1000, inspector information disabled. +Raises on error. + +```scheme +(wpo-compile "src/myapp.sls" "build/myapp.wpo") +``` + +--- + +#### `build-release` + +```scheme +(build-release source-paths output-path + [optimize-level: integer] + [wpo-output: string]) +→ wpo-output-path or #f +``` + +Higher-level release build over multiple source files. Uses `compile-whole-program` +on the **first** element of `source-paths` (the program entry point) with: +- `optimize-level` defaulting to `3` +- `cp0-effort-limit` 100 +- `generate-inspector-information` disabled +- `compile-imported-libraries` enabled + +The WPO output file defaults to `output-path` with a `.wpo` suffix appended. +Returns the WPO output path on success, `#f` if an error occurs (the error +message is printed but not re-raised). + +```scheme +(build-release + '("src/main.sls" "src/util.sls" "src/model.sls") + "bin/myapp") +;; Prints: +;; [release] WPO compile (3 sources) +;; [release] WPO: bin/myapp.wpo +;; => "bin/myapp.wpo" +``` + +--- + +#### `tree-shake-imports` + +```scheme +(tree-shake-imports source-path) → list-of-import-specs +``` + +Reads `source-path` and returns all `import` specs while simultaneously walking +every form to collect used symbols into an `eq?` hashtable. The symbol table is +built but not yet used to filter imports — the function returns the import list +as-is. Intended as a static analysis aid; the collected usage information is +available for downstream filtering in a future pass. + +```scheme +(tree-shake-imports "src/myapp.sls") +;; => ((chezscheme) (myapp util) (myapp model)) +``` + +--- + +### Step 43 — Cross-Compilation + +Cross-compilation in `(jerboa build)` is done by substituting the host C +compiler with a cross-toolchain wrapper specified in a `cross-target` record. + +#### `make-cross-target` + +```scheme +(make-cross-target os arch cc ar) → cross-target +``` + +Creates a cross-target descriptor. `os` and `arch` are symbols; `cc` and `ar` +are strings naming the cross-compiler and archiver executables. + +```scheme +(make-cross-target 'linux 'aarch64 "aarch64-linux-gnu-gcc" "aarch64-linux-gnu-ar") +``` + +#### `cross-target?` + +```scheme +(cross-target? x) → boolean +``` + +Returns `#t` if `x` is a cross-target record. + +#### Accessors + +| Procedure | Returns | +|-----------|---------| +| `(cross-target-os t)` | OS symbol (e.g. `'linux`, `'macos`) | +| `(cross-target-arch t)` | Architecture symbol (e.g. `'x86-64`, `'aarch64`) | +| `(cross-target-cc t)` | C compiler executable string | +| `(cross-target-ar t)` | Archiver executable string | + +#### Predefined targets + +| Binding | OS | Arch | CC | AR | +|---------|----|------|----|----| +| `target-linux-x64` | `linux` | `x86-64` | `x86_64-linux-gnu-gcc` | `x86_64-linux-gnu-ar` | +| `target-linux-aarch64` | `linux` | `aarch64` | `aarch64-linux-gnu-gcc` | `aarch64-linux-gnu-ar` | +| `target-macos-x64` | `macos` | `x86-64` | `o64-clang` | `x86_64-apple-darwin-ar` | +| `target-macos-aarch64` | `macos` | `aarch64` | `oa64-clang` | `arm64-apple-darwin-ar` | + +#### `compile-for-target` + +```scheme +(compile-for-target target c-path output-path [extra-flags]) → (values rc cmd) +``` + +Compiles a C file `c-path` to `output-path` using the target's C compiler. +Adds platform-appropriate flags automatically: +- Linux: `-fPIE -pie` +- macOS: `-mmacosx-version-min=11.0` + +Returns two values: the shell return code and the full command string (useful +for debugging). + +```scheme +(define-values (rc cmd) + (compile-for-target target-linux-aarch64 + "/tmp/myapp/main.c" + "bin/myapp-arm64")) + +(when (not (= rc 0)) + (error 'build "cross-compile failed" cmd)) +``` + +--- + +### Step 44 — Static Linking + +#### `static-link-flags` + +```scheme +(static-link-flags static-libs) → string +``` + +Returns GCC flags for a fully static build using glibc: `-static -static-libgcc` +followed by each archive in `static-libs` (space-separated), then +`-lm -lpthread -ldl`. + +```scheme +(static-link-flags '("libcsv.a" "libm.a")) +;; => "-static -static-libgcc libcsv.a libm.a -lm -lpthread -ldl" +``` + +--- + +#### `musl-link-flags` + +```scheme +(musl-link-flags static-libs) → string +``` + +Returns link flags for a musl libc static build. If `musl-gcc` or +`x86_64-linux-musl-gcc` is found on `$PATH`, returns `-static <archives> -lm +-lpthread`. Otherwise falls back to `static-link-flags`-style glibc static +flags. Musl-linked binaries have no runtime dependency on glibc. + +```scheme +(musl-link-flags '()) +;; => "-static -lm -lpthread" (if musl-gcc is available) +``` + +--- + +#### `build-static-binary` + +```scheme +(build-static-binary source-path output-path [options ...]) → output-path +``` + +Convenience wrapper: calls `build-binary` with `static: #t` prepended to the +option list. All other `build-binary` keyword options are accepted. + +```scheme +(build-static-binary "src/server.sls" "bin/server-static") +``` + +--- + +#### `link-static-archives` + +```scheme +(link-static-archives archives output-ar) → (values rc cmd) +``` + +Combines multiple `.a` archive files into a single fat archive using `ar crs`. +Raises an error if `archives` is empty. Returns the shell return code and the +`ar` command string. + +```scheme +(link-static-archives + '("build/libruntime.a" "build/libscheme.a") + "build/libfull.a") +``` + +--- + +## Part 2: `(jerboa cache)` + +### Imports + +```scheme +(import (jerboa cache)) +``` + +The compilation cache stores compiled `.so` files in a content-addressed +directory keyed by a 128-bit hash of `source-content || dep-hashes || +chez-version || opt-level`. A cache hit copies the stored file directly to the +output path, skipping recompilation entirely. + +--- + +### Cache Layout + +``` +~/.jerboa/cache/ + <hex128>.so ← one file per unique (source, deps, version, opt) tuple +``` + +The directory is created lazily on first `cache-store!`. + +--- + +### `cache-directory` + +```scheme +(cache-directory) → string (current directory) +(cache-directory "/path") → sets the directory for this thread +``` + +A `make-parameter` defaulting to `$HOME/.jerboa/cache`. Override it for +testing or CI environments: + +```scheme +(parameterize ([cache-directory "/tmp/ci-cache"]) + (with-compilation-cache ...)) +``` + +--- + +### `cache-key` + +```scheme +(cache-key source-path dep-hashes opt-level) → string +``` + +Computes a 128-bit hex cache key by hashing: +- The full text content of `source-path` +- Each string in `dep-hashes` concatenated in order (`#f` values become `""`) +- The decimal string of `opt-level` +- The Chez Scheme version string + +Uses a dual-stream FNV-1a variant (two independent hash accumulators) producing +a 32-character hex string. + +```scheme +(cache-key "lib/myapp/util.sls" + '("3b0c2d4f" "9a1e6c8d") + 2) +;; => "a3f7c2910b4e8d1f6a3b9c1e7f4d2a5b" +``` + +--- + +### `cache-lookup` + +```scheme +(cache-lookup key) → path or #f +``` + +Returns the absolute path to the cached `.so` if the key exists in the cache +directory, `#f` otherwise. Does not copy the file. + +--- + +### `cache-store!` + +```scheme +(cache-store! key so-path) → void +``` + +Copies the `.so` at `so-path` into the cache under `key`. A no-op if the entry +already exists (content-addressed storage is idempotent). Creates the cache +directory if it does not exist. + +--- + +### `with-compilation-cache` + +```scheme +(with-compilation-cache source-path output-path dep-hashes opt-level compile-thunk) +→ output-path +```