Guard load-shared-object in signal.sls and document static binary gotchas
ober
5ecb3ef0d82ef24a422bd61c06b225aeb7009997
new file mode 100644 --- /dev/null +++ b/docs/static-binary-gotchas.md @@ -0,0 +1,384 @@ +# Static Binary Gotchas — Lessons from jerboa-shell + +This document captures practical, hard-won knowledge from building jerboa-shell as both a glibc dynamic binary and a musl fully-static binary. These are real bugs encountered and fixed, not theoretical concerns. + +**Reference project**: `~/mine/jerboa-shell` + +--- + +## 1. `load-shared-object` Crashes Static Builds + +### The Problem + +Any call to `(load-shared-object ...)` in a statically-linked musl binary raises: + +``` +Exception in load-shared-object: not supported +``` + +musl's static libc does not include `dlopen`. When Chez Scheme is built with `--static CC=musl-gcc`, the `load-shared-object` function raises an exception unconditionally. + +### Why It's Insidious + +The crash happens during **library initialization** — before any user code runs. If any `.so` in the boot file chain calls `load-shared-object` at the top level, the entire binary fails to start. + +### Affected Patterns + +Every module that does `(load-shared-object ...)` at library init time is a landmine for static builds: + +```scheme +;; Pattern 1: dlopen(NULL) to resolve linked symbols +(define _lib (load-shared-object "")) + +;; Pattern 2: Loading libc explicitly for foreign-procedure +(define libc (load-shared-object "libc.so.6")) + +;; Pattern 3: Loading project-specific FFI shared libs +(define _ffi (load-shared-object "./libfoo.so")) +``` + +### The Fix + +Wrap every `load-shared-object` call in `guard`: + +```scheme +;; Safe for both dynamic and static builds +(define _lib + (guard (e [#t #f]) ;; silently skip if dlopen unavailable + (load-shared-object ""))) + +(define libc + (guard (e [#t #f]) + (load-shared-object "libc.so.6"))) +``` + +In static builds, `foreign-procedure` still works for symbols registered via `Sforeign_symbol()` in C — it doesn't need `load-shared-object` for those. + +### Files That Needed This Fix in jerboa-shell + +| File | Call | Why it exists | +|------|------|---------------| +| `src/compat/gambit.sls` | `(load-shared-object "")` | dlopen(NULL) for -rdynamic symbols | +| `src/compat/gambit.sls` | `(load-shared-object "./libjsh-ffi.so")` | FFI shim for interpreted mode | +| `src/jsh/ffi.sls` | `(load-shared-object "")` | Same as gambit.sls | +| `src/jsh/ffi.sls` | `(load-shared-object "./libjsh-ffi.so")` | Same | +| `~/mine/jerboa/lib/std/os/signal.sls` | `(load-shared-object "libc.so.6")` | For `kill()` via foreign-procedure | + +### Rule for Jerboa Library Authors + +**Any module that might end up in a boot file MUST guard its `load-shared-object` calls.** The modules `std/os/signal`, `std/os/temporaries`, and `std/net/grpc` all call `load-shared-object` — if they're in the boot file, they'll crash static builds. + +--- + +## 2. `Sforeign_symbol` — The Complete FFI Registration Table + +### The Problem + +In a static binary (no `dlopen`), Chez Scheme's `foreign-procedure` can only find C functions that were explicitly registered via `Sforeign_symbol()` in C code before the Scheme heap boots. + +If you miss even one symbol, you get: + +``` +Exception in foreign-procedure: no entry for "ffi_file_uid" +``` + +### The Fix + +The musl build script (`build-jsh-musl.ss`) generates a C function `register_ffi_symbols()` that registers every C function used by any Scheme module in the boot file. This must be called **after** `Sbuild_heap()` but before any Scheme code runs that calls `foreign-procedure`. + +### Three Categories of Symbols to Register + +#### Category 1: Project FFI Functions (from ffi-shim.c) + +These are custom C functions written for the project. Grep for them: + +```bash +# Find all ffi_ functions defined in your C shim +grep '^[a-z_].*ffi_' ffi-shim.c | grep -oP 'ffi_\w+' | sort -u +``` + +**Every one must be registered** with both `extern void` declarations and `Sforeign_symbol()` calls. + +#### Category 2: POSIX libc Functions + +Functions called via `foreign-procedure` or `define-foreign` directly by name: + +```bash +# Find all C function names referenced in Scheme code +grep -rh 'define-foreign\b\|foreign-procedure' src/ | \ + grep -oP '"[a-zA-Z_][a-zA-Z0-9_]*"' | sort -u +``` + +For jerboa-shell, this includes: `fork`, `_exit`, `close`, `dup`, `dup2`, `read`, `write`, `lseek`, `access`, `unlink`, `getpid`, `getppid`, `kill`, `sysconf`, `setpgid`, `getpgid`, `tcsetpgrp`, `tcgetpgrp`, `setsid`, `getuid`, `geteuid`, `getegid`, `isatty`, `unsetenv`. + +**Don't forget jerboa library modules in the boot file** — e.g., `std/os/signal.sls` uses `kill`, and `std/os/fdio.sls` uses `read`, `write`, `close`. + +#### Category 3: Variadic/Macro POSIX Functions + +Some POSIX "functions" are actually macros or variadic functions that can't be passed to `Sforeign_symbol()` by address. These need thin C wrappers: + +```c +// Wrappers for variadic/macro POSIX functions +static int wrap_open(const char *path, int flags, int mode) { + return open(path, flags, mode); +} +static int wrap_fcntl(int fd, int cmd, int arg) { + return fcntl(fd, cmd, arg); +} +static int wrap_mkfifo(const char *path, int mode) { + return mkfifo(path, mode); +} +static int wrap_umask(int mask) { + return (int)umask((mode_t)mask); +} + +// Register wrapper under the original name +Sforeign_symbol("open", (void*)wrap_open); +Sforeign_symbol("fcntl", (void*)wrap_fcntl); +``` + +### Maintenance Process + +When adding new `foreign-procedure` or `define-foreign` calls to any module in the boot file: + +1. Add the C function name to the `register_ffi_symbols()` lists in `build-jsh-musl.ss` +2. If it's a variadic/macro function, add a wrapper +3. Rebuild the musl binary and test + +--- + +## 3. `Sregister_boot_file_bytes` — The Right API + +### The Problem + +There are multiple Chez Scheme APIs for registering boot files. Using the wrong one causes subtle failures. + +### API Comparison + +| API | Use Case | Status | +|-----|----------|--------| +| `Sregister_boot_file_bytes(name, data, len)` | Embedded byte arrays in C | **Correct** — simple, works everywhere | +| `Sregister_boot_file_fd(name, fd)` | File descriptor based | **Avoid** — requires memfd, more complex, same result | +| `Sregister_boot_file(path)` | File on disk | Not useful for self-contained binaries | + +### Why `Sregister_boot_file_bytes` Is Better + +The boot data is already in the binary's `.rodata` section as a C byte array. `Sregister_boot_file_bytes` reads directly from that memory — zero copies, zero syscalls, zero failure modes. + +The memfd approach (`Sregister_boot_file_fd`) writes the data to a kernel-backed anonymous file, seeks back to 0, and then Chez reads it back. This is strictly worse: +- Extra syscalls (`memfd_create`, `write`, `lseek`) +- Extra memory (kernel buffer + user buffer) +- Extra failure modes (memfd_create can fail, write can be short) +- More code to maintain + +### Pattern + +```c +// In the C main: +#include "jsh_petite_boot.h" // petite_boot_data[], petite_boot_size +#include "jsh_scheme_boot.h" // scheme_boot_data[], scheme_boot_size +#include "jsh_jsh_boot.h" // jsh_boot_data[], jsh_boot_size + +Sscheme_init(NULL); +Sregister_boot_file_bytes("petite", (void*)petite_boot_data, petite_boot_size); +Sregister_boot_file_bytes("scheme", (void*)scheme_boot_data, scheme_boot_size); +Sregister_boot_file_bytes("jsh", (void*)jsh_boot_data, jsh_boot_size); +Sbuild_heap(NULL, NULL); +``` + +Note: The **program** `.so` still uses memfd (for `Sscheme_script`), because `Sscheme_script` requires a file path. But boot files don't need this — they have a dedicated bytes API. + +--- + +## 4. The Gerbil→Chez Post-Build Patching System + +### The Problem + +jerboa-shell's `.sls` files are auto-generated from Gerbil `.ss` sources by the Gherkin compiler (via `build-jerboa.ss`). The generated code frequently needs fixes that can't be done at the source level because: + +1. The Gerbil compiler drops `(only ...)` imports that look unused (e.g., parameter mutations) +2. The Gerbil compiler transforms `let` to `let*` and reorders definitions +3. Keyword-style constructor calls need positional conversion for Chez +4. R6RS restrictions on mutable exports require `identifier-syntax` rewrites + +### The Patch System + +`build-jerboa.ss` defines `patch-file!` inside a `(let () ...)` block: + +```scheme +(let () + (define (patch-file! path old new) + ;; string-replace on file contents + ...) + + ;; All patches must be INSIDE this let block + (patch-file! "src/jsh/environment.sls" old-string new-string) + (patch-file! "src/jsh/main.sls" old-string new-string) + ...) +``` + +**Critical**: `patch-file!` is local to the `let` block. New patches must be added INSIDE the closing `)`. If you add them after, you get `variable patch-file! is not bound`. + +### Gotcha: Matching Generated Code + +When writing patch strings, you must match the **generated** `.sls` code exactly, not the `.ss` source. Common differences: + +| .ss Source | Generated .sls | +|-----------|---------------| +| `(let ((x ...)))` | `(let* ([x ...]))` | +| `(let* ...)` | `(let* ...)` (preserved) | +| Keywords: `parent: env` | Dropped or rewritten | +| `(only :jsh/sandbox ...)` | May be dropped entirely if "unused" | + +**Always rebuild, then inspect the generated `.sls` to find the exact string to match.** + +### Example: Wiring `*current-jsh-env*` + +The Gerbil compiler drops `(only :jsh/sandbox *current-jsh-env*)` and `(*current-jsh-env* env)` because it looks like a side-effect-only parameter mutation with no visible use. The fix is a post-build patch: + +```scheme +;; Add import +(patch-file! "src/jsh/main.sls" + "(jsh stage))" + "(jsh stage)\n (only (jsh sandbox) *current-jsh-env*))") + +;; Set parameter after env creation +(patch-file! "src/jsh/main.sls" + "(let* ([env (init-shell-env args-hash)])\n (cond" + "(let* ([env (init-shell-env args-hash)])\n (*current-jsh-env* env)\n (cond") +``` + +--- + +## 5. musl vs glibc Build: Two Separate Pipelines + +### Architecture + +The project has two independent build pipelines that share source code but diverge at the C compilation/linking stage: + +| Aspect | glibc (`make jsh`) | musl (`make musl-jsh`) | +|--------|---------------------|------------------------| +| Build script | `build-binary-jsh.ss` | `build-jsh-musl.ss` | +| C main | `jsh-main.c` (separate file) | Generated inline in build script | +| WPO | Yes (`compile-whole-program`) | No (uses `jsh.so` directly) | +| FFI resolution | `load-shared-object` + `-rdynamic` | `Sforeign_symbol()` registration | +| Boot API | `Sregister_boot_file_bytes` | `Sregister_boot_file_bytes` | +| Linker | `gcc -rdynamic` | `musl-gcc -static` | +| Headers | Generated → compiled → deleted | Generated in /tmp → compiled → deleted | +| Output | `./jsh` (dynamic ELF) | `./jsh-musl` (static ELF, ~6.5 MB) | + +### Key Difference: The Generated C Main + +The musl build generates `jsh_main_musl.c` inline (as Scheme string output). It includes: + +1. `Sforeign_symbol` registration for ALL FFI functions +2. C wrappers for variadic/macro POSIX functions (`open`, `fcntl`, `mkfifo`, `umask`) +3. The same memfd + `Sscheme_script` pattern as the glibc build + +Changes to FFI bindings require updating BOTH: +- `jsh-main.c` (if it has FFI declarations — currently it doesn't) +- `build-jsh-musl.ss` (the `register_ffi_symbols` function and extern declarations) + +--- + +## 6. Boot File Module Ordering + +### The Problem + +`make-boot-file` requires modules in strict dependency order. If module A imports module B, B must appear **before** A in the boot file list. + +### The Ordering + +The boot file loads modules in this order: + +1. **Jerboa stdlib** — `jerboa/core`, `jerboa/runtime`, `std/error`, `std/format`, `std/sort`, `std/pregexp`, `std/sugar`, `std/misc/*`, `std/stm`, `std/foreign`, `std/os/*`, `std/transducer`, `std/log`, `std/capability/*` +2. **Gherkin runtime** — `compat/types`, `compat/gambit-compat`, `runtime/*`, `reader/reader`, `compiler/compile`, `boot/gherkin` +3. **Compat layer** — `src/compat/gambit.so` +4. **Application modules** — `ffi`, `pregexp-compat`, `stage`, `static-compat`, `ast`, `registry`, `macros`, `util`, `environment`, `lexer`, `arithmetic`, `glob`, `fuzzy`, `history`, `parser`, `functions`, `signals`, `expander`, `redirect`, `control`, `jobs`, `builtins`, `pipeline`, `executor`, `completion`, `prompt`, `lineedit`, `fzf`, `script`, `startup`, `sandbox`, `main` + +### Adding a New Module + +When adding a new jsh module: + +1. Add it to the compile list in `build-jsh-musl.ss` step 1 (correct tier) +2. Add it to the boot file module list in step 4 (after its dependencies) +3. If it uses `foreign-procedure`, add symbols to the `register_ffi_symbols` list in step 5 +4. Add it to `build-binary-jsh.ss` in the same places (for glibc builds) + +--- + +## 7. WPO (Whole-Program Optimization) Fragility + +### The Problem + +`compile-whole-program` requires `.wpo` files with matching compilation instances for every library. If even one module was compiled in a different Scheme session or with different parameters, WPO fails with: + +``` +Exception in compile-whole-program: "src/jsh/ast.wpo" does not define +expected compilation instance of library (jsh ast) +``` + +### The musl Workaround + +The musl build skips WPO entirely and uses the direct `jsh.so` from `compile-program`: + +```scheme +;; Step 3: Skip WPO for musl builds +(define program-so "jsh.so") +``` + +This is simpler, more reliable, and the binary size difference is negligible (~100 KB). + +### When WPO Works + +WPO works when ALL modules are compiled in the same Scheme session with `compile-imported-libraries` and `generate-wpo-files` enabled. The glibc build (`build-binary-jsh.ss`) does this but can still fail if stale `.wpo` files exist from a previous session. + +**Fix**: `rm -f src/jsh/*.wpo` before a clean build. + +--- + +## 8. Testing Static Binaries + +### Smoke Tests + +```bash +# Basic execution +./jsh-musl -c 'echo hello' + +# Variable assignment +./jsh-musl -c 'x=42; echo $x' + +# Command substitution +./jsh-musl -c 'echo $(echo nested)' + +# Loops +./jsh-musl -c 'for i in 1 2 3; do echo $i; done' + +# Meta-commands (requires *current-jsh-env* wiring) +./jsh-musl -c ',trace echo test' +./jsh-musl -c ',profile echo test' + +# Portability test +cp jsh-musl /tmp/jsh-test && /tmp/jsh-test -c 'echo works from /tmp' +``` + +### Verification + +```bash +# Confirm fully static +file jsh-musl +# Expected: ELF 64-bit LSB executable, x86-64, ... statically linked + +ldd jsh-musl +# Expected: not a dynamic executable +``` + +### Common Failure Modes + +| Error | Cause | Fix | +|-------|-------|-----| +| `load-shared-object: not supported` | Unguarded `load-shared-object` call | Add `guard` wrapper | +| `no entry for "foo"` | Missing `Sforeign_symbol` registration | Add to `register_ffi_symbols` | +| `shell not initialized` | `*current-jsh-env*` not set | Check post-build patches | +| Segfault on startup | Boot file ordering wrong | Check dependency order | +| Binary hangs | Threading bug (program in boot file) | Use `Sscheme_script` for program | --- a/lib/jerboa/build.sls +++ b/lib/jerboa/build.sls @@ -148,7 +148,7 @@ [len (bytevector-length data)]) (call-with-string-output-port (lambda (port) - (format port "static const unsigned char ~a[] = {~%" var-name) + (format port "static unsigned char ~a[] = {~%" var-name) (do ([i 0 (+ i 1)]) ((= i len)) (format port "0x~2,'0x" (bytevector-u8-ref data i)) @@ -157,7 +157,7 @@ (when (= (mod (+ i 1) 16) 0) (newline port))) (format port "~%};~%") - (format port "static const unsigned int ~a_len = ~a;~%" var-name len))))) + (format port "static unsigned int ~a_len = ~a;~%" var-name len))))) ;; ========== Step 41: C Main Template ========== --- a/lib/jerboa/build/musl.sls +++ b/lib/jerboa/build/musl.sls @@ -1,8 +1,22 @@ #!chezscheme ;;; (jerboa build musl) — Static Binary Delivery with musl libc ;;; -;;; Provides musl-specific build functionality for creating fully static -;;; executables with zero runtime dependencies. +;;; Builds fully static executables using: +;;; - musl-gcc for C compilation +;;; - A musl-built Chez Scheme (configured with --static CC=musl-gcc) +;;; - Chez's native static boot embedding (main.o + static_boot_init) +;;; +;;; The musl-built Chez installation provides: +;;; main.o — Chez's own main() with arg parsing, REPL, --script, etc. +;;; libkernel.a — The Chez runtime (compiled with musl) +;;; libz.a — zlib (compiled with musl) +;;; liblz4.a — lz4 (compiled with musl) +;;; scheme.h — C header for Chez API +;;; *.boot — Boot files to embed +;;; +;;; We generate a static_boot.c that provides static_boot_init(), which +;;; main.o calls to register embedded boot files. The application's compiled +;;; .so is bundled into an app boot file via make-boot-file. (library (jerboa build musl) (export @@ -21,6 +35,7 @@ ;; Paths musl-libkernel-path + musl-chez-lib-dir musl-boot-files musl-crt-objects @@ -36,11 +51,13 @@ ;; ========== Configuration ========== - ;; Path to musl-built Chez Scheme installation - ;; Default: /opt/chez-musl (can be overridden) + ;; Path to musl-built Chez Scheme installation. + ;; Probed in order: $JERBOA_MUSL_CHEZ_PREFIX, ~/chez-musl, /opt/chez-musl (define *musl-chez-prefix* (make-parameter (or (getenv "JERBOA_MUSL_CHEZ_PREFIX") + (let ([home-prefix (format "~a/chez-musl" (getenv "HOME"))]) + (if (file-directory? home-prefix) home-prefix #f)) "/opt/chez-musl"))) (define (musl-chez-prefix) (*musl-chez-prefix*)) @@ -62,17 +79,9 @@ (close-port from-stderr) (if (eof-object? line) #f - (let ([trimmed (musl-string-trim-right line)]) + (let ([trimmed (%musl-string-trim-right line)]) (if (string=? trimmed "") #f trimmed))))))) - (define (musl-string-trim-right s) - (let loop ([i (- (string-length s) 1)]) - (if (< i 0) - "" - (if (char-whitespace? (string-ref s i)) - (loop (- i 1)) - (substring s 0 (+ i 1)))))) - (define (musl-gcc-path) "Return path to musl-gcc wrapper, or #f if not found" (or (find-musl-executable "musl-gcc") @@ -84,13 +93,12 @@ (define (musl-sysroot) "Return the musl sysroot directory" - ;; Query musl-gcc for its sysroot (let ([gcc (musl-gcc-path)]) (if gcc (let ([result (with-output-to-string (lambda () (system (format "~a -print-sysroot 2>/dev/null" gcc))))]) - (let ([trimmed (musl-string-trim-right result)]) + (let ([trimmed (%musl-string-trim-right result)]) (if (string=? trimmed "") ;; Fallback: standard musl location "/usr/lib/x86_64-linux-musl" @@ -103,38 +111,83 @@ "Return the Chez Scheme machine type (e.g., ta6le)" (symbol->string (machine-type))) - (define (musl-libkernel-path) - "Return path to musl-built libkernel.a" + (define (musl-chez-lib-dir) + "Find the csv<version>/<machine> directory under the musl Chez prefix. + Scans lib/ for csv* directories since (scheme-version) returns a + human-readable string unsuitable for path construction." (let* ([prefix (musl-chez-prefix)] [machine (chez-machine-type)] - ;; Try common patterns - [paths (list - (format "~a/lib/csv~a/~a/libkernel.a" - prefix (scheme-version) machine) - (format "~a/lib/~a/libkernel.a" prefix machine) - (format "~a/libkernel.a" prefix))]) - (let loop ([ps paths]) - (if (null? ps) - (error 'musl-libkernel-path - "Cannot find musl libkernel.a" - (musl-chez-prefix)) - (if (file-exists? (car ps)) - (car ps) - (loop (cdr ps))))))) + [lib-dir (format "~a/lib" prefix)]) + (if (not (file-directory? lib-dir)) + (error 'musl-chez-lib-dir + "Chez musl lib directory not found" lib-dir) + ;; Scan for csv* directories, pick the newest (last sorted) + (let* ([entries (directory-list lib-dir)] + [csv-dirs (filter + (lambda (e) + (and (> (string-length e) 3) + (string=? (substring e 0 3) "csv"))) + entries)] + ;; Sort so the highest version comes last + [sorted (sort string<? csv-dirs)]) + (if (null? sorted) + (error 'musl-chez-lib-dir + "No csv* directory found in" lib-dir) + ;; Try each csv dir (newest first) for one containing machine/ + (let loop ([dirs (reverse sorted)]) + (if (null? dirs) + (error 'musl-chez-lib-dir + "No csv*/<machine> directory found" + (cons lib-dir machine)) + (let ([candidate (format "~a/~a/~a" lib-dir (car dirs) machine)]) + (if (file-directory? candidate) + candidate + (loop (cdr dirs))))))))))) + + (define (musl-libkernel-path) + "Return path to musl-built libkernel.a" + (let ([path (format "~a/libkernel.a" (musl-chez-lib-dir))]) + (if (file-exists? path) + path + (error 'musl-libkernel-path + "Cannot find musl libkernel.a" path)))) (define (musl-boot-files) "Return list of (name . path) for musl-built boot files" - (let* ([prefix (musl-chez-prefix)] - [machine (chez-machine-type)] - [boot-dir (format "~a/lib/csv~a/~a" - prefix (scheme-version) machine)]) - (if (file-directory? boot-dir) + (let ([dir (musl-chez-lib-dir)]) + (let ([petite (format "~a/petite.boot" dir)] + [scheme (format "~a/scheme.boot" dir)]) + (unless (file-exists? petite) + (error 'musl-boot-files "petite.boot not found" petite)) + (unless (file-exists? scheme) + (error 'musl-boot-files "scheme.boot not found" scheme)) (list - (cons "petite" (format "~a/petite.boot" boot-dir)) - (cons "scheme" (format "~a/scheme.boot" boot-dir))) - (error 'musl-boot-files - "Cannot find musl boot directory" - boot-dir)))) + (cons "petite" petite) + (cons "scheme" scheme))))) + + (define (musl-scheme-h-path) + "Return path to scheme.h from musl Chez installation" + (let ([path (format "~a/scheme.h" (musl-chez-lib-dir))]) + (if (file-exists? path) + path + (error 'musl-scheme-h-path "scheme.h not found" path)))) + + (define (musl-main-o-path) + "Return path to main.o from musl Chez installation" + (let ([path (format "~a/main.o" (musl-chez-lib-dir))]) + (if (file-exists? path) + path + (error 'musl-main-o-path "main.o not found" path)))) + + (define (musl-libz-path) + "Return path to libz.a (zlib) from musl Chez installation, or #f" + (let ([path (format "~a/libz.a" (musl-chez-lib-dir))]) + (if (file-exists? path) path #f))) + + (define (musl-liblz4-path) + "Return path to liblz4.a from musl Chez installation, or #f" + (let ([path (format "~a/liblz4.a" (musl-chez-lib-dir))]) + (if (file-exists? path) path #f))) (define (musl-crt-objects) "Return paths to musl CRT objects needed for static linking" @@ -153,32 +206,49 @@ [(not (musl-available?)) (cons 'error "musl-gcc not found. Install musl-tools package.")] - [(not (file-exists? (musl-chez-prefix))) + [(not (file-directory? (musl-chez-prefix))) (cons 'error (format "musl Chez prefix not found: ~a\n\ - Build Chez with musl or set JERBOA_MUSL_CHEZ_PREFIX" + Build Chez with: ./configure --threads --static CC=musl-gcc\n\ + Then: make install" (musl-chez-prefix)))] - [(guard (e [#t #f]) (musl-libkernel-path) #t) + [(guard (e [#t #f]) (musl-chez-lib-dir) #t) => (lambda (_) - (let ([crt (musl-crt-objects)]) - (let loop ([objs crt]) - (if (null? objs) - (cons 'ok "musl toolchain validated") - (if (file-exists? (car objs)) - (loop (cdr objs)) - (cons 'error - (format "CRT object not found: ~a" - (car objs))))))))] + (cond + [(not (guard (e [#t #f]) (musl-libkernel-path) #t)) + (cons 'error "musl libkernel.a not found")] + [(not (guard (e [#t #f]) (musl-main-o-path) #t)) + (cons 'error "musl main.o not found (Chez not built with --static?)")] + [(not (guard (e [#t #f]) (musl-scheme-h-path) #t)) + (cons 'error "scheme.h not found in musl Chez installation")] + [else + (let ([crt (musl-crt-objects)]) + (let loop ([objs crt]) + (if (null? objs) + (cons 'ok + (format "musl toolchain validated (~a)" + (musl-chez-lib-dir))) + (if (file-exists? (car objs)) + (loop (cdr objs)) + (cons 'error + (format "CRT object not found: ~a" + (car objs)))))))]))] [else - (cons 'error "musl libkernel.a not found")])) + (cons 'error + (format "Cannot locate csv*/<machine> directory under ~a" + (musl-chez-prefix)))])) ;; ========== Link Command Generation ========== (define (musl-link-command output-path object-files static-libs) "Generate the musl-gcc link command for a static binary. + Uses musl-gcc -static which handles CRT objects and -lc automatically. + We only need to specify our object files, libkernel.a, and any extra + static libraries. + Parameters: output-path - Path for the output executable object-files - List of .o files to link @@ -187,42 +257,36 @@ Returns: Command string" (let* ([gcc (musl-gcc-path)] [libkernel (musl-libkernel-path)] - [crt (musl-crt-objects)] - [crt1 (car crt)] - [crti (cadr crt)] - [crtn (caddr crt)] - [sysroot (musl-sysroot)] + [libz (musl-libz-path)] + [liblz4 (musl-liblz4-path)] ;; Object files as space-separated string [objs (apply string-append (map (lambda (o) (format " '~a'" o)) object-files))] - ;; Static libraries - [libs (apply string-append - (map (lambda (a) (format " '~a'" a)) - static-libs))] + ;; Chez runtime archives + [chez-libs (apply string-append + (filter values + (list (format " '~a'" libkernel) + (and libz (format " '~a'" libz)) + (and liblz4 (format " '~a'" liblz4)))))] + + ;; User static libraries + [user-libs (apply string-append + (map (lambda (a) (format " '~a'" a)) + static-libs))] - ;; Standard libraries (provided by musl) - [std-libs "-lm -lpthread"]) + ;; Standard libraries needed by Chez runtime + [std-libs "-lm -lrt -lpthread"]) - ;; Full link command - ;; Note: Order matters! CRT objects must be first and last - (format "~a -static -nostdlib \ - '~a' '~a' \ - ~a \ - '~a' \ - ~a \ - ~a \ - '~a' \ - -o '~a'" + ;; musl-gcc -static handles CRT objects and libc linking automatically + (format "~a -static~a~a~a ~a -o '~a'" gcc - crt1 crti ;; CRT start objects - objs ;; Application objects - libkernel ;; Chez runtime - libs ;; User static libs (zlib, lz4, etc.) - std-libs ;; musl libc - crtn ;; CRT end object + objs ;; Application + Chez main.o + static_boot.o + chez-libs ;; Chez runtime archives + user-libs ;; User static libs + std-libs ;; Math, rt, pthreads output-path))) ;; ========== High-Level Build ========== @@ -230,12 +294,18 @@ (define (build-musl-binary source-path output-path . opts) "Build a fully static binary using musl libc. + Uses the Chez Scheme static build infrastructure: + - The installed main.o provides main() with argument parsing + - We generate static_boot.c with embedded boot files + - The application .so is bundled into an app boot file + Parameters: - source-path - Path to main .sls source file + source-path - Path to main .ss/.sls source file output-path - Path for output executable Keyword options: optimize-level: - Optimization level (0-3, default 2) + libdirs: - Library directories for compilation (colon-separated or list) static-libs: - Additional static libraries to link extra-c-files: - Additional C files to compile extra-cflags: - Additional C compiler flags @@ -250,14 +320,20 @@ ;; Parse options (let* ([opt-level (%musl-kwarg 'optimize-level: opts 2)] + [libdirs (%musl-kwarg 'libdirs: opts #f)] [static-libs (%musl-kwarg 'static-libs: opts '())] [extra-c (%musl-kwarg 'extra-c-files: opts '())] [extra-cflags (%musl-kwarg 'extra-cflags: opts "")] [verbose? (%musl-kwarg 'verbose: opts #f)] ;; Build directory - [build-dir (format "/tmp/jerboa-musl-~a" (current-time))] - [gcc (musl-gcc-path)]) + [build-dir (format "/tmp/jerboa-musl-~a" + (time-second (current-time)))] + [gcc (musl-gcc-path)] + [scheme-h-dir (let ([p (musl-scheme-h-path)]) + ;; directory containing scheme.h + (%musl-path-dir p))] + [chez-main-o (musl-main-o-path)]) ;; Create build directory (system (format "mkdir -p '~a'" build-dir)) @@ -267,38 +343,42 @@ (lambda () ;; Step 1: Compile Scheme to .so - (when verbose? (display "[1/5] Compiling Scheme...\n")) + (when verbose? (printf "[1/5] Compiling Scheme source: ~a~n" source-path)) (let ([so-path (format "~a/program.so" build-dir)]) (parameterize ([optimize-level opt-level] + [compile-imported-libraries #t] [generate-inspector-information #f]) (compile-program source-path so-path)) - ;; Step 2: Generate boot file + ;; Step 2: Create app boot file (bundles the .so into a boot file) (when verbose? (display "[2/5] Creating boot file...\n")) - (let ([app-boot (format "~a/app.boot" build-dir)] - [boots (musl-boot-files)]) + (let* ([boots (musl-boot-files)] + [app-boot (format "~a/app.boot" build-dir)]) (make-boot-file app-boot (list "petite" "scheme") so-path) - ;; Step 3: Generate C main with embedded boot files - (when verbose? (display "[3/5] Generating C...\n")) - (let ([main-c (format "~a/main.c" build-dir)]) - (generate-musl-main-c main-c - (map cdr boots) ;; boot file paths - app-boot - so-path) + ;; Step 3: Generate static_boot.c (embeds all boot files) + (when verbose? (display "[3/5] Generating static_boot.c...\n")) + (let ([static-boot-c (format "~a/static_boot.c" build-dir)]) + (generate-static-boot-c static-boot-c + (map cdr boots) ;; petite.boot, scheme.boot paths + app-boot) ;; Step 4: Compile C files (when verbose? (display "[4/5] Compiling C...\n")) - (let* ([main-o (format "~a/main.o" build-dir)] + (let* ([static-boot-o (format "~a/static_boot.o" build-dir)] + [include-flag (format "-I'~a'" scheme-h-dir)] [compile-cmd - (format "~a -c -static -O2 ~a -o '~a' '~a'" - gcc extra-cflags main-o main-c)] - [rc (system compile-cmd)]) + (format "~a -c -O2 ~a ~a -o '~a' '~a'" + gcc include-flag extra-cflags + static-boot-o static-boot-c)] + [rc (begin + (when verbose? (printf " ~a~n" compile-cmd)) + (system compile-cmd))]) (unless (= rc 0) (error 'build-musl-binary - "C compilation failed" + "static_boot.c compilation failed" compile-cmd)) ;; Compile extra C files @@ -306,9 +386,12 @@ (map (lambda (c-file) (let ([o-file (format "~a/~a.o" build-dir - (%musl-path-root (%musl-path-last c-file)))]) - (let ([cmd (format "~a -c -static -O2 ~a -o '~a' '~a'" - gcc extra-cflags o-file c-file)]) + (%musl-path-root + (%musl-path-last c-file)))]) + (let ([cmd (format "~a -c -O2 ~a ~a -o '~a' '~a'" + gcc include-flag extra-cflags + o-file c-file)]) + (when verbose? (printf " ~a~n" cmd)) (unless (= (system cmd) 0) (error 'build-musl-binary "Extra C compilation failed" @@ -318,12 +401,16 @@ ;; Step 5: Link (when verbose? (display "[5/5] Linking...\n")) - (let* ([all-objs (cons main-o extra-objs)] + (let* ([all-objs (cons* chez-main-o + static-boot-o + extra-objs)] [link-cmd (musl-link-command output-path all-objs static-libs)] - [rc (system link-cmd)]) + [rc (begin + (when verbose? (printf " ~a~n" link-cmd)) + (system link-cmd))]) (unless (= rc 0) (error 'build-musl-binary "Linking failed" @@ -331,7 +418,9 @@ ;; Success (when verbose? - (display (format "Built: ~a\n" output-path))) + (printf "~nBuilt: ~a~n" output-path) + (system (format "ls -lh '~a'" output-path)) + (system (format "file '~a'" output-path))) output-path))))))) ;; Cleanup @@ -340,23 +429,18 @@ ;; ========== C Code Generation ========== - (define (generate-musl-main-c output-path boot-paths app-boot-path so-path) - "Generate the C main() that embeds boot files and initializes Chez. + (define (generate-static-boot-c output-path boot-paths app-boot-path) + "Generate static_boot.c that provides static_boot_init(). - This is similar to generate-main-c from (jerboa build) but includes - musl-specific adjustments: - - No dlopen (all code is statically linked) - - memfd_create for loading the program .so" + This function is called by Chez's own main.o (compiled with + -DSTATIC_BOOT=static_boot_init) to register embedded boot files + before Sbuild_heap() is called. + + Boot files are embedded as C byte arrays using file->c-array + from (jerboa build)." (call-with-output-file output-path (lambda (out) - ;; Includes - (display "#define _GNU_SOURCE\n" out) - (display "#include <stdio.h>\n" out) - (display "#include <stdlib.h>\n" out) - (display "#include <string.h>\n" out) - (display "#include <unistd.h>\n" out) - (display "#include <sys/mman.h>\n" out) (display "#include \"scheme.h\"\n\n" out) ;; Embed boot files as C arrays @@ -373,58 +457,21 @@ (display (file->c-array app-boot-path "app_boot") out) (newline out) - ;; Embed program .so - (display (file->c-array so-path "program_so") out) - (newline out) - - ;; Main function - (display " -int main(int argc, char *argv[]) { - /* Save arguments in environment (bypass Chez arg parsing) */ - char buf[32]; - snprintf(buf, sizeof(buf), \"%d\", argc - 1); - setenv(\"JERBOA_ARGC\", buf, 1); - for (int i = 1; i < argc; i++) { - snprintf(buf, sizeof(buf), \"JERBOA_ARG%d\", i - 1); - setenv(buf, argv[i], 1); - } - - /* Initialize Chez Scheme */ - Sscheme_init(NULL); - - /* Register boot files from embedded data */ - Sregister_boot_file_bytes(\"petite\", petite_boot, petite_boot_len); - Sregister_boot_file_bytes(\"scheme\", scheme_boot, scheme_boot_len); - Sregister_boot_file_bytes(\"app\", app_boot, app_boot_len); - - /* Build the heap */ - Sbuild_heap(NULL, NULL); - - /* Load program via memfd (Linux-specific) */ - int fd = memfd_create(\"jerboa-program\", MFD_CLOEXEC); - if (fd < 0) { - perror(\"memfd_create\"); - return 1; - } - - if (write(fd, program_so, program_so_len) != (ssize_t)program_so_len) { - perror(\"write program\"); - return 1; - } - - char prog_path[64]; - snprintf(prog_path, sizeof(prog_path), \"/proc/self/fd/%d\", fd); - - /* Run the program */ - int status = Sscheme_script(prog_path, 0, NULL); - - /* Cleanup */ - close(fd); - Sscheme_deinit(); + ;; The static_boot_init function + (display "void static_boot_init(void) {\n" out) + (for-each + (lambda (boot-path) + (let ([name (%musl-path-root (%musl-path-last boot-path))]) + (fprintf out + " Sregister_boot_file_bytes(\"~a\", ~a_boot, ~a_boot_len);\n" + name name name))) + boot-paths) + (display + " Sregister_boot_file_bytes(\"app\", app_boot, app_boot_len);\n" + out) + (display "}\n" out))) - return status; -} -" out)))) + output-path) ;; ========== Cross-Compilation ========== @@ -466,11 +513,24 @@ int main(int argc, char *argv[]) { (let ([parts (%musl-string-split path #\/)]) (if (null? parts) path (car (reverse parts))))) + (define (%musl-path-dir path) + "Return the directory portion of a path" + (let ([idx (%musl-string-index-right path #\/)]) + (if idx (substring path 0 idx) "."))) + (define (%musl-path-root path) "Return path without extension" (let ([dot (%musl-string-index-right path #\.)]) (if dot (substring path 0 dot) path))) + (define (%musl-string-trim-right s) + (let loop ([i (- (string-length s) 1)]) + (if (< i 0) + "" + (if (char-whitespace? (string-ref s i)) + (loop (- i 1))