Add musl static binary delivery documentation
ober
31e1a79cf7b0f81834257c30098f7aab873027a8
--- a/docs/implement.md +++ b/docs/implement.md @@ -2370,6 +2370,68 @@ Every build artifact is identified by the hash of its inputs. Same inputs → sa **Files**: `lib/jerboa/build/reproducible.sls` (~350 LOC) **Tests**: 15 tests +### 10.4 Static Binary Delivery with musl libc + +Build fully static executables with zero runtime dependencies using musl libc. These binaries run on any Linux system regardless of glibc version — ideal for deployment to containers, embedded systems, and legacy servers. + +```scheme +;; Build a fully static binary with musl +;; $ jerboa build --static --musl myapp.ss -o myapp-static +;; Linking with musl-gcc... +;; Built: myapp-static (4.2 MB, statically linked) + +;; Verify it has no dynamic dependencies +;; $ ldd myapp-static +;; => not a dynamic executable + +;; Programmatic API +(import (jerboa build)) + +(build-binary "src/server.sls" "dist/server" + 'static: #t + 'musl: #t) + +;; Cross-compile static ARM64 binary +(build-binary "src/server.sls" "dist/server-arm64" + 'static: #t + 'musl: #t + 'target: target-linux-aarch64) +``` + +**Implementation**: +- Detect musl toolchain: `musl-gcc` (wrapper) or `x86_64-linux-musl-gcc` (cross) +- Compile Chez Scheme runtime with musl: requires rebuilding `libkernel.a` against musl headers +- Link flags: `-static -nostdlib` + musl's `crt1.o`, `crti.o`, `crtn.o`, `libc.a` +- Handle musl-specific differences: + - No `dlopen` in static builds — all FFI must be linked at build time + - Thread-local storage via `__thread` instead of `pthread_key_create` + - Signal handling works identically (POSIX compliant) +- Alpine Linux compatibility: musl is the native libc, so binaries work unmodified +- Size optimization: musl binaries are typically 20-30% smaller than glibc static builds + +**Toolchain Setup**: +```bash +# Debian/Ubuntu: install musl toolchain +apt install musl-tools + +# Or build musl-cross-make for cross-compilation +git clone https://github.com/richfelker/musl-cross-make +cd musl-cross-make +make TARGET=aarch64-linux-musl install + +# Set path for cross builds +export PATH=$HOME/musl-cross/bin:$PATH +``` + +**Limitations**: +- No runtime `dlopen` (dynamic library loading) — all native extensions must be statically linked +- Name resolution: musl's `getaddrinfo` doesn't read `/etc/nsswitch.conf` (uses `/etc/resolv.conf` directly) +- Locale: musl has minimal locale support (C/POSIX only by default) +- Thread stack size: musl defaults to 80KB (vs glibc's 8MB) — may need `pthread_attr_setstacksize` + +**Files**: `lib/jerboa/build/musl.sls` (~300 LOC), `support/musl-chez-build.sh` (~150 LOC) +**Tests**: 12 tests + --- ## Implementation Order @@ -2442,7 +2504,8 @@ Every build artifact is identified by the hash of its inputs. Same inputs → sa | 32 | File watching + incremental build | 10.1 | 400 | High | | 33 | Cross-compilation pipeline | 10.2 | 400 | Medium | | 34 | Reproducible builds | 10.3 | 350 | Medium | -| **Subtotal** | | | **~3,350** | | +| 35 | Static musl builds | 10.4 | 450 | High | +| **Subtotal** | | | **~3,800** | | --- @@ -2455,8 +2518,8 @@ Every build artifact is identified by the hash of its inputs. Same inputs → sa | 4c: Systems | 2,800 | ~8 | ~150 | | 4d: Developer Experience | 4,000 | ~10 | ~200 | | 4e: Data & Distribution | 2,650 | ~7 | ~150 | -| 4f: Toolchain & Interop | 3,350 | ~8 | ~150 | -| **Total Phase 4** | **~21,200** | **~51** | **~1,030** | +| 4f: Toolchain & Interop | 3,800 | ~9 | ~162 | +| **Total Phase 4** | **~21,650** | **~52** | **~1,042** | Combined with existing ~52,000 lines across 138+ modules, Jerboa would be ~73,000 lines across ~190 modules with ~2,550 tests. Still dramatically more compact than Racket (~700K), Guile (~300K), or Gerbil+Gambit (~80K). new file mode 100644 --- /dev/null +++ b/docs/musl.md @@ -0,0 +1,1399 @@ +# Static Binary Delivery with musl libc + +This document provides comprehensive implementation details for building fully static Jerboa executables using musl libc. These binaries have zero runtime dependencies and run on any Linux system regardless of the installed glibc version. + +--- + +## Table of Contents + +1. [Overview and Motivation](#1-overview-and-motivation) +2. [Architecture](#2-architecture) +3. [Toolchain Requirements](#3-toolchain-requirements) +4. [Building Chez Scheme with musl](#4-building-chez-scheme-with-musl) +5. [The musl Build Module](#5-the-musl-build-module) +6. [Link Flags and CRT Objects](#6-link-flags-and-crt-objects) +7. [FFI Considerations](#7-ffi-considerations) +8. [Cross-Compilation with musl](#8-cross-compilation-with-musl) +9. [Runtime Differences from glibc](#9-runtime-differences-from-glibc) +10. [Testing Static Binaries](#10-testing-static-binaries) +11. [Troubleshooting](#11-troubleshooting) +12. [Implementation Checklist](#12-implementation-checklist) + +--- + +## 1. Overview and Motivation + +### Why musl? + +musl libc is a lightweight, fast, and standards-compliant C library implementation designed for static linking. Unlike glibc, which has complex NSS (Name Service Switch) plugins and dlopen dependencies that break static builds, musl is designed from the ground up to work correctly in fully static executables. + +### Benefits + +| Benefit | Description | +|---------|-------------| +| **Zero dependencies** | Binary runs on any Linux kernel (2.6.39+) without runtime libraries | +| **Smaller size** | musl binaries are typically 20-30% smaller than glibc static builds | +| **Reproducibility** | No dependency on system glibc version = reproducible deployments | +| **Container-friendly** | Works in `FROM scratch` Docker images (no base OS needed) | +| **Alpine native** | Alpine Linux uses musl as its system libc — zero compat issues | +| **Security** | Smaller attack surface, no dlopen gadgets for ROP chains | + +### Use Cases + +- **CLI tools**: Ship a single binary that works on any Linux distro +- **Embedded systems**: Deploy to resource-constrained devices +- **Legacy servers**: Run modern code on old RHEL/CentOS without library conflicts +- **Containers**: Minimal Docker images (< 10 MB total) +- **Air-gapped systems**: No need to install dependencies + +--- + +## 2. Architecture + +### Standard Jerboa Binary (Dynamic) + +``` +┌─────────────────────────────────────┐ +│ myapp (ELF executable) │ +├─────────────────────────────────────┤ +│ Embedded boot files (C arrays) │ +│ petite.boot, scheme.boot, app.boot│ +├─────────────────────────────────────┤ +│ Embedded program.so (C array) │ +├─────────────────────────────────────┤ +│ FFI shim code (compiled in) │ +└─────────────────────────────────────┘ + │ + │ Dynamic links to: + ▼ +┌─────────────────────────────────────┐ +│ libc.so.6 (glibc) │ +│ libm.so.6 │ +│ libpthread.so.0 │ +│ libdl.so.2 │ +│ libz.so.1 │ +│ liblz4.so.1 │ +└─────────────────────────────────────┘ +``` + +### musl Static Binary + +``` +┌─────────────────────────────────────┐ +│ myapp (ELF executable, static) │ +├─────────────────────────────────────┤ +│ Embedded boot files (C arrays) │ +├─────────────────────────────────────┤ +│ Embedded program.so (C array) │ +├─────────────────────────────────────┤ +│ FFI shim code │ +├─────────────────────────────────────┤ +│ libkernel.a (Chez runtime, musl) │ +├─────────────────────────────────────┤ +│ libc.a (musl) │ +│ libm.a (part of musl) │ +│ libpthread.a (part of musl) │ +├─────────────────────────────────────┤ +│ libz.a (zlib, static) │ +│ liblz4.a (lz4, static) │ +├─────────────────────────────────────┤ +│ CRT objects (crt1.o, crti.o, crtn.o)│ +└─────────────────────────────────────┘ + │ + │ NO dynamic links + ▼ + (nothing) +``` + +--- + +## 3. Toolchain Requirements + +### Host Packages (Debian/Ubuntu) + +```bash +# Install musl toolchain +sudo apt install musl-tools musl-dev + +# This provides: +# /usr/bin/musl-gcc — GCC wrapper that uses musl headers/libs +# /usr/lib/x86_64-linux-musl/ — musl libc.a, crt*.o, headers +# /usr/include/x86_64-linux-musl/ — musl headers + +# Verify installation +musl-gcc --version +ls /usr/lib/x86_64-linux-musl/ +``` + +### Host Packages (Fedora/RHEL) + +```bash +sudo dnf install musl-gcc musl-libc-static musl-devel +``` + +### Host Packages (Arch Linux) + +```bash +sudo pacman -S musl +``` + +### Manual musl Installation + +If packages are unavailable: + +```bash +# Download and build musl from source +wget https://musl.libc.org/releases/musl-1.2.5.tar.gz +tar xf musl-1.2.5.tar.gz +cd musl-1.2.5 + +# Configure with a custom prefix +./configure --prefix=/opt/musl --disable-shared + +# Build and install +make -j$(nproc) +sudo make install + +# Create the musl-gcc wrapper +cat > /opt/musl/bin/musl-gcc << 'EOF' +#!/bin/sh +exec "${REALGCC:-gcc}" "$@" -specs "/opt/musl/lib/musl-gcc.specs" +EOF +chmod +x /opt/musl/bin/musl-gcc + +# Add to PATH +export PATH=/opt/musl/bin:$PATH +``` + +### Static Libraries for Dependencies + +Chez Scheme requires zlib and lz4. These must also be static: + +```bash +# Install static versions (Debian/Ubuntu) +sudo apt install zlib1g-dev liblz4-dev + +# The static libraries are: +# /usr/lib/x86_64-linux-gnu/libz.a +# /usr/lib/x86_64-linux-gnu/liblz4.a + +# Or build from source with musl: +cd /tmp +wget https://zlib.net/zlib-1.3.1.tar.gz +tar xf zlib-1.3.1.tar.gz +cd zlib-1.3.1 +CC=musl-gcc ./configure --static --prefix=/opt/musl-libs +make && make install + +cd /tmp +wget https://github.com/lz4/lz4/archive/v1.9.4.tar.gz +tar xf v1.9.4.tar.gz +cd lz4-1.9.4 +make CC=musl-gcc PREFIX=/opt/musl-libs install +``` + +--- + +## 4. Building Chez Scheme with musl + +The Chez Scheme runtime (`libkernel.a`) must be rebuilt against musl headers and linked with musl's libc. This is the most complex part of the musl integration. + +### Step 1: Obtain Chez Scheme Source + +```bash +git clone https://github.com/cisco/ChezScheme.git +cd ChezScheme +git checkout v10.0.0 # or your version +``` + +### Step 2: Configure for Static musl Build + +```bash +# Set up environment for musl +export CC=musl-gcc +export CFLAGS="-static -O2" +export LDFLAGS="-static" + +# Configure Chez for the target architecture +./configure --threads --installprefix=/opt/chez-musl + +# Key: Edit the generated Mf-* makefile to use musl-gcc +# The makefile is in <machine>/s/Mf-<machine> (e.g., ta6le/s/Mf-ta6le) +``` + +### Step 3: Patch the Makefile for musl + +The Chez build system uses `gcc` directly. We need to override it: + +```bash +# Find the machine type +MACHINE=$(./configure --help | grep "machine type" | head -1 | awk '{print $NF}') +# Typically: ta6le (threaded, amd64, linux, elf) + +# Edit the makefile +cd $MACHINE/s +sed -i 's/^CC = gcc$/CC = musl-gcc/' Mf-$MACHINE +sed -i 's/^CFLAGS = /CFLAGS = -static /' Mf-$MACHINE + +# Also edit c/Mf-* for the C runtime +cd ../../c +sed -i 's/^CC = gcc$/CC = musl-gcc/' Mf-$MACHINE +``` + +### Step 4: Build Chez Scheme + +```bash +cd .. # back to ChezScheme root +make -j$(nproc) + +# The build produces: +# $MACHINE/boot/$MACHINE/petite.boot +# $MACHINE/boot/$MACHINE/scheme.boot +# $MACHINE/boot/$MACHINE/libkernel.a <-- This is the key artifact +``` + +### Step 5: Verify libkernel.a is musl-compatible + +```bash +# Check that libkernel.a doesn't reference glibc symbols +nm $MACHINE/boot/$MACHINE/libkernel.a | grep -E "@@GLIBC" +# Should produce NO output + +# Check for dynamic linking symbols +nm $MACHINE/boot/$MACHINE/libkernel.a | grep dlopen +# Should show 'U dlopen' (undefined) — we'll handle this +``` + +### Step 6: Install musl-Built Chez + +```bash +sudo make install + +# Files installed to /opt/chez-musl: +# /opt/chez-musl/lib/csv10.0.0/ta6le/petite.boot +# /opt/chez-musl/lib/csv10.0.0/ta6le/scheme.boot +# /opt/chez-musl/lib/csv10.0.0/ta6le/libkernel.a +# /opt/chez-musl/lib/csv10.0.0/ta6le/scheme.h +``` + +### musl Chez Build Script + +Create `support/musl-chez-build.sh`: + +```bash +#!/bin/bash +# musl-chez-build.sh — Build Chez Scheme with musl libc +set -euo pipefail + +CHEZ_VERSION="${1:-v10.0.0}" +INSTALL_PREFIX="${2:-/opt/chez-musl}" +BUILD_DIR="/tmp/chez-musl-build" + +# Check for musl-gcc +if ! command -v musl-gcc &>/dev/null; then + echo "ERROR: musl-gcc not found. Install musl-tools package." + exit 1 +fi + +# Clean and create build directory +rm -rf "$BUILD_DIR" +mkdir -p "$BUILD_DIR" +cd "$BUILD_DIR" + +# Clone Chez Scheme +echo "==> Cloning Chez Scheme $CHEZ_VERSION..." +git clone --depth 1 --branch "$CHEZ_VERSION" \ + https://github.com/cisco/ChezScheme.git +cd ChezScheme + +# Configure +echo "==> Configuring..." +./configure --threads --installprefix="$INSTALL_PREFIX" + +# Detect machine type +MACHINE=$(ls -d */ | grep -E '^[a-z]+[0-9]+[a-z]+$' | head -1 | tr -d '/') +echo "==> Machine type: $MACHINE" + +# Patch makefiles to use musl-gcc +echo "==> Patching makefiles for musl..." + +# Patch c/Mf-base (common C makefile) +if [ -f "c/Mf-base" ]; then + sed -i 's/^CC = gcc$/CC = musl-gcc/' c/Mf-base + sed -i 's/^CC = cc$/CC = musl-gcc/' c/Mf-base +fi + +# Patch machine-specific makefile +if [ -f "$MACHINE/s/Mf-$MACHINE" ]; then + sed -i 's/^CC = gcc$/CC = musl-gcc/' "$MACHINE/s/Mf-$MACHINE" +fi + +# Add static flags to CFLAGS +find . -name 'Mf-*' -exec sed -i 's/CFLAGS = /CFLAGS = -static /' {} \; + +# Build +echo "==> Building..." +make -j$(nproc) + +# Install +echo "==> Installing to $INSTALL_PREFIX..." +sudo make install + +# Verify +echo "==> Verifying musl build..." +if nm "$INSTALL_PREFIX/lib/csv"*/*/libkernel.a 2>/dev/null | grep -q "@@GLIBC"; then + echo "WARNING: libkernel.a contains glibc references" +else + echo "SUCCESS: libkernel.a is musl-compatible" +fi + +echo "==> musl Chez Scheme installed to $INSTALL_PREFIX" +echo " Boot files: $INSTALL_PREFIX/lib/csv*/*/*.boot" +echo " Runtime: $INSTALL_PREFIX/lib/csv*/*/libkernel.a" +``` + +--- + +## 5. The musl Build Module + +### Module Interface: `(jerboa build musl)` + +```scheme +(library (jerboa build musl) + (export + ;; Detection + musl-available? + musl-gcc-path + musl-sysroot + + ;; Configuration + musl-chez-prefix + musl-chez-prefix-set! + + ;; Build + build-musl-binary + musl-link-command + + ;; Paths + musl-libkernel-path + musl-boot-files + musl-crt-objects + + ;; Cross-compilation + make-musl-cross-target + musl-cross-targets) + + (import (chezscheme) + (jerboa build)) + + ...) +``` + +### Implementation: `lib/jerboa/build/musl.sls` + +```scheme +#!chezscheme +;;; (jerboa build musl) — Static Binary Delivery with musl libc +;;; +;;; Provides musl-specific build functionality for creating fully static +;;; executables with zero runtime dependencies. + +(library (jerboa build musl) + (export + ;; Detection + musl-available? + musl-gcc-path + musl-sysroot + + ;; Configuration + musl-chez-prefix + musl-chez-prefix-set! + + ;; Build + build-musl-binary + musl-link-command + + ;; Paths + musl-libkernel-path + musl-boot-files + musl-crt-objects + + ;; Validation + validate-musl-setup + + ;; Cross-compilation + make-musl-cross-target + musl-cross-available?) + + (import (chezscheme) + (jerboa build)) + + ;; ========== Configuration ========== + + ;; Path to musl-built Chez Scheme installation + ;; Default: /opt/chez-musl (can be overridden) + (define *musl-chez-prefix* + (make-parameter + (or (getenv "JERBOA_MUSL_CHEZ_PREFIX") + "/opt/chez-musl"))) + + (define (musl-chez-prefix) (*musl-chez-prefix*)) + (define (musl-chez-prefix-set! path) (*musl-chez-prefix* path)) + + ;; ========== Detection ========== + + (define (find-executable name) + "Search PATH for an executable, return full path or #f" + (let ([result (with-output-to-string + (lambda () + (system (format "which '~a' 2>/dev/null" name))))]) + (let ([trimmed (string-trim-right result)]) + (if (string=? trimmed "") #f trimmed)))) + + (define (string-trim-right s) + (let loop ([i (- (string-length s) 1)]) + (if (< i 0) + "" + (if (char-whitespace? (string-ref s i)) + (loop (- i 1)) + (substring s 0 (+ i 1)))))) + + (define (musl-gcc-path) + "Return path to musl-gcc wrapper, or #f if not found" + (or (find-executable "musl-gcc") + (find-executable "x86_64-linux-musl-gcc"))) + + (define (musl-available?) + "Check if musl toolchain is available" + (and (musl-gcc-path) #t)) + + (define (musl-sysroot) + "Return the musl sysroot directory" + ;; Query musl-gcc for its sysroot + (let ([gcc (musl-gcc-path)]) + (if gcc + (let ([result (with-output-to-string + (lambda () + (system (format "~a -print-sysroot 2>/dev/null" gcc))))]) + (let ([trimmed (string-trim-right result)]) + (if (string=? trimmed "") + ;; Fallback: standard musl location + "/usr/lib/x86_64-linux-musl" + trimmed))) + #f))) + + ;; ========== Path Resolution ========== + + (define (chez-machine-type) + "Return the Chez Scheme machine type (e.g., ta6le)" + (symbol->string (machine-type))) + + (define (musl-libkernel-path) + "Return path to musl-built libkernel.a" + (let* ([prefix (musl-chez-prefix)] + [machine (chez-machine-type)] + ;; Try common patterns + [paths (list + (format "~a/lib/csv~a/~a/libkernel.a" + prefix (scheme-version) machine) + (format "~a/lib/~a/libkernel.a" prefix machine) + (format "~a/libkernel.a" prefix))]) + (let loop ([ps paths]) + (if (null? ps) + (error 'musl-libkernel-path + "Cannot find musl libkernel.a" + (musl-chez-prefix)) + (if (file-exists? (car ps)) + (car ps) + (loop (cdr ps))))))) + + (define (musl-boot-files) + "Return list of (name . path) for musl-built boot files" + (let* ([prefix (musl-chez-prefix)] + [machine (chez-machine-type)] + [boot-dir (format "~a/lib/csv~a/~a" + prefix (scheme-version) machine)]) + (if (file-directory? boot-dir) + (list + (cons "petite" (format "~a/petite.boot" boot-dir)) + (cons "scheme" (format "~a/scheme.boot" boot-dir))) + (error 'musl-boot-files + "Cannot find musl boot directory" + boot-dir)))) + + (define (musl-crt-objects) + "Return paths to musl CRT objects needed for static linking" + (let ([sysroot (or (musl-sysroot) "/usr/lib/x86_64-linux-musl")]) + (list + (format "~a/crt1.o" sysroot) + (format "~a/crti.o" sysroot) + (format "~a/crtn.o" sysroot)))) + + ;; ========== Validation ========== + + (define (validate-musl-setup) + "Validate that musl toolchain is properly configured. + Returns (ok . message) or (error . message)" + (cond + [(not (musl-available?)) + (cons 'error "musl-gcc not found. Install musl-tools package.")] + + [(not (file-exists? (musl-chez-prefix))) + (cons 'error + (format "musl Chez prefix not found: ~a\n\ + Build Chez with musl or set JERBOA_MUSL_CHEZ_PREFIX" + (musl-chez-prefix)))] + + [(guard (e [#t #f]) (musl-libkernel-path) #t) + => (lambda (_) + (let ([crt (musl-crt-objects)]) + (let loop ([objs crt]) + (if (null? objs) + (cons 'ok "musl toolchain validated") + (if (file-exists? (car objs)) + (loop (cdr objs)) + (cons 'error + (format "CRT object not found: ~a" + (car objs))))))))] + + [else + (cons 'error "musl libkernel.a not found")])) + + ;; ========== Link Command Generation ========== + + (define (musl-link-command output-path object-files static-libs) + "Generate the musl-gcc link command for a static binary. + + Parameters: + output-path - Path for the output executable + object-files - List of .o files to link + static-libs - List of additional .a archives + + Returns: Command string" + (let* ([gcc (musl-gcc-path)] + [libkernel (musl-libkernel-path)] + [crt (musl-crt-objects)] + [crt1 (car crt)] + [crti (cadr crt)] + [crtn (caddr crt)] + [sysroot (musl-sysroot)] + + ;; Object files as space-separated string + [objs (apply string-append + (map (lambda (o) (format " '~a'" o)) + object-files))] + + ;; Static libraries + [libs (apply string-append + (map (lambda (a) (format " '~a'" a)) + static-libs))] + + ;; Standard libraries (provided by musl) + [std-libs "-lm -lpthread"]) + + ;; Full link command + ;; Note: Order matters! CRT objects must be first and last + (format "~a -static -nostdlib \ + '~a' '~a' \ + ~a \ + '~a' \ + ~a \ + ~a \ + '~a' \ + -o '~a'" + gcc + crt1 crti ;; CRT start objects + objs ;; Application objects + libkernel ;; Chez runtime + libs ;; User static libs (zlib, lz4, etc.) + std-libs ;; musl libc + crtn ;; CRT end object + output-path))) + + ;; ========== High-Level Build ========== + + (define (build-musl-binary source-path output-path . opts) + "Build a fully static binary using musl libc. + + Parameters: + source-path - Path to main .sls source file + output-path - Path for output executable + + Keyword options: + optimize-level: - Optimization level (0-3, default 2) + static-libs: - Additional static libraries to link + extra-c-files: - Additional C files to compile + extra-cflags: - Additional C compiler flags + verbose: - Print commands as they execute + + Returns: output-path on success, raises on error" + + ;; Validate setup first + (let ([status (validate-musl-setup)]) + (unless (eq? (car status) 'ok) + (error 'build-musl-binary (cdr status)))) + + ;; Parse options + (let* ([opt-level (kwarg 'optimize-level: opts 2)] + [static-libs (kwarg 'static-libs: opts '())] + [extra-c (kwarg 'extra-c-files: opts '())] + [extra-cflags (kwarg 'extra-cflags: opts "")] + [verbose? (kwarg 'verbose: opts #f)] + + ;; Build directory + [build-dir (format "/tmp/jerboa-musl-~a" (current-time))] + [gcc (musl-gcc-path)]) + + ;; Create build directory + (system (format "mkdir -p '~a'" build-dir)) + + (dynamic-wind + (lambda () #f) + + (lambda () + ;; Step 1: Compile Scheme to .so + (when verbose? (display "[1/5] Compiling Scheme...\n")) + (let ([so-path (format "~a/program.so" build-dir)]) + (parameterize ([optimize-level opt-level] + [generate-inspector-information #f]) + (compile-program source-path so-path)) + + ;; Step 2: Generate boot file + (when verbose? (display "[2/5] Creating boot file...\n")) + (let ([app-boot (format "~a/app.boot" build-dir)] + [boots (musl-boot-files)]) + (make-boot-file app-boot + (map car boots) ;; ("petite" "scheme") + so-path) + + ;; Step 3: Generate C main with embedded boot files + (when verbose? (display "[3/5] Generating C...\n")) + (let ([main-c (format "~a/main.c" build-dir)]) + (generate-musl-main-c main-c + (map cdr boots) ;; boot file paths + app-boot + so-path) + + ;; Step 4: Compile C files + (when verbose? (display "[4/5] Compiling C...\n")) + (let* ([main-o (format "~a/main.o" build-dir)] + [compile-cmd + (format "~a -c -static -O2 ~a -o '~a' '~a'" + gcc extra-cflags main-o main-c)] + [rc (system compile-cmd)]) + (unless (= rc 0) + (error 'build-musl-binary + "C compilation failed" + compile-cmd)) + + ;; Compile extra C files + (let ([extra-objs + (map (lambda (c-file) + (let ([o-file (format "~a/~a.o" + build-dir + (path-root (path-last c-file)))]) + (let ([cmd (format "~a -c -static -O2 ~a -o '~a' '~a'" + gcc extra-cflags o-file c-file)]) + (unless (= (system cmd) 0) + (error 'build-musl-binary + "Extra C compilation failed" + cmd)) + o-file))) + extra-c)]) + + ;; Step 5: Link + (when verbose? (display "[5/5] Linking...\n")) + (let* ([all-objs (cons main-o extra-objs)] + [link-cmd (musl-link-command + output-path + all-objs + static-libs)] + [rc (system link-cmd)]) + (unless (= rc 0) + (error 'build-musl-binary + "Linking failed" + link-cmd)) + + ;; Success + (when verbose? + (display (format "Built: ~a\n" output-path))) + output-path))))))) + + ;; Cleanup + (lambda () + (system (format "rm -rf '~a'" build-dir)))))) + + ;; ========== C Code Generation ========== + + (define (generate-musl-main-c output-path boot-paths app-boot-path so-path) + "Generate the C main() that embeds boot files and initializes Chez. + + This is similar to generate-main-c from (jerboa build) but includes + musl-specific adjustments: + - No dlopen (all code is statically linked) + - memfd_create for loading the program .so" + + (call-with-output-file output-path + (lambda (out) + ;; Includes + (display "#define _GNU_SOURCE\n" out) + (display "#include <stdio.h>\n" out) + (display "#include <stdlib.h>\n" out) + (display "#include <string.h>\n" out) + (display "#include <unistd.h>\n" out) + (display "#include <sys/mman.h>\n" out) + (display "#include \"scheme.h\"\n\n" out) + + ;; Embed boot files as C arrays + (for-each + (lambda (boot-path) + (let ([name (path-root (path-last boot-path))]) + (display (file->c-array boot-path + (format "~a_boot" name)) + out) + (newline out))) + boot-paths) + + ;; Embed app boot + (display (file->c-array app-boot-path "app_boot") out) + (newline out) + + ;; Embed program .so + (display (file->c-array so-path "program_so") out) + (newline out) + + ;; Main function + (display " +int main(int argc, char *argv[]) { + /* Save arguments in environment (bypass Chez arg parsing) */ + char buf[32]; + snprintf(buf, sizeof(buf), \"%d\", argc - 1); + setenv(\"JERBOA_ARGC\", buf, 1); + for (int i = 1; i < argc; i++) { + snprintf(buf, sizeof(buf), \"JERBOA_ARG%d\", i - 1); + setenv(buf, argv[i], 1); + } + + /* Initialize Chez Scheme */ + Sscheme_init(NULL); + + /* Register boot files from embedded data */ + Sregister_boot_file_bytes(\"petite\", petite_boot, petite_boot_len); + Sregister_boot_file_bytes(\"scheme\", scheme_boot, scheme_boot_len); + Sregister_boot_file_bytes(\"app\", app_boot, app_boot_len); + + /* Build the heap */ + Sbuild_heap(NULL, NULL); + + /* Load program via memfd (Linux-specific) */ + int fd = memfd_create(\"jerboa-program\", MFD_CLOEXEC); + if (fd < 0) { + perror(\"memfd_create\"); + return 1; + } + + if (write(fd, program_so, program_so_len) != (ssize_t)program_so_len) { + perror(\"write program\"); + return 1; + } + + char prog_path[64]; + snprintf(prog_path, sizeof(prog_path), \"/proc/self/fd/%d\", fd); + + /* Run the program */ + int status = Sscheme_script(prog_path, 0, NULL); + + /* Cleanup */ + close(fd); + Sscheme_deinit(); + + return status; +} +" out)))) + + ;; ========== Cross-Compilation ========== + + (define (make-musl-cross-target arch) + "Create a cross-compilation target for musl. + + Supported architectures: + 'x86-64 - x86_64-linux-musl-gcc + 'aarch64 - aarch64-linux-musl-gcc + 'armhf - arm-linux-musleabihf-gcc + 'riscv64 - riscv64-linux-musl-gcc" + (let ([prefix (case arch + [(x86-64) "x86_64-linux-musl"] + [(aarch64) "aarch64-linux-musl"] + [(armhf) "arm-linux-musleabihf"] + [(riscv64) "riscv64-linux-musl"] + [else (error 'make-musl-cross-target + "Unknown architecture" arch)])]) + (make-cross-target 'linux arch + (format "~a-gcc" prefix) + (format "~a-ar" prefix)))) + + (define (musl-cross-available? arch) + "Check if cross-compilation toolchain for arch is available" + (let ([target (make-musl-cross-target arch)]) + (and (find-executable (cross-target-cc target)) #t))) + + ;; ========== Helpers ========== + + (define (kwarg key opts . default-args) + (let ([default (if (null? default-args) #f (car default-args))]) + (let loop ([lst opts]) + (cond [(or (null? lst) (null? (cdr lst))) default] + [(eq? (car lst) key) (cadr lst)] + [else (loop (cddr lst))])))) + + (define (path-last path) + "Return the last component of a path" + (let ([parts (string-split path #\/)]) + (if (null? parts) path (car (reverse parts))))) + + (define (path-root path) + "Return path without extension" + (let ([dot (string-index-right path #\.)]) + (if dot (substring path 0 dot) path))) + + (define (string-split str char) + (let loop ([chars (string->list str)] [current '()] [result '()]) + (cond + [(null? chars) + (reverse (if (null? current) + result + (cons (list->string (reverse current)) result)))] + [(char=? (car chars) char) + (loop (cdr chars) + '() + (if (null? current) + result + (cons (list->string (reverse current)) result)))] + [else + (loop (cdr chars) (cons (car chars) current) result)]))) + + (define (string-index-right str char) + (let loop ([i (- (string-length str) 1)]) + (if (< i 0) + #f + (if (char=? (string-ref str i) char) + i + (loop (- i 1)))))) + + ) ;; end library +``` + +--- + +## 6. Link Flags and CRT Objects + +### Understanding the musl Link Line + +A musl static link requires careful ordering of objects: + +``` +musl-gcc -static -nostdlib \ + /usr/lib/x86_64-linux-musl/crt1.o \ # C runtime startup + /usr/lib/x86_64-linux-musl/crti.o \ # Init prologue + main.o \ # Application code + ffi-shim.o \ # FFI bindings + libkernel.a \ # Chez Scheme runtime + -lz -llz4 \ # Compression libs + -lm -lpthread \ # Math and threading + /usr/lib/x86_64-linux-musl/crtn.o \ # Init epilogue + -o myapp +``` +