docs: add jpkg user guide + jerboa-* migration playbook
ober
06e983c45da3b531054379036ee23dffa2a215b6
new file mode 100644 --- /dev/null +++ b/docs/jpkg-guide.md @@ -0,0 +1,438 @@ +# jpkg — The Jerboa Package Manager (User & Reference Guide) + +`jpkg` is Jerboa's package manager. It ships inside the one `jerboa` +multicall binary (like `jmcp`/`jlsp`/`jerbuild`) and is selected either by +the `jpkg` name or as `jerboa pkg ...`. There is nothing extra to install: +if you have `jerboa`, you have `jpkg`. + +This document is the **user-facing guide and reference**. For the design +rationale and threat model, see [`jpkg-plan.md`](jpkg-plan.md). For moving +existing `jerboa-*` repos onto jpkg, see +[`jpkg-migration.md`](jpkg-migration.md). + +--- + +## 1. Why jpkg (what makes it different) + +Most package managers optimize for convenience and bolt on security later. +jpkg inverts that. It is a **secure-by-default data system**, not "npm for +Jerboa". The properties below are not optional add-ons — they are the +defaults, enforced by code with tests. + +- **Install never runs package code.** `jpkg install` is a pure data + operation: resolve → fetch → verify → extract → link. Arbitrary build or + "postinstall" scripts do not exist. Running code is a *separate*, + explicit, sandboxed `jpkg build`. +- **Reproducible artifacts.** `jpkg pack` produces **byte-identical** + `.jpkg` files for unchanged input — sorted paths, zeroed uid/gid/mtime, + fixed modes, deterministic gzip. Anyone can re-pack your source and get + the same SHA-256 (`jpkg verify --rebuild`). +- **Verify before you open.** A `.jpkg` is validated — digest, size, gzip + framing, then full ustar structure — *before a single byte is + extracted*. Path traversal, absolute paths, symlinks, hardlinks, device + files, setuid bits, and oversize archives are rejected up front. +- **The lockfile is the security boundary.** `jpkg.lock` pins exact + versions, registries, artifact digests, and manifest digests. + `jpkg install` materializes *exactly* the lock — it never silently + resolves something new. +- **Signed metadata, not trusted transport.** Registries use + [TUF](https://theupdateframework.io/): threshold-signed root/targets/ + snapshot/timestamp roles with **rollback** and **freeze** protection and + **key rotation**. Mirrors are untrusted; the client trusts signatures and + digests, not HTTPS. +- **Authorship you can check.** Releases carry an Ed25519 **package + signature** and optional **SLSA/in-toto provenance** (DSSE), checked + against per-scope publisher keys. A fail-closed policy + (`JPKG_REQUIRE_SIGNATURES` / `JPKG_REQUIRE_PROVENANCE`) refuses unsigned + installs. +- **Capabilities are declared and gated.** A package that wants native + code, build-time network, or FFI must *declare* it; the active policy + must *permit* it. Undeclared sharp edges are refused. +- **Advisories & yanks.** `jpkg audit` checks your locked graph against + OSV-shaped advisories and current yank status, blocking installs of + affected versions by policy. +- **Federation & transparency.** Scopes can be delegated to maintainer + keys with thresholds; namespaces can be transferred under registry + authority; every publish is appended to an **append-only, hash-chained + transparency log** with inclusion + consistency (anti-equivocation) + checks. +- **No native dependency for verification.** SHA-256, SHA-512 and Ed25519 + are pure Scheme (verified against NIST FIPS-180-4 and RFC 8032 vectors), + so verification behaves identically in a dev tree, the multicall binary, + and a fully static build. + +What jpkg explicitly does **not** do: no install-time lifecycle scripts; no +"clone a git repo and run it" as the default trust model; no unsigned +public-registry packages; no hidden network during builds; no global trust +in all transitive maintainers. It cannot prove signed code is benign — it +reduces ambiguity and blast radius, and makes bad acts attributable, +auditable, and revocable. + +--- + +## 2. The 60-second mental model + +``` +jpkg.sexp = your intent — dependencies as semver RANGES, plus identity +jpkg.lock = the exact answer — versions + digests, hash-pinned. COMMIT IT. +.jpkg = an immutable, signed, reproducible artifact (deterministic tar.gz) +registry = signed metadata (TUF) + content-addressed blobs +the store = ~/.jerboa/pkg — verified artifacts, shared across projects +.jpkg/deps = your project's environment: symlinks into the store +``` + +- `add` / `update` resolve ranges → write `jpkg.lock`. +- `install` reads `jpkg.lock` and only the lock — deterministic, offline- + friendly, CI-safe. +- Installing copies verified files; it never executes them. `build` runs + code, in a sandbox, under policy. + +--- + +## 3. Quick start (no registry required) + +Everything local works with zero infrastructure: + +```sh +mkdir demo && cd demo +jpkg init @me/demo # writes a minimal jpkg.sexp +jpkg pack # → me-demo-0.1.0.jpkg (reproducible) +jpkg verify me-demo-0.1.0.jpkg +jpkg verify --reproduce # prints the deterministic artifact digest +``` + +Scaffold a fresh package with source and docs: + +```sh +jpkg new @me/hello # creates hello/{jpkg.sexp,src/main.ss,README.md} +cd hello && jpkg build # sandboxed build + deterministic attestation +``` + +--- + +## 4. Project files + +### 4.1 `jpkg.sexp` — the manifest (data, never code) + +The manifest is a Jerboa-readable **data** file, parsed by a guarded reader +with depth/size budgets. Unknown fields, duplicate fields, and malformed +shapes are hard errors. + +```scheme +(package + (name "@scope/name") ; required; @scope/name, lowercase + (version "0.1.0") ; required; strict semver 2.0.0 + (description "short text") + (license "Apache-2.0") + (source "https://example.org/...") ; must be https:// + (jerboa ">=0.1.0") ; required Jerboa version range + (modules ((root "src") ; library root inside the package + (exports ((name main) ; exported module paths (symbols) + (name util))))) + (dependencies (("@scope/dep" "^1.2.0"))) + (dev-dependencies (("@scope/test-helper" "^0.3.0"))) + (capabilities ; declare sharp edges (see §9) + ((native-code reason: "sqlite extension") + (network build: #f test: #t) + (ffi libraries: ("sqlite3")) + (executables ("bin/tool"))))) +``` + +Names are `@scope/name` (optionally `@scope/collection.subpackage`), +lowercase `[a-z0-9]` segments with single internal `-`/`.`. This identifies +the *distribution unit*; **module import names are unchanged** — package +names and module names are different namespaces. + +The manifest has two canonical renderings, both produced by jpkg: +- a normalized **sexp** form (embedded in artifacts, stable round-trip), +- **canonical JSON** (RFC 8785 subset) — the signing subject. + +### 4.2 `jpkg.lock` — the resolved graph (commit this) + +Generated by `add`/`update`. One entry per package: name, version, +registry, artifact-sha256, artifact-size, manifest-sha256, dependencies, +yank status — or, for a dev link, a `link` path with no digests (flagged +non-reproducible). Entries are sorted; the file round-trips deterministically. + +`jpkg install` installs **exactly** this and nothing else. + +### 4.3 `jpkg.policy.sexp` — optional project policy override + +```scheme +(policy + (mode dev) ; default | strict | dev | offline (not unsafe) + (allow (native-build) (ffi) (local-deps)) + (require (signatures) (provenance))) +``` + +`allow` may only **loosen within** what the mode permits; `require` may only +**tighten**. `unsafe` is never selectable from a file — it must be an +explicit `--policy unsafe` on the command line. See §8. + +--- + +## 5. Command reference + +Global: `jpkg --help`, `jpkg --version`. Every command also works as +`jerboa pkg <command>`. + +### Project lifecycle +| Command | What it does | +|---|---| +| `jpkg init [NAME]` | Create `jpkg.sexp` (NAME defaults to `@local/<dir>`). | +| `jpkg new NAME` | Scaffold a new package directory (manifest + `src/main.ss` + README). | +| `jpkg add PKG[@VERSION]` | Add a dependency (range), resolve, write lock, install. | +| `jpkg remove PKG` | Drop a dependency, re-resolve, prune the environment. | +| `jpkg install` | Install **exactly** `jpkg.lock` (no resolution). | +| `jpkg update [PKG ...]` | Re-resolve to newer allowed versions. | +| `jpkg uninstall PKG` | Remove a package from the project environment only. | +| `jpkg list` | List locked packages. | +| `jpkg env [-- CMD ...]` | Print dep library paths, or run CMD with `JERBOA_PKG_PATH` set. | + +### Local development +| Command | What it does | +|---|---| +| `jpkg link PKG PATH` | Use a local checkout for PKG (lock records it as non-reproducible). | +| `jpkg unlink PKG` | Remove the local link. | + +`@VERSION` in `add` may be an exact version or any range (`^1.2`, `~1.2.3`, +`>=1 <2`, `1.2.3-rc.1`). With no version, jpkg pins `^<latest-stable>`. +Pre-releases are never selected implicitly — only by an exact pin. + +### Artifacts & verification +| Command | What it does | +|---|---| +| `jpkg pack [--output FILE]` | Build a deterministic `.jpkg` from the current project. | +| `jpkg verify` | Validate `jpkg.sexp` (+ lock/store status if present). | +| `jpkg verify --strict` | Also fail if any locked dep is a local link. | +| `jpkg verify FILE.jpkg` | Full artifact validation (digest, structure, embedded manifest). | +| `jpkg verify --reproduce` | Print this project's deterministic artifact digest. | +| `jpkg verify --rebuild DIGEST` | Prove the current source reproduces DIGEST. | + +### Build (sandboxed) +| Command | What it does | +|---|---| +| `jpkg build [--policy MODE]` | Compile under a policy-controlled sandbox; writes a build attestation. | +| `jpkg clean` | Remove `.jpkg/build`. | +| `jpkg policy [--policy MODE]` | Show the effective policy (what's allowed/required). | + +### Registry, publishing, audit +| Command | What it does | +|---|---| +| `jpkg search QUERY` | Search configured registries, ranked. | +| `jpkg dir list\|add NAME PATH\|remove NAME` | Manage the registry list (config file). | +| `jpkg publish --registry DIR --key FILE [--builder ID] [--source URL] [--no-provenance]` | Sign, attest, publish. | +| `jpkg publish --registry DIR --key FILE --keygen` | Generate a signing key file (mode 0600) and exit. | +| `jpkg audit` | Check the locked graph vs advisories, yanks, and the transparency log. | + +`jpkg audit` exit codes: `0` clean, `1` advisory-only finding, `2` a +finding **blocks** installs (advisory action `block-new`/`block-all`, or an +inconsistent transparency log). + +--- + +## 6. Registries + +A registry is a static, mirrorable directory tree. Point jpkg at one or +more, in priority order: + +```sh +export JERBOA_PKG_REGISTRIES="main=/srv/jpkg/main,internal=/srv/jpkg/internal" +# …or persist to the config file: +jpkg dir add main /srv/jpkg/main +jpkg dir list +``` + +A package name is **owned by the first configured registry that has it**; +other registries are never consulted for that name. Once a name is locked +to a registry, jpkg refuses to silently resolve it from a *different* +registry — this is the **dependency-confusion guard**. + +### Registry layout + +``` +metadata/root.json TUF root: keys + role thresholds (offline-signed) +metadata/timestamp.json freshness marker → snapshot +metadata/snapshot.json consistent view → targets (+ delegated roles) +metadata/targets.json signed file digests; may carry delegations +metadata/delegations/@scope.json per-scope delegated targets role +publishers.json trusted publisher keys + builders, per scope (a TUF target) +transparency.json append-only hash-chained publish log (a TUF target) +advisories/*.json OSV advisory records (TUF targets) +packages/@scope/name/1.2.3/release.json resolver metadata (digests, deps, yank) +packages/@scope/name/1.2.3/signature.json Ed25519 package signature +packages/@scope/name/1.2.3/provenance.json SLSA/in-toto DSSE provenance +blobs/sha256/<digest> the immutable .jpkg artifacts +``` + +A **plain** directory (no `metadata/`) works for purely local/self-hosted +dev: jpkg reads files directly. The moment `metadata/root.json` exists, the +registry is treated as **TUF** and every byte consumed must match signed +targets — directory listings become untrusted (so a mirror can't hide or +inject versions). + +### Local state (the store) + +``` +~/.jerboa/pkg/ (override with $JERBOA_PKG_HOME) + roots/<reg>.root.json pinned trusted TUF root (TOFU on first use) + registries/<reg>.state monotonic version state (rollback guard) + registries/config.sexp the `jpkg dir` registry list + store/sha256/<digest> verified, read-only artifacts (content-addressed) + src/<digest>/ unpacked source cache + transparency/<reg>.json last-seen log head (consistency monitoring) +``` + +Project environments under `.jpkg/deps/` are cheap **symlink views** into +the store, not copies. + +--- + +## 7. Publishing + +For a self-hosted or staging registry the flow is one command. The first +publish to an empty directory bootstraps a TUF registry where your key is +both the metadata authority and the scope publisher: + +```sh +# one-time: create a signing key (Ed25519 seed, 64 hex chars, mode 0600) +jpkg publish --registry /srv/jpkg/main --key ~/.keys/jpkg.key --keygen + +# publish the current project +cd my-package +jpkg publish --registry /srv/jpkg/main --key ~/.keys/jpkg.key \ + --builder "ci://my-org/build" --source "git://…/my-package" +``` + +Each publish: packs deterministically → signs the canonical subject → +emits SLSA/in-toto provenance (omit with `--no-provenance`) → writes +`release.json`/`signature.json`/`provenance.json` and the blob → appends to +the transparency log → re-signs TUF so the new files are covered targets. +Versions are **immutable** — re-publishing an existing version is refused. + +The official-registry model is trusted-CI publishing (OIDC + Sigstore); +that pipeline is out of scope for the local tooling, but the **client-side +verification is identical** — TUF + signature + provenance. + +--- + +## 8. Policy modes + +`jpkg policy` shows the effective policy; `--policy MODE` overrides it for a +command; `jpkg.policy.sexp` sets a project default. + +| Mode | Signatures | Provenance | Native build | Build network | Local deps | +|---|---|---|---|---|---| +| `default` | required | required | no | no | no | +| `strict` | required | required | no | no | no (and no exceptions) | +| `dev` | optional | optional | opt-in | opt-in | **yes** | +| `offline` | required | required | no | no | no (cached only) | +| `unsafe` | off | off | yes | yes | yes (CLI-only) | + +`strict` additionally pins identity and forbids policy exceptions and +local/path deps. `offline` requires already-cached metadata/artifacts. +`unsafe` records exceptions and is never a default. + +--- + +## 9. Capabilities (declared sharp edges) + +A package must *declare* anything beyond pure, hermetic Scheme: + +```scheme +(capabilities + (native-code reason: "sqlite extension") ; compiles native code + (network build: #f test: #t) ; network during build / test + (ffi libraries: ("sqlite3")) ; links foreign libraries + (executables ("bin/tool"))) ; ships executables +``` + +At `jpkg build`, the **gate** runs: each declared capability must be +*permitted* by the active policy, or the build is refused with a precise +reason. Undeclared capabilities are always refused. Example: + +``` +build refused: package declares native-code but policy default forbids +native builds (use a dev/unsafe policy or add (allow (native-build))) +``` + +### The build sandbox + +`jpkg build` runs the build command with: read-only sources + deps, one +writable build directory, **no network**, no ambient `$HOME`, deterministic +environment. Backends: Landlock (Linux), Seatbelt (macOS), Capsicum +(FreeBSD). On a platform with no kernel sandbox, the build still runs but +prints a clear degraded-isolation warning — and `strict` refuses rather +than pretend. Every build emits a deterministic, timestamp-free +**attestation** (`.jpkg/build/attestation.json`): policy, backend, +capabilities, command, and the build-log SHA-256. + +--- + +## 10. Audit, advisories, yanks + +`jpkg audit` walks `jpkg.lock` and reports, per package: + +- **Advisories** — OSV-shaped records affecting `name@version` (SEMVER + ranges, explicit versions, open-ended ranges), with a policy action + `warn` / `block-new` / `block-all` and the patched (`fixed`) versions. +- **Yanks** — versions the registry now marks yanked. Yanking excludes a + version from *new* resolutions but never breaks an existing lock + (reproducibility is preserved). +- **Transparency** — whether each locked release appears in the registry's + signed transparency log, and whether that log is **consistent** with the + last one you saw (equivocation / history-rewrite detection). + +Run it in CI: a non-zero exit on a blocking finding fails the pipeline. + +--- + +## 11. Federation & hardening + +- **Scope delegation.** The top targets role delegates `packages/@scope/*` + to per-scope **maintainer keys** with their own **threshold**. The client + follows the delegation, verifies the delegated role against that + authority and the snapshot, then accepts targets under the scope. A + mirror that swaps a delegated role file is caught by the snapshot hash. +- **Namespace transfer.** A scope can be re-delegated to new maintainer + keys under the top role's authority; the old keys can no longer produce + accepted metadata. This is threshold-maintainer ownership transfer. +- **Transparency monitoring.** The hash-chained log gives independent + monitors **inclusion** proofs (a release is recorded) and **consistency** + proofs (a new log extends the old one byte-for-byte). `jpkg audit` + performs both against a locally cached head. +- **Reproducible rebuild.** Because pack is deterministic, + `jpkg verify --rebuild <digest>` re-packs a source checkout and proves it + produces the published artifact — independent reproduction without + trusting the publisher's machine. + +--- + +## 12. Environment variables + +| Variable | Effect | +|---|---| +| `JERBOA_PKG_HOME` | Store + state root (default `~/.jerboa/pkg`). | +| `JERBOA_PKG_REGISTRIES` | `name=path,name=path` registry list (overrides the config file). | +| `JPKG_REQUIRE_SIGNATURES` | If set, installs fail closed unless the release has a valid publisher signature. | +| `JPKG_REQUIRE_PROVENANCE` | If set, installs also require valid SLSA/in-toto provenance. | +| `JERBOA_PKG_PATH` | Set by `jpkg env -- CMD`: `:`-joined dep library roots for the child. | + +(`JPKG_TUF_NOW` exists only to make expiry deterministic in tests.) + +--- + +## 13. Exit codes + +`0` success · `1` command failed · `2` usage error **or** an `audit` +blocking finding · `3` (legacy) unimplemented — no command returns this any +more. + +--- + +## 14. Where the code lives + +All under `lib/std/pkg/` in this repo; `(std pkg cli)` is the entry point +and transitively imports the rest. Tests are `tests/test-jpkg*.ss`, run with +`make test-jpkg`. The CLI is exposed three ways: the `jpkg` symlink next to +`jerboa`, `jerboa pkg ...`, and (in a dev checkout) `bin/jerboa pkg ...`. new file mode 100644 --- /dev/null +++ b/docs/jpkg-migration.md @@ -0,0 +1,256 @@ +# Migrating `jerboa-*` repos onto jpkg + +This guide explains how to put the existing `~/mine/jerboa-*` projects onto +`jpkg` — giving each one a real package identity, declared dependencies, a +reproducible signed artifact, and a path to share code through a registry +instead of sibling checkouts and vendored copies. + +It is written for the repos as they exist today: a `.jerbuild` config drives +`jerbuild build` to produce a binary, sources live under `src/` or a +named module dir (e.g. `gitsafe/`), and cross-repo code is shared by +checkout/vendoring rather than a declared dependency. + +**jpkg and `.jerbuild` are complementary, not competing.** Keep `.jerbuild` +for producing the final binary. jpkg adds the *package layer*: identity, +versioned dependencies, a verifiable distribution artifact, and audit. + +--- + +## 0. TL;DR + +For a leaf package (no jerboa-* dependencies): + +```sh +cd ~/mine/jerboa-gitsafe +jerboa pkg init @lisp/gitsafe # creates jpkg.sexp +# edit jpkg.sexp: set version, license, (modules ((root "gitsafe"))), jerboa req +jerboa pkg verify # manifest is valid +jerboa pkg pack # → lisp-gitsafe-X.Y.Z.jpkg (reproducible) +git add jpkg.sexp && git commit -m "jpkg: add package manifest" +``` + +For a package that depends on other jerboa-* repos: do the leaf steps for +each dependency first, stand up a local registry (§4), publish them, then +declare the dependencies in the dependent's `jpkg.sexp` and `jpkg add` them. + +--- + +## 1. Decide scope and identity + +All these repos live in the sourcehut `~lisp` namespace, so use the scope +**`@lisp`**. The package name should match the repo's purpose, not +necessarily the repo name: + +| Repo | Suggested package | +|---|---| +| `jerboa-gitsafe` | `@lisp/gitsafe` | +| `jerboa-websearch` | `@lisp/websearch` (or `@lisp/jerbsearch`) | +| `jerboa-secmonlib` | `@lisp/secmon` | +| `jerboa-crypto` | `@lisp/crypto` | + +Scopes are how delegation and publisher trust are organized later (§6), so +keep them consistent. + +--- + +## 2. Add a manifest (`jpkg.sexp`) + +Run `jerboa pkg init @lisp/<name>`, then edit. The key field is `modules`' +`root` — point it at the directory that actually holds the package's `.ss` +sources (this is what `pack` ships and what consumers import): + +```scheme +(package + (name "@lisp/gitsafe") + (version "0.3.0") ; pick a real semver; bump on release + (description "Git secret scanner") + (license "Apache-2.0") + (source "https://git.sr.ht/~lisp/jerboa-gitsafe") + (jerboa ">=0.1.0") + (modules ((root "gitsafe") ; gitsafe/*.ss (src/ for most repos) + (exports ((gitsafe main) (gitsafe scanner))))) + (dependencies ()) ; fill in §5 + (dev-dependencies ()) + (capabilities ())) ; declare sharp edges — §3 +``` + +Validate before committing: + +```sh +jerboa pkg verify # rejects unknown/duplicate fields, bad names/versions +``` + +> `init` defaults the version to `0.1.0` and license to `UNLICENSED`. Set +> both deliberately — the version is what consumers pin against. + +--- + +## 3. Declare capabilities honestly + +If the repo's `.jerbuild` has `rust-crates`, `ffi-symbols`, a `pre-build` +that fetches anything, or the package opens sockets/`load-shared-object` at +build/test, that is a **capability** and must be declared, or jpkg's build +gate will (correctly) refuse it under the default policy. + +Map the common `.jerbuild` features: + +| `.jerbuild` feature | jpkg capability | +|---|---| +| `(rust-crates …)` / native FFI shim | `(native-code reason: "…")` and/or `(ffi libraries: (…))` | +| `(pre-build "… curl …")` or any network in build | `(network build: #t …)` | +| ships a binary (`(output …)`) consumers run | `(executables ("…"))` | +| pure Scheme only | `(capabilities ())` — nothing to declare | + +Example for a repo with a Rust shim (e.g. `jerboa-websearch`): + +```scheme +(capabilities + (native-code reason: "jerboa-native-rs HTML parser / TLS") + (ffi libraries: ("jerboa_native"))) +``` + +Consumers then build it under `--policy dev` (or a `jpkg.policy.sexp` with +`(allow (native-build) (ffi))`), which makes the trust decision explicit +instead of implicit. + +--- + +## 4. Stand up a local registry (for cross-repo deps) + +Sibling-checkout / vendored dependencies become **declared** dependencies +resolved from a registry. For local and team use, a self-hosted registry is +one directory plus one key. + +```sh +mkdir -p ~/srv/jpkg/lisp +# one signing key for the @lisp scope (keep it safe; mode 0600 is set for you) +jerboa pkg publish --registry ~/srv/jpkg/lisp --key ~/.keys/lisp.key --keygen + +# point your shell at it (or: jerboa pkg dir add lisp ~/srv/jpkg/lisp) +export JERBOA_PKG_REGISTRIES="lisp=$HOME/srv/jpkg/lisp" +``` + +The first publish bootstraps a TUF registry whose root/targets/snapshot/ +timestamp authority and `@lisp` publisher key are all this key — fine for +self-hosting. (Split the roles later via key rotation + delegation, §6.) + +Publish each leaf package: + +```sh +cd ~/mine/jerboa-secmonlib +jerboa pkg publish --registry ~/srv/jpkg/lisp --key ~/.keys/lisp.key \ + --source "https://git.sr.ht/~lisp/jerboa-secmonlib" +``` + +Now `@lisp/secmon` is a verifiable, immutable release with a signature, +provenance, and a transparency-log entry. + +--- + +## 5. Convert cross-repo dependencies + +Suppose `jerboa-secmon` uses code currently checked out from +`jerboa-secmonlib`. After publishing `@lisp/secmon` (the lib): + +```sh +cd ~/mine/jerboa-secmon +jerboa pkg init @lisp/secmon-app # if not already +jerboa pkg add @lisp/secmon # resolves, writes jpkg.lock, installs +git add jpkg.sexp jpkg.lock +``` + +`jpkg add` writes `jpkg.lock` (commit it) and links the dependency under +`.jpkg/deps/`. To compile against it, put the dep roots on the library +path: + +```sh +jerboa pkg env -- scheme --libdirs "lib:$JERBOA_PKG_PATH" --script build.ss +# or read `jerboa pkg env` and add the printed paths to your .jerbuild libdirs +``` + +### Local iteration across repos (`link`) + +While you hack on both the app and the lib at once, don't republish on +every change — link the working checkout: + +```sh +cd ~/mine/jerboa-secmon +jerboa pkg link @lisp/secmon ~/mine/jerboa-secmonlib +# …edit the lib freely; the app sees changes immediately… +jerboa pkg unlink @lisp/secmon # back to the registry version +``` + +Linked deps are recorded in `jpkg.lock` as **non-reproducible**; +`jerboa pkg verify --strict` fails while a link is present, and `publish` +refuses to publish with unresolved links — so a link can never leak into a +release. + +--- + +## 6. Hardening a shared registry (optional, later) + +Once more than one person publishes, split trust: + +- **Delegate the scope** to maintainer keys with a threshold, so publishing + to `@lisp/*` doesn't require the registry root key. +- **Rotate** the root to offline-held keys. +- **Transfer** a package's namespace to a new maintainer set when ownership + changes. + +These use the TUF generator API (`tuf-registry-delegate!`, +`tuf-rotate-root!`, `tuf-transfer-scope!`) — see `jpkg-guide.md` §11 and the +tests in `tests/test-jpkg-federation.ss` for working examples. + +Consumers should then turn on fail-closed policy in CI: + +```sh +export JPKG_REQUIRE_SIGNATURES=1 +export JPKG_REQUIRE_PROVENANCE=1 # if your publishes include provenance +jerboa pkg install # refuses anything unsigned/unattested +jerboa pkg audit # advisories, yanks, transparency +``` + +--- + +## 7. Per-repo checklist + +For each `jerboa-*` repo: + +- [ ] `jerboa pkg init @lisp/<name>`; set version, license, `source`. +- [ ] Set `(modules ((root "<srcdir>") (exports …)))` to the real source dir. +- [ ] Declare `capabilities` matching `.jerbuild` (native/ffi/network/exec). +- [ ] `jerboa pkg verify` passes. +- [ ] `jerboa pkg pack` produces a `.jpkg`; `jerboa pkg verify FILE.jpkg` passes. +- [ ] Commit `jpkg.sexp` (and `jpkg.lock` once it has deps). +- [ ] Publish to the registry; confirm with `jerboa pkg search <name>`. +- [ ] Replace sibling/vendored deps with `jerboa pkg add @lisp/<dep>`. +- [ ] Wire dep paths into the build via `jerboa pkg env` / `.jerbuild` libdirs. +- [ ] In CI, set `JPKG_REQUIRE_SIGNATURES` and run `jerboa pkg audit`. + +--- + +## 8. What NOT to change + +- **Keep `.ss` sources as Jerboa** — jpkg ships your `src/`/module dir as-is; + it does not convert to `.sls`. +- **Keep `.jerbuild`** for the binary build. jpkg manages the package and + its dependencies; `jerbuild` still produces the executable. +- **Don't hand-edit `jpkg.lock`** — it round-trips deterministically and is + the security boundary; let `add`/`update` manage it. +- **Don't put secrets in the manifest** — it's public, signed data. Keys + live in key files (mode 0600), never in `jpkg.sexp`. + +--- + +## 9. Order of operations for the whole fleet + +1. Migrate **leaf** libraries first (no jerboa-* deps): `jerboa-crypto`, + `jerboa-secmonlib`, `jerboa-compat`, `jerboa-temp-dir`, etc. +2. Stand up the `~lisp` registry (§4) and publish those leaves. +3. Migrate **mid-tier** packages that depend on the leaves; `add` the deps. +4. Migrate **apps** (binaries) last; they pin everything via `jpkg.lock`. +5. Turn on fail-closed policy + `audit` in each repo's CI. + +Because each step ends in a committed `jpkg.sexp`/`jpkg.lock` and a verified +artifact, the fleet can be migrated incrementally — a half-migrated tree is +always in a buildable, releasable state. --- a/docs/jpkg-plan.md +++ b/docs/jpkg-plan.md @@ -1,5 +1,10 @@ # jpkg Package Manager Plan +> **Status: fully implemented (Phases 0–7).** This document is the design +> authority and threat model. For day-to-day usage and a complete command +> reference, see [`jpkg-guide.md`](jpkg-guide.md). For converting existing +> `jerboa-*` repos onto jpkg, see [`jpkg-migration.md`](jpkg-migration.md). + ## Purpose `jpkg` is the Jerboa package manager. It should be installed as a symlink or