updates
ober
2b36fae62829aed172cade9ae15e7a643091fefc
--- a/README.md +++ b/README.md @@ -27,6 +27,18 @@ menu, fetch the extras bundle and its selected dependencies, and produce make extras ``` +The optional bundle currently covers 20 selectable groups, including built-in +coreutils, mux, encrypted embed/pass/vault storage, SSH, YubiKey, AWS, +wormhole, recording, sandboxing, resource limits, profiling, proxying, +WireGuard, process watching, hardening, AWK, and sed. See +[`docs/extras.md`](docs/extras.md) for the complete command reference, +selection rules, platform requirements, security boundaries, and known +limitations. + +The top-level target uses `_vendor/jerboa-shell-extras/embed` as its embed +input. It does not automatically read or encrypt `~/.embed`; the extras guide +shows the explicit encrypted build command. + ## Test ```sh --- a/SECURITY.md +++ b/SECURITY.md @@ -41,6 +41,30 @@ production-readiness tracker are complete. capacity boundaries, launcher symlink/replacement attacks, unsafe loader candidates, and high-output command substitution. +## Optional Feature Bundle + +`make extras` fetches the separate `jerboa-shell-extras` project and its +dependencies and produces `jsh-extras`. That enhanced artifact is outside the +base binary's documented FFI and release-evidence scope unless it is audited +and tested separately. + +Depending on the selected features, `jsh-extras` may embed encrypted files, +handle passwords and SSH/cloud credentials, mount FUSE filesystems, access a +YubiKey, alter network interfaces, open mux/web-terminal/proxy listeners, +contact remote services, watch process metadata, or record terminal contents. +The sandbox, Landlock cage, resource-limit, and hardening features also have +platform-specific enforcement boundaries. Selecting a feature is not proof +that every advertised protection is supported by the target kernel or build. + +The top-level extras target uses an empty vendor-local embed directory by +default and does not read `~/.embed`. Supplying a personal embed directory is +an explicit release operation: review its contents, enable encryption, build +from a terminal, and treat both the generated source and final binary as +sensitive artifacts. + +See [`docs/extras.md`](docs/extras.md) for the full feature inventory, current +limitations, privilege requirements, and security notes. + ## Sensitive Data Shell commands, arguments, environment variables, redirection paths, history, new file mode 100644 --- /dev/null +++ b/docs/extras.md @@ -0,0 +1,502 @@ +# Optional feature bundle + +`jerboa-shell` keeps the default `jsh` binary small. The optional features are +maintained in the separate +[`jerboa-shell-extras`](https://git.sr.ht/~lisp/jerboa-shell-extras) project and +are fetched only when requested. They are built into a separate binary named +`jsh-extras`; they do not change the normal `jsh`, `jsh-macos`, or `make +binary` build. + +The extras build executes and compiles code from an external checkout plus its +dependencies. Review the security notes below before including credentials, +private keys, or privileged networking features. + +## Quick start + +From the root of this repository, run: + +```sh +make extras +./jsh-extras +``` + +`make extras` fetches or updates `_vendor/jerboa-shell-extras`, opens an +interactive selector, fetches the dependencies needed by the selected bundle, +and builds `./jsh-extras`. In the selector: + +- Up and Down move through the list. +- Space toggles the current feature. +- `a` selects all features. +- `n` selects no optional features. +- Enter saves the selection and starts the build. +- `q` exits without saving. + +Inside the enhanced shell, use: + +```text +,features +,help +``` + +`,features` reports what was compiled into that particular binary. `,help` +shows only the optional comma commands available in that build. Those two +commands are the authoritative runtime check; the existence of a dependency +or source file does not mean its feature was compiled in. + +## Feature selection + +The bundle resolves features in this order: + +1. `JSH_FEATURES` +2. `JERBOA_FEATURES`, retained as a compatibility alias +3. `features.conf`, or the path named by `JSH_FEATURES_CONF` +4. every known feature when no environment selection or config file exists + +Feature names may be separated by commas or spaces: + +```sh +JSH_FEATURES=aws,ssh make extras +JSH_FEATURES="embed pass yubikey" make extras +``` + +The aliases `all` and `full` enable everything. `none`, `minimal`, and `core` +disable every optional feature and leave only the base shell. An unknown name +is a build error. The `pass` feature automatically enables its declared +dependency, `embed`. + +The top-level `make extras` target still opens the interactive selector even +when `JSH_FEATURES` is set; the environment selection takes precedence when +the build is resolved. For unattended builds, first fetch the checkout and +then invoke its host build directly: + +```sh +make ensure-shell-extras +make -C _vendor/jerboa-shell-extras binary \ + JERBOA_SHELL_DIR="$PWD" \ + JSH_FEATURES=aws,ssh +``` + +That direct command leaves the platform-named binary in the extras checkout. +The top-level `make extras` target is the supported path that copies the result +to `./jsh-extras`. + +## Embedded files and encryption + +The top-level target deliberately passes this source directory to the extras +build: + +```text +_vendor/jerboa-shell-extras/embed +``` + +It creates the directory when necessary. Therefore, `make extras` does **not** +read or encrypt `~/.embed`, and an empty default directory does not trigger an +encryption-passphrase prompt. + +To build an enhanced shell from `~/.embed`, run the extras target directly +from an interactive terminal and explicitly enable encryption: + +```sh +make ensure-shell-extras +make -C _vendor/jerboa-shell-extras extras \ + JERBOA_SHELL_DIR="$PWD" \ + JSH_EMBED="$HOME/.embed" \ + JSH_EMBED_ENCRYPT=1 \ + EXTRAS_OUTPUT="$PWD/jsh-extras" +``` + +The generator reads the passphrase from `/dev/tty` with echo disabled and asks +for confirmation. It prompts only when standard input is a terminal. In a +noninteractive build it creates or reuses an empty embed table rather than +trying to read a passphrase. + +An already-generated encrypted embed table is reused without prompting. To +change the embedded files or encryption passphrase, remove the generated +`_vendor/jerboa-shell-extras/src/jsh/embed-data.sls` and rebuild. Do not commit +that generated file or any source secret from `~/.embed` to this repository. + +At runtime, encrypted files remain inaccessible until `,unlock` succeeds: + +```text +,unlock [--timeout 1h] +,lock +,privacy +``` + +`,lock` zeroes the active embed key and clears in-memory history. `,privacy` +toggles suppression of history and recording. The embed feature also provides +`embed-ls`, `embed-cat`, and `embed-fd` for the `//embed/` virtual path. + +## Complete feature reference + +The current bundle declares 20 selectable groups. Some groups add comma +commands, some register ordinary shell commands as builtins, and `harden` +changes startup/runtime behavior without adding a command. + +| Feature | User-visible surface | Purpose and requirements | +|---|---|---| +| `coreutils` | Ordinary commands such as `ls`, `cat`, `cp`, `grep`, `awk`, and `sed` | Registers in-process Rust uutils plus Scheme AWK and sed. See the inventory below. Native applets must be present in the built artifact. | +| `mux` | `,server`, `,attach`, `,servers`, `,mux`, `,mux-ssh-auth-sock`, `,webterm` | Local and remote terminal multiplexing, TLS/mTLS, routing, SSH-agent sharing, and a browser terminal. Opens sockets and may consume embedded certificate/key material. | +| `embed` | `,unlock`, `,lock`, `,privacy`; `embed-ls`, `embed-cat`, `embed-fd` | Encrypted `//embed/` virtual files, encrypted history after unlock, and privacy mode. Build-time input and TTY behavior are described above. | +| `ssh` | `,ssh`, `,scp`, `,ssh-list`, `,ssh-add`, `,ssh-agent` | In-process SSH/SFTP client and built-in SSH agent. Private keys may come from disk or `//embed`; host-key and credential handling remain security-sensitive. | +| `yubikey` | `,yk`, `,yubi`, `,unlock --yk` | Hardware-token presence, embed keyslot registration/unlock, and OATH account/code operations. Requires a supported YubiKey transport such as PC/SC or the target's native USB backend. | +| `aws` | `,aws`, `,pssm` | Jerboa AWS CLI and parallel SSM execution. Uses the credentials and region made available to the process; commands can mutate remote resources. | +| `worm` | `,worm` | Magic-wormhole text, file, directory, and SSH-key transfer plus mailbox/transit relay modes. Requires network access. | +| `pass` | `,pass-store`, `,pass`, `,pass-import-firefox`, `,pass-login`, `,login` | Password storage in the encrypted embed overlay and clipboard retrieval. Automatically enables `embed` and requires a successful `,unlock`. | +| `vault` | `,vault` | Creates, mounts, unmounts, and reports on an AES-256-GCM FUSE vault. Requires a working FUSE backend and mount permission. Current limitation: `,vault lock` and `,vault unlock` are advertised but return “not implemented”; unmount and mount instead. | +| `record` | `,record`, `,play` | Session recording, playback, indexing, reports, streaming, and live playback. Output can contain commands, terminal contents, paths, and secrets. | +| `sandbox` | `,sb`, `,(with-sandbox ...)` | Runs a child command/script or Scheme expression under the available platform sandbox backend. Policies and guarantees differ by OS; treat an unavailable backend as an error, not as equivalent protection. | +| `cage` | `,cage` | Irreversibly confines the running shell with Linux Landlock. This is Linux-specific and affects the shell itself, unlike `,sb`, which launches a child. | +| `rl` | `,rl` | Runs a command with a hard memory limit expressed as 1–100 percent of physical RAM. Linux uses cgroup v2/OOM handling; FreeBSD uses `RLIMIT_AS`; other targets may not provide equivalent enforcement. | +| `profiler` | `,profile`, `,trace`, `,debug` | Timing, pipeline visualization, and time-travel debugging. Diagnostic output may contain command text and runtime values. | +| `proxy` | `,proxy` | Local authenticated or unauthenticated SOCKS5 proxy, optionally routed through mux. Sets proxy environment variables while active and exposes a listening socket. | +| `wireguard` | `,wg` | Starts, stops, validates, and reports on WireGuard profiles through the native backend. Tunnel changes normally require OS networking privileges. | +| `procwatch` | `,procwatch`; ordinary `procwatch` | Watches process creation and parent/script ancestry, with filtering and JSONL output. Process metadata may expose command lines and host details. | +| `harden` | No command | Enables the bundle's anti-debug, integrity, and supported syscall-hardening paths. Availability is platform/build dependent and is not a substitute for a sandbox. | +| `awk` | No independent command registration | Currently a selection marker with no standalone runtime surface. The built-in `awk` command is registered by `coreutils`; select `coreutils` when that is the intended behavior. | +| `sed` | No independent command registration | Currently a selection marker with no standalone runtime surface. The built-in `sed` command is registered by `coreutils`; select `coreutils` when that is the intended behavior. | + +### Coreutils inventory + +With `coreutils` enabled, the enhanced shell registers these ordinary command +names when the corresponding native entry point is present: + +```text +ls dir vdir stat du df dircolors pathchk +cat cp mv rm ln mkdir rmdir mktemp touch link unlink readlink realpath +install shred truncate mkfifo mknod dd chmod chown chgrp +head tail tac tee wc nl fold expand unexpand fmt +cut paste join comm sort uniq tr numfmt grep +id whoami hostname uname uptime who groups users pinky logname +arch nproc tty hostid date seq expr factor +base64 base32 basenc od cksum md5sum sha1sum sha224sum sha256sum +sha384sum sha512sum b2sum sum +env timeout nice nohup chroot sleep yes printenv +shuf split csplit tsort stty pr ptx basename dirname +awk sed +``` + +`true`, `false`, `echo`, `printf`, `pwd`, `test`, `kill`, and `sync` are not +replaced by the optional Rust applets. Use each command's `--help` output for +its detailed option set. These implementations aim at familiar GNU/uutils +behavior but should not be assumed byte-for-byte identical to the host's GNU +or BSD utility. + +### SSH + +```text +,ssh (user@)host[:port] [-p port] [-i keyfile] [command ...] +,scp [-P port] [-i keyfile] [--password password] SOURCE DEST +,ssh-list +,ssh-add FILE +,ssh-agent [start|stop] +``` + +`,ssh` connects without spawning the system `ssh`. `,scp` uses the bundled +SFTP path. `,ssh-add` accepts filesystem or embedded key paths. Prefer secure +prompts or unlocked embedded keys over putting a password in command arguments, +because arguments may be recorded in history or process diagnostics. + +### AWS and SSM + +```text +,aws SERVICE ACTION [OPTIONS ...] +,pssm PATTERN COMMAND [ARGS ...] +``` + +The commands forward their arguments to the bundled `jerboa-aws` entry points. +With no `,pssm` arguments, its upstream help is displayed. Authentication, +region selection, service/action support, and exit behavior come from that +package. Run read-only identity checks such as `,aws sts get-caller-identity` +before issuing mutating operations. + +### Vault + +```text +,vault create FILE [BLOCKS] +,vault mount FILE MOUNTPOINT +,vault unmount +,vault status +``` + +The default size is 1024 4096-byte blocks. Create asks for and confirms a +passphrase; mount asks for the existing passphrase. The mount trusts the `jsh` +process and its descendants. In the current implementation, use unmount/mount +as the lock/unlock lifecycle: the literal `,vault lock` and `,vault unlock` +subcommands are not implemented. + +### Resource limits + +```text +,rl PERCENT COMMAND [ARGS ...] +``` + +`PERCENT` must be from 1 through 100 and is calculated from total physical +memory. Example: + +```text +,rl 50 make -j8 +``` + +This is separate from the base shell's launch-policy `,limit` and `,limits` +commands, which support additional resource kinds and have their own +per-platform enforcement status. + +### WireGuard + +```text +,wg up [--name PROFILE] [--interface IFACE] [--dry-run] [--no-routes] [PROFILE|PATH|//embed/path.conf] +,wg down [PROFILE] +,wg status [PROFILE] +,wg validate [PROFILE|PATH|//embed/path.conf] +``` + +The default configuration is +`//embed/wireguard/dns2-client1.conf`. If that embedded path is absent, the +handler also checks `~/.embed/wireguard/dns2-client1.conf` for that default. +Use `validate` or `up --dry-run` before changing network state. + +### Multiplexer and web terminal + +The main mux forms are: + +```text +,server [--name NAME] [--auth] [--replace] +,attach [--name NAME] [--from OFFSET] +,servers +,mux server -p PORT [--no-auth] [--no-mtls] [--name NAME] +,mux attach HOST[:PORT] [-D PORT] [--no-mtls] [--ws] [--from OFFSET] +,mux router -p PORT [--no-auth] [--foreground] +,mux list +,mux gen-certs [--ip IP] [--days N] [--cert PATH] [--key PATH] +,mux-ssh-auth-sock [PATH|--clear] +,webterm [-p PORT] +``` + +The attach prefix is Ctrl-b by default. Common suffix keys are `d` to detach, +`c` for a new window, `n`/`p` for next/previous, `%` or `"` to split, and `x` +to kill a pane. Set `JSH_MUX_PREFIX=C-a` when Ctrl-b is intercepted. + +### Password store + +```text +,pass-store +,pass +,pass-import-firefox [PATH] +,pass-login [--password|--user|--url|--list] SITE +,login [--password|--user|--url|--list] SITE +``` + +The store is available only after `,unlock`. `,pass-store` prompts for a name +and secret. `,pass` retrieves a named secret and copies it to the platform +clipboard. `,login` and its `,pass-login` alias search imported login labels, +hosts, URLs, and usernames; the default copied field is the password. + +Firefox imports expect a JSON export containing URL, username, and password +fields. With no path, the command checks `//embed/firefox.json` and then +`~/.embed/firefox.json`. Clipboard tools and desktop sessions vary by OS, so +verify that clipboard clearing and retention meet the local security policy. + +### YubiKey + +`,yk` and `,yubi` are aliases: + +```text +,yubi +,yubi register +,yubi list +,yubi code ACCOUNT +,yubi change +,unlock --yk +``` + +The bare command reports device presence. `register` wraps the active embed +key into a YubiKey-backed keyslot, so first unlock the embed store with its +normal passphrase. `list` and `code` operate on OATH accounts. `change` changes +the OATH application password and prompts for the old and new values without +echo. A registered token does not replace the need to retain a recovery path +for the original encrypted data. + +### Wormhole transfers + +```text +,worm send [--text TEXT|FILE|DIR] +,worm receive CODE [--output PATH] +,worm relay [PORT] +,worm transit-relay [PORT] +,worm ssh invite +,worm ssh accept CODE +``` + +The send and receive paths use the bundled `jerboa-wormhole` protocol. File +and directory transfers may start a temporary transit relay when no relay is +configured. Set `WORMHOLE_TRANSIT_RELAY` to choose one explicitly, +`WORMHOLE_TRANSIT_HOST` when automatic address selection is wrong, or +`JSH_WORM_AUTO_TRANSIT=0` to disable automatic transit-relay startup. + +### Recording and playback + +```text +,record start [--stream node://host:port/cookie] [FILE] +,record stop|pause|mark|status +,record list|search|stats|report +,record index|export|replay +,play [--speed N] [--cat|--commands|--follow] FILE|last +,play --live node://host:port/cookie +``` + +Recordings use asciicast-style files and may be indexed for search and reports. +Streaming and live playback use a cookie-bearing actor URL. Treat recordings, +indexes, exported scripts, stream cookies, and output previews as sensitive; +privacy mode should be enabled before entering secrets rather than after they +have already been recorded. + +### Sandbox and cage + +Run one child under a sandbox policy: + +```text +,sb [-r PATH] [-w PATH] [-x COMMAND] [--net|--no-net] [-t MS] [-c COMMAND|-e EXPR|SCRIPT] +``` + +`-r`, `-w`, and `-x` are repeatable allow rules. Network access is denied by +default by the documented policy. Linux uses Landlock/seccomp paths; macOS and +FreeBSD use their available platform mechanisms. Test the required policy axes +on the actual release target. + +Confine the current shell irreversibly on Linux: + +```text +,cage PATH [-r PATH] [-w PATH] [-x PATH] [--no-net] [--no-sys] [--no-tmp] +``` + +The cage applies to `jsh` and all later descendants. Restart the shell to +leave it. `--no-sys` omits automatically allowed system paths and can prevent +dynamic programs, DNS, certificates, or terminal support from working. + +### Profiler and debugger + +```text +,trace [-c COMMAND|SCRIPT] +,profile [-c COMMAND|SCRIPT] +,debug [-c COMMAND|SCRIPT] +``` + +`,trace` prints parsed pipeline stages and wall time. `,profile` records command, +expansion, substitution, and selected compound-command timing. `,debug` records +events for stepping, rewinding, error inspection, and local-state inspection. +All three can expose command text and values in terminal output or saved logs. + +### SOCKS5 proxy + +```text +,proxy start [-p PORT] [--user USER --pass PASSWORD] +,proxy start --via-mux HOST:PORT [--mux NAME] [-p PORT] [--no-mtls] [--ws] +,proxy status +,proxy stop +``` + +The listener binds locally and sets upper- and lowercase `ALL_PROXY`, +`HTTP_PROXY`, and `HTTPS_PROXY` variables for child processes while active. +With `--via-mux`, the remote mux performs outbound connects. Passing a proxy +password on the command line can expose it through history or diagnostics; +use only on a trusted local session. + +### Process watcher + +```text +,procwatch [--seconds N] [--interval MS] [--user USER|UID] + [--all] [--jsonl] [--match TEXT] [--parent TEXT] + [--parents N] +``` + +The watcher reads `/proc` on Linux and uses `ps`-based sampling on macOS and +FreeBSD. It can report PID, PPID, user, command line, script/interpreter hints, +match reasons, and sampled ancestry. It does not intentionally read process +environments, but command lines themselves often contain secrets. + +### Hardening + +`harden` has no command. It enables supported anti-debug, integrity, and +syscall-filtering paths in applicable artifacts, primarily hardened static +Linux builds. Protections that depend on `/proc`, seccomp, executable hashes, +or target-specific startup code are not automatically equivalent on other +platforms. `JSH_DEV=1` disables the hardening path for development and must not +be set when evaluating a release candidate. + +## Platform and privilege notes + +The feature selector describes requested build contents, not a promise that +every host provides every runtime facility. + +- `cage` requires Linux Landlock. +- `sandbox` selects the backend available for the target OS; policies and + supported axes differ among Linux, macOS, and FreeBSD. +- `vault` requires FUSE libraries, device access, and permission to mount. +- `wireguard` requires its native library and sufficient networking privilege. +- `yubikey` requires supported hardware and a usable PC/SC or native USB + transport. +- `rl` has explicitly documented Linux and FreeBSD enforcement paths; do not + assume identical behavior elsewhere. +- `harden` protections depend on target, build mode, and kernel support. +- mux, proxy, AWS, wormhole, and remote SSH require network access and may open + listeners or contact external services. + +A build warning about a missing native archive or symbol means that selecting +the feature was not enough to make it operational. Check `,features`, exercise +the relevant command with a non-destructive operation, and retain the build +log with release evidence. + +## Security boundaries + +The enhanced binary has a much larger authority and dependency surface than +the base shell: + +- Embed generation can copy every file under its input directory into the + binary. Encryption protects those bytes at rest but does not make an + unlocked process harmless. +- Passwords, SSH keys, cloud credentials, YubiKey-derived key material, + recording data, and vault passphrases require separate lifecycle and backup + decisions. +- Mux, webterm, proxy, wormhole, SSH, AWS, and WireGuard create network-facing + or remote-control behavior. Bind addresses, authentication, certificate + validation, and firewall policy matter. +- Sandbox, cage, resource-limit, and hardening features have platform-specific + guarantees. A feature name is not evidence that a particular policy axis is + enforced. +- Session recordings, history, diagnostics, process watching, profiler output, + and audit logs can retain secrets even when the original command succeeds. +- The external extras checkout and its dependency graph are not covered by the + base shell's native-FFI audit or release evidence merely because `make + extras` was launched here. Audit and test the resulting enhanced artifact as + its own release candidate. + +`jsh-extras` remains a shell, not a security boundary. Run only code and +features you trust with the authority of your user account. + +## Troubleshooting + +No feature menu appears: + +- The selector requires a TTY. Run `make extras` directly in an interactive + terminal. + +No embed passphrase prompt appears: + +- The top-level target uses `_vendor/jerboa-shell-extras/embed`, not + `~/.embed`. +- The generator prompts only with a TTY. +- An existing encrypted generated table is reused without prompting. + +A command reports “not available in this build”: + +- Run `,features` and rebuild with the relevant feature selected. +- For `pass`, ensure both `pass` and its automatically added `embed` dependency + appear. +- For native features, inspect the build log for a missing library or symbol. + +Files appear unchanged after rebuilding: + +- Remove stale generated/compiled artifacts in the extras checkout before + debugging runtime behavior. Do not delete or modify the source + `~/.embed` directory as part of cleanup.