λgitsite

~ober/jerboa

Imported from ~/mine/jerboa

download snapshot

about

# Jerboa

Jerboa is a Scheme dialect built on Chez Scheme. It keeps the parts of
Gerbil that make day-to-day Scheme pleasant, adds a large Clojure-style
data and concurrency layer, and ships a production standard library on
top of native Chez code plus an optional memory-safe Rust backend.

All user-facing Jerboa code is written in `.ss` files:

```scheme
(import (jerboa prelude))

(def (hello name)
  (displayln (str "hello, " name)))

(hello "world")
```

Run a file with:

```bash
jerboa run hello.ss
```

Do not write `(library ...)` forms in user code. The `.sls` files in
this repository are implementation internals — **never create or hand-edit
`*.sls` files**; write `.ss` and let the build generate them. Likewise,
**never invoke the raw `scheme`/`chez`/`petite` binary** (no
`scheme --script`, `scheme --program`, `.chez/bin/scheme`, `$SCHEME`):
`jerboa run`, `jerboa eval`, `jerboa test`, and the `jerboa` REPL are the
only supported entry points.

## Quick Start

The prelude is the default way to write Jerboa. It exports the core
language, common data structures, strings, lists, hashes, result types,
iterators, JSON/CSV, paths, datetime, regex helpers, file I/O, pretty
printing, and compatibility aliases for common Scheme names.

```scheme
(import (jerboa prelude))

(defstruct point (x y))

(def (describe-point p)
  (match p
    ((? point? pt) (str "point(" (point-x pt) ", " (point-y pt) ")"))
    (_ "not a point")))

(def (main)
  (def p (make-point 3 4))
  (displayln (describe-point p))
  (displayln (sort '(5 1 3) <))
  (displayln (for/collect ([x (in-range 5)]) (* x x)))
  (displayln (-> " a,b,c " (string-trim) (string-split #\,)))
  (displayln (json-object->string '(1 2 3)))
  (displayln (unwrap (->? (ok 10) (+ 5) (* 2)))))

(main)
```

Important syntax note: square brackets are just parentheses, like
Gerbil and Chez. Use quoted lists such as `'(1 2 3)` when you want a
literal list.

## What Jerboa Includes

At this snapshot, `lib/std` plus `lib/jerboa` contain 647 `.ss` modules. The prelude
exports the everyday language; specialized libraries are imported from
`(std ...)`, `(jerboa ...)`, and related module trees.

| Area | Highlights |
|---|---|
| Core language | `def`, `def*`, `defn`, optional/rest args, typed parameters, `defrule`/`defrules`, `define-values`, sealed records, enums, active patterns, ergonomic casts, method dispatch, `match`/`match2`, contracts, `try`/`catch`/`finally`, `with-resource`, result types, threading macros, iterator macros. |
| Reader syntax | `[...]` as parentheses, `{method obj args}` method dispatch, trailing `name:` keywords, `:std/path` imports, heredoc strings, raw strings and regex reader support, optional Clojure reader mode via `#!cloj`. |
| Prelude data | Hash tables, alists/plists, list utilities, functional combinators, strings, paths, file I/O, JSON, CSV, datetime, formatting, pretty printing, regex helpers, atoms, volatiles, shared state, lightweight SQLite/TCP helpers. |
| Persistent and generic collections | Persistent maps/vectors/sets/queues, sorted maps/sets, HAMT maps, weak collections, ephemerons, immutable values, lazy sequences, generic collection protocols, relation operations, tables, dataframes, datalog, lenses, zippers, Specter-style navigation. |
| Clojure compatibility | Sequences, reducers, transducers, atoms, agents, refs/STM, protocols, multimethods, metadata, futures/promises/deref, EDN, walkers, zippers, nested data helpers, datafy, core.async-style CSP channels. |
| Concurrency | Native OS threads with no GIL, atomics, mutex/condition wrappers, M:N fibers, fiber-aware events/channels, async/await, structured concurrency, work pools, resource pools, barriers, wait groups, actor systems, supervisors, distributed actors, CRDTs, Raft, STM, CSP. |
| Networking and web | HTTP client/server, secure rustls HTTPS daemon, fiber HTTP server, WebSocket/fiber WebSocket, HTTP/2, DNS, routers, rate limiting, connection pools, sendfile/zero-copy paths, TCP/UDP/TLS/rustls, SSH, S3, SMTP, SOCKS5, gRPC, JSON-RPC, 9P, FastCGI, Rack-style web adapters, event streams. |
| Text and protocols | JSON and JSON Schema, CSV, YAML, XML, HTML/SXML, TOML, INI, EDN, safe s-expression wire data, MessagePack, CBOR, Transit, protobuf, base58/base64/hex, UTF-8/16/32, globbing, diffs, templates, regex, native regex, PCRE2, `rx`, PEG parsers. |
| Storage and databases | SQLite, PostgreSQL, DuckDB, LevelDB, DBI layer, query compilation, connection pools, mmap and mmap-btree helpers, content-addressed storage, image/closure persistence, package stores. |
| Native Rust backend | Optional `libjerboa_native` bindings for ring crypto, rustls TLS, Rust HTTP request parsing, ReDoS-resistant regex, flate2 compression, rusqlite/postgres/duckdb, secure memory, epoll, inotify, Landlock, packet capture, and panic-contained FFI entry points. |
| Security and capabilities | Audit logging, auth, cages, capability and capability-typed I/O, import audit, flow/taint tracking, IO interception, Landlock, seccomp, Capsicum, seatbelt, sandboxing, sanitizers, secret handling, privilege separation, safe/pure audit tools. |
| OS and runtime | Environment/path/fd/signal/process modules, errno/fcntl/flock, mmap, temp files, tty, aproc, exec identity, inotify, epoll, kqueue, io_uring, platform detection, supervised services. |
| Typed and effect systems | Typed parser/checker, Rust/LLVMIR wrappers, affine and linear types, refinement, phantom, GADT, HKT, rows, typeclasses, effect typing, deep/scoped/resource effects, contracts and chaperones. |
| Compiler and WASM | Compiler passes, partial evaluation, PGO, specialization, staging/comptime, secure compiler/link modules, WebAssembly format/codegen/runtime/GC/sandbox/WASI modules, Slang docs and secure WASM target work. |
| Tooling and operations | REPL/nREPL, LSP, docs generation, package manager, reproducible builds, SBOM and build verification, watch mode, logging, metrics, tracing/spans, health checks, circuit breakers, profiling, flamegraphs, heap/thread/timetravel debugging, fuzzing, QuickCheck, property tests, benchmarks. |

## Recent Additions

The latest standard-library push filled in several areas that the old
README did not cover:

- **Concurrency and scheduling:** `(std misc fiber)`, `(std misc event)`,
  `(std misc custodian)`, `(std misc delimited)`, enhanced
  `(std misc pool)`.
- **Data structures:** `(std misc persistent)`, `(std misc lazy-seq)`,
  `(std misc weak)`, `(std misc collection)`, `(std misc relation)`.
- **Serialization and protocols:** `(std text msgpack)`, `(std net 9p)`,
  `(std misc binary-type)`.
- **Testing and debugging:** `(std test)`, `(std test quickcheck)`,
  `(std test check)`, `(std misc profile)`, `(std misc equiv)`,
  `(std misc diff)`.
- **Metaprogramming:** `(std misc typeclass)`, `(std misc ck-macros)`,
  `(std misc fmt)`, `(std misc chaperone)`, `(std misc advice)`.
- **System utilities:** `(std misc config)`, `(std misc cont-marks)`,
  `(std misc terminal)`, `(std misc highlight)`,
  `(std misc guardian-pool)`, `(std misc amb)`,
  `(std misc memoize)`.

See [`docs/whats-new.md`](docs/whats-new.md) for the detailed module
notes and examples.

## Documentation

Use [`docs/index.md`](docs/index.md) as the maintained map for the language
repository. Stable language, library, build, and security references live at
top-level `docs/`; dated audits and review records live under
[`docs/reviews/`](docs/reviews/).

The shortest path through the docs is:

- New users: [`docs/quickstart.md`](docs/quickstart.md), then
  [`docs/JERBOA-LANG.md`](docs/JERBOA-LANG.md).
- Library users: [`docs/libraries.md`](docs/libraries.md), then the module
  family docs linked from it.
- Security and release reviewers:
  [`docs/security-reference.md`](docs/security-reference.md),
  [`docs/release-security.md`](docs/release-security.md), and
  [`docs/status.md`](docs/status.md).
  [`docs/kimi3-security-recommmendations.md`](docs/kimi3-security-recommmendations.md)
  is the active backlog, not the reference for implemented behavior.

## Examples

### Persistent HAMT Map

```scheme
(import (jerboa prelude)
        (std misc persistent))

(def users
  (hamt-set
    (hamt-set hamt-empty 'alice 1)
    'bob 2))

(displayln (hamt-ref users 'alice #f))
(displayln (hamt-contains? users 'bob))
```

### Property-Based Test

```scheme
(import (jerboa prelude)
        (std test check))

(def sort-preserves-length
  (for-all ([xs (gen:list (gen:integer))])
    (= (length (sort xs <)) (length xs))))

(displayln (check-property 100 sort-preserves-length))
```

## Why Chez Scheme

Jerboa uses Chez because it has the runtime properties a systems
language needs:

- Real OS threads with no global interpreter lock.
- Thread-safe generational GC and cheap compiled closures.
- Fast fixnum arithmetic and bytevector-heavy protocol code.
- Stable FFI via `foreign-procedure` and `foreign-callable`.
- A mature native-code compiler with whole-program optimization paths.
- Apache 2.0 licensing for Chez Scheme; see [`LICENSE-CHEZ`](LICENSE-CHEZ)
  and [`NOTICE`](NOTICE).

The repository can use stock Chez 10.x. It also vendors an additive
Chez tree under [`vendor/ChezScheme/`](vendor/ChezScheme) for primitives
the standard library can use without changing the user-facing language.

## Licensing and Notices

Jerboa vendors Chez Scheme under [`vendor/ChezScheme/`](vendor/ChezScheme).
Chez Scheme is licensed under Apache License 2.0, and its upstream NOTICE text
must remain available when Jerboa source or binary artifacts redistribute
Chez-derived code. Keep [`LICENSE-CHEZ`](LICENSE-CHEZ), [`NOTICE`](NOTICE),
`vendor/ChezScheme/LICENSE`, and `vendor/ChezScheme/NOTICE` with source
distributions. Jerboa release tarballs include the top-level notice files and
a `share/licenses/jerboa/` copy of the original Chez license and notice.

## Reader Syntax

Jerboa extends the Chez reader with Gerbil-inspired syntax:

```scheme
(let ([x 1] [y 2]) (+ x y))  ;; brackets are parentheses
{area circle}                 ;; -> (~ circle 'area)
name:                         ;; keyword object #:name
:std/net/request              ;; -> (std net request)

#<<END
multi-line string
END
```

Both import spellings work:

```scheme
(import :std/net/request)
(import (std net request))
```

## Gerbil and Clojure Compatibility

Most user-level Gerbil-style code works: `def`, `defstruct`, method
dispatch, `match`, `try`, `:std/*` module paths, and square-bracket
binding forms are first-class Jerboa syntax.

Jerboa is not the Gerbil expander on Chez. These do not carry over
unchanged:

- Gerbil expander internals such as `:gerbil/expander`.
- Gambit `##` primitives except where Jerboa provides explicit wrappers.
- Gerbil-specific export extensions unless translated.
- Gambit thread/runtime APIs that have no Chez equivalent.

The Clojure layer is provided as Jerboa libraries rather than by
embedding Clojure. It covers the common data, sequence, state, protocol,
multimethod, STM, and CSP idioms while staying inside Chez Scheme.

## Build and Test

Common local targets:

```bash
make build          # build core Jerboa libraries
make binary         # native binary build for macOS/FreeBSD/other local hosts
make audit          # production security and release-readiness gate
make release-evidence # audit, SBOM, reproducibility, and release smoke evidence
make test           # core test suite
make test-features  # feature-phase tests
make test-native    # Rust native backend tests
make test-all       # broad local test run
```

On Linux, the release pipeline also uses:

```bash
make podman-build
```

For MCP tooling data and portable MCP binaries:

```bash
make jmcp
make jmcp-portable
```

## Requirements and Platforms

- `jerboa` on `PATH`, or `bin/jerboa` from a source checkout.
- Optional Rust toolchain for `libjerboa_native` features.
- Optional legacy `chez-*` C libraries for older wrappers that remain
  available while the Rust backend supersedes them.

Supported platform work in this repository covers Linux glibc/musl,
FreeBSD, macOS on Intel and Apple Silicon, and Android/Termux.
Security confinement maps to the strongest local mechanism available:
Landlock/seccomp on Linux, Capsicum on FreeBSD, and sandbox-exec style
support on macOS.

## Project Structure

```text
lib/
  jerboa/          core reader, macros, runtime, prelude, package/build glue,
                   typed wrappers, WASM support
  std/             standard-library .ss modules
jerboa-native-rs/  optional Rust shared library backend
mcp/               active Jerboa MCP server
data/              MCP cookbook, API signatures, feature and changelog data
docs/              language, library, build, security, WASM, typed, and ops docs
tests/             Jerboa test suites
benchmarks/        benchmark harnesses
tools/             generators, linters, build and audit helpers
vendor/            vendored Chez Scheme tree
```

Start with the canonical documentation map:

- [`docs/index.md`](docs/index.md) for the maintained language-repository map.
- [`docs/quickstart.md`](docs/quickstart.md) for installing and running code.
- [`docs/JERBOA-LANG.md`](docs/JERBOA-LANG.md) for the language reference.
- [`docs/libraries.md`](docs/libraries.md) and
  [`docs/api-index.md`](docs/api-index.md) for standard-library coverage.
- [`docs/security-reference.md`](docs/security-reference.md) and
  [`docs/safety-guide.md`](docs/safety-guide.md) for implemented hardening.

## License

Jerboa is licensed under the Apache License 2.0.

Jerboa bundles a vendored copy of Chez Scheme
([`vendor/ChezScheme/`](vendor/ChezScheme)), also under the Apache
License 2.0. Its NOTICE and full license text are reproduced at
[`LICENSE-CHEZ`](LICENSE-CHEZ).

recent commits