Add comprehensive stdlib documentation and project roadmap
ober
7c2aa07379471f32141cfe1a8b3af317e1052423
new file mode 100644 --- /dev/null +++ b/docs/concurrency-extended.md @@ -0,0 +1,585 @@ +# Concurrency & Control Flow Extensions + +Advanced concurrency primitives, resource management, and control flow operators +for Jerboa's standard library. + +## Table of Contents + +- [Event System (`std misc event`)](#event-system) +- [Custodians (`std misc custodian`)](#custodians) +- [Resource Pool (`std misc pool`)](#resource-pool) +- [Delimited Continuations (`std misc delimited`)](#delimited-continuations) +- [Continuation Marks (`std misc cont-marks`)](#continuation-marks) +- [Non-deterministic Backtracking (`std misc amb`)](#non-deterministic-backtracking) + +--- + +## Event System + +**Module:** `(std misc event)` +**File:** `lib/std/misc/event.sls` + +```scheme +(import (std misc event)) +``` + +### Overview + +Events are lazy values that may or may not be ready. They can be composed with +`choice`, transformed with `wrap`/`handle`, and synchronized with `sync` and +`sync/timeout`. Channels provide synchronous rendezvous-style communication +between threads, built on events. + +The design follows the Concurrent ML (CML) model: events are first-class values +that represent potential communications. They are not "fired" until synchronized. + +### API Reference + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `make-event` | `(make-event poll-thunk)` | Create an event from a poll thunk. The thunk must return `(values ready? value)`. | +| `event-ready?` | `(event-ready? evt)` | Poll the event once; returns `#t` if it is currently ready. | +| `event-value` | `(event-value evt)` | Block (spin-wait with backoff) until the event fires, then return its value. | +| `sync` | `(sync evt ...)` | Wait for any one of the given events to fire. Returns the value of the first ready event. Blocks indefinitely. | +| `sync/timeout` | `(sync/timeout timeout-ms evt ...)` | Like `sync` but returns `#f` if no event fires within `timeout-ms` milliseconds. | +| `choice` | `(choice evt ...)` | Combine multiple events into one. When polled, tries each in order and returns the first ready value. | +| `wrap` | `(wrap evt proc)` | Transform an event's value: when `evt` fires with value `v`, the wrapped event fires with `(proc v)`. | +| `handle` | `(handle evt proc)` | Alias for `wrap`. | +| `always-event` | `(always-event val)` | An event that is immediately ready with `val`. | +| `never-event` | `never-event` | A value (not a procedure). An event that is never ready. | +| `timer-event` | `(timer-event delay-ms)` | An event that fires with `#t` after `delay-ms` milliseconds. | +| `make-channel` | `(make-channel)` | Create a synchronous rendezvous channel. | +| `channel-send` | `(channel-send ch val)` | Blocking send: deposits `val` and waits until a receiver consumes it. | +| `channel-recv` | `(channel-recv ch)` | Blocking receive: waits for a sender and returns the sent value. | +| `channel-send-event` | `(channel-send-event ch val)` | An event for sending `val`. Fires (with `(void)`) when a receiver is waiting. | +| `channel-recv-event` | `(channel-recv-event ch)` | An event that fires with the received value when one is available. | + +### Examples + +**Basic event polling:** + +```scheme +(import (std misc event)) + +;; An event that is always ready +(let ([e (always-event 42)]) + (event-ready? e)) ; => #t + (event-value e)) ; => 42 + +;; never-event is never ready +(event-ready? never-event) ; => #f +``` + +**Timer with sync/timeout:** + +```scheme +;; Wait up to 500ms for a timer that fires at 100ms +(sync/timeout 500 (timer-event 100)) ; => #t (the timer's value) + +;; Timeout expires before the timer fires +(sync/timeout 10 (timer-event 5000)) ; => #f +``` + +**Choosing between events:** + +```scheme +;; First-ready-wins: timer vs. channel receive +(let ([ch (make-channel)]) + ;; In another thread: (channel-send ch "hello") + (sync + (wrap (timer-event 1000) (lambda (v) 'timeout)) + (wrap (channel-recv-event ch) (lambda (msg) (list 'got msg))))) +;; => 'timeout if nothing sent within 1s, or '(got "hello") if sent +``` + +**Rendezvous channels between threads:** + +```scheme +(import (std misc event) (chezscheme)) + +(let ([ch (make-channel)]) + ;; Producer thread + (fork-thread + (lambda () + (channel-send ch "hello") + (channel-send ch "world"))) + ;; Consumer + (let ([a (channel-recv ch)] + [b (channel-recv ch)]) + (list a b))) +;; => ("hello" "world") +``` + +**Custom event from a poll thunk:** + +```scheme +;; An event that fires when a file exists +(define (file-exists-event path) + (make-event + (lambda () + (if (file-exists? path) + (values #t path) + (values #f #f))))) + +(sync/timeout 5000 (file-exists-event "/tmp/ready.flag")) +``` + +--- + +## Custodians + +**Module:** `(std misc custodian)` +**File:** `lib/std/misc/custodian.sls` + +```scheme +(import (std misc custodian)) +``` + +### Overview + +Custodians are hierarchical resource groups inspired by Racket's custodian +model. Every managed resource (ports, handles, custom objects) belongs to a +custodian. Shutting down a custodian recursively shuts down all its children and +releases all their resources. The `with-custodian` form provides automatic +cleanup on normal exit, exceptions, or continuation escapes. + +### API Reference + +| Procedure / Syntax | Signature | Description | +|--------------------|-----------|-------------| +| `make-custodian` | `(make-custodian)` or `(make-custodian parent)` | Create a child custodian. Defaults to `(current-custodian)` as parent. | +| `custodian?` | `(custodian? x)` | Returns `#t` if `x` is a custodian. | +| `current-custodian` | `(current-custodian)` | Parameter holding the current custodian. | +| `custodian-register!` | `(custodian-register! resource shutdown-proc)` or `(custodian-register! custodian resource shutdown-proc)` | Register a resource with a shutdown thunk. Returns the resource. If custodian is omitted, uses `(current-custodian)`. | +| `custodian-shutdown-all` | `(custodian-shutdown-all c)` | Recursively shut down custodian `c`: close all resources, shut down all children, remove from parent. Errors during individual resource cleanup are swallowed so one failure does not prevent others from cleaning up. | +| `custodian-managed-list` | `(custodian-managed-list c)` | Return a list of all managed resources and child custodians for `c`. | +| `custodian-open-input-file` | `(custodian-open-input-file path)` or `(custodian-open-input-file path custodian)` | Open an input port registered with the custodian for automatic cleanup. | +| `custodian-open-output-file` | `(custodian-open-output-file path)` or `(custodian-open-output-file path custodian)` | Open an output port registered with the custodian for automatic cleanup. | +| `with-custodian` | `(with-custodian body ...)` | Run `body ...` under a fresh custodian. The custodian is shut down when the body exits (normally, by exception, or by continuation escape). | + +### Examples + +**Automatic cleanup with `with-custodian`:** + +```scheme +(import (std misc custodian)) + +(with-custodian + (let ([p (custodian-open-input-file "data.txt")]) + (read p))) +;; Port is automatically closed when with-custodian exits +``` + +**Hierarchical resource management:** + +```scheme +(let ([parent (make-custodian)]) + (parameterize ([current-custodian parent]) + (let ([child (make-custodian)]) + (parameterize ([current-custodian child]) + ;; Register resources under the child custodian + (let ([handle (list 'connection)]) + (custodian-register! handle (lambda () (display "closed\n"))))) + ;; Inspect what the parent manages + (custodian-managed-list parent))) + ;; Shut down the parent — all children and their resources are cleaned up + (custodian-shutdown-all parent)) +``` + +**Registering custom resources:** + +```scheme +(with-custodian + ;; Register a custom handle with a cleanup procedure + (let ([sock (open-tcp-connection "example.com" 80)]) + (custodian-register! sock (lambda () (close-port sock))) + (put-bytevector sock #vu8(71 69 84)) + (get-bytevector-all sock))) +;; sock is closed automatically on exit +``` + +--- + +## Resource Pool + +**Module:** `(std misc pool)` +**File:** `lib/std/misc/pool.sls` + +```scheme +(import (std misc pool)) +``` + +### Overview + +A thread-safe, generic resource pool. Resources are created on demand up to a +configurable maximum, reused when idle, and optionally evicted after an idle +timeout. The pool uses a mutex and condition variable internally, so `pool-acquire` +can block without busy-waiting when the pool is full. + +### API Reference + +| Procedure / Syntax | Signature | Description | +|--------------------|-----------|-------------| +| `make-pool` | `(make-pool creator destroyer max-size)` or `(make-pool creator destroyer max-size idle-timeout)` | Create a pool. `creator` is a thunk returning a new resource. `destroyer` takes a resource and frees it. `max-size` is the maximum total resources (idle + in-use). `idle-timeout` is `#f` (no expiry) or a number of seconds after which idle resources are destroyed. | +| `pool?` | `(pool? x)` | Returns `#t` if `x` is a pool. | +| `pool-acquire` | `(pool-acquire p)` or `(pool-acquire p timeout)` | Get a resource from the pool. Reuses an idle resource if available, creates a new one if below max, or blocks. `timeout` is `#f` (block forever) or seconds. Returns the resource, or `#f` on timeout. | +| `pool-release` | `(pool-release p resource)` | Return a resource to the pool, making it available for others. | +| `with-resource` | `(with-resource pool (var) body ...)` | Acquire a resource, bind it to `var`, evaluate `body ...`, and release the resource on exit (even if an exception is raised). Uses `dynamic-wind`. | +| `pool-drain` | `(pool-drain p)` | Destroy all idle resources. In-use resources are not affected. | +| `pool-stats` | `(pool-stats p)` | Returns an alist: `((total . N) (idle . N) (in-use . N))`. Also evicts expired idle resources before counting. | + +### Examples + +**Basic connection pool:** + +```scheme +(import (std misc pool)) + +(define db-pool + (make-pool + (lambda () (open-db-connection "localhost:5432")) ; creator + (lambda (c) (close-db-connection c)) ; destroyer + 10)) ; max 10 connections + +;; Acquire, use, release +(let ([conn (pool-acquire db-pool)]) + (query conn "SELECT 1") + (pool-release db-pool conn)) +``` + +**Using `with-resource` for automatic release:** + +```scheme +(with-resource db-pool (conn) + (query conn "SELECT * FROM users WHERE id = 1")) +;; conn is released back to the pool even on error +``` + +**Acquire with timeout:** + +```scheme +;; Wait at most 5 seconds for a resource +(let ([conn (pool-acquire db-pool 5)]) + (if conn + (begin (query conn "SELECT 1") + (pool-release db-pool conn)) + (display "pool exhausted, try again later\n"))) +``` + +**Idle timeout for resource eviction:** + +```scheme +;; Resources idle for more than 60 seconds are destroyed +(define pool + (make-pool + (lambda () (make-fresh-resource)) + (lambda (r) (destroy-resource r)) + 20 ; max-size + 60)) ; idle-timeout in seconds +``` + +**Pool statistics:** + +```scheme +(pool-stats db-pool) +;; => ((total . 3) (idle . 1) (in-use . 2)) +``` + +**Drain idle resources:** + +```scheme +(pool-drain db-pool) +;; All idle resources are destroyed; in-use resources are unaffected +(pool-stats db-pool) +;; => ((total . 2) (idle . 0) (in-use . 2)) +``` + +--- + +## Delimited Continuations + +**Module:** `(std misc delimited)` +**File:** `lib/std/misc/delimited.sls` + +```scheme +(import (std misc delimited)) +``` + +### Overview + +Provides delimited continuations via `reset`/`shift` (Danvy and Filinski style) +and a prompt-based API (`call-with-prompt`/`abort-to-prompt`). + +`reset` establishes a delimiter (prompt) around an expression. `shift` captures +the continuation up to the nearest enclosing `reset` as a procedure `k`. The +captured continuation can be called zero, one, or multiple times. + +The implementation uses the Filinski encoding on top of `call/cc`. + +### API Reference + +| Procedure / Syntax | Signature | Description | +|--------------------|-----------|-------------| +| `reset` | `(reset body ...)` | Establish a continuation delimiter. Returns the value of `body ...`, or the value passed to `shift` if `shift` does not invoke `k`. | +| `shift` | `(shift k body ...)` | Capture the continuation up to the nearest `reset` as `k`, then evaluate `body ...`. If `k` is never called, the `reset` returns the result of `body ...`. | +| `make-prompt-tag` | `(make-prompt-tag)` or `(make-prompt-tag name)` | Create a prompt tag for use with `call-with-prompt`. | +| `call-with-prompt` | `(call-with-prompt tag thunk handler)` | Run `thunk` under a prompt identified by `tag`. If `abort-to-prompt` is called with the same `tag`, control returns to the prompt and the result is the value(s) passed to `abort-to-prompt`. | +| `abort-to-prompt` | `(abort-to-prompt tag val ...)` | Abort to the nearest prompt matching `tag`, returning `val ...`. Raises an error if no matching prompt is found. | + +### Examples + +**Basic reset/shift:** + +```scheme +(import (std misc delimited)) + +;; shift captures the continuation (+ 1 []) up to reset +(reset (+ 1 (shift k (k 10)))) ; => 11 + +;; If k is not called, the reset returns the shift body's value +(reset (+ 1 (shift k 42))) ; => 42 + +;; k can be called multiple times +(reset (+ 1 (shift k (+ (k 10) (k 20))))) ; => 32 +;; (k 10) => 11, (k 20) => 21, 11 + 21 = 32 +``` + +**Building a list with shift:** + +```scheme +;; Collect elements via shift +(reset + (let ([x (shift k (cons 'a (k 'ignored)))]) + (let ([y (shift k (cons 'b (k 'ignored)))]) + '()))) +;; => (a b) +``` + +**Prompt-based abort:** + +```scheme +(let ([tag (make-prompt-tag 'my-prompt)]) + (call-with-prompt tag + (lambda () + (+ 1 (abort-to-prompt tag 99))) + (lambda (v) v))) +;; => 99 +``` + +**Simulating exceptions with shift:** + +```scheme +(define (try thunk handler) + (reset + (handler (shift k (k (thunk)))))) + +;; Not really needed with Chez's guard, but shows the pattern +``` + +--- + +## Continuation Marks + +**Module:** `(std misc cont-marks)` +**File:** `lib/std/misc/cont-marks.sls` + +```scheme +(import (std misc cont-marks)) +``` + +### Overview + +Continuation marks let you attach key-value metadata to continuation frames. +This module re-exports Chez Scheme's native continuation mark support with +SRFI 157 / Racket-compatible names. Continuation marks are useful for +implementing dynamic parameters, stack traces, profiling, and other +context-passing patterns without modifying function signatures. + +**Important:** Chez Scheme uses `eq?` for key comparison. Use symbols or fixnums +as keys. String or pair keys only work if the exact same object is used for +both setting and lookup. + +### API Reference + +| Procedure / Syntax | Signature | Description | +|--------------------|-----------|-------------| +| `with-continuation-mark` | `(with-continuation-mark key val body)` | Evaluate `body` in a context where the current continuation frame has `key` mapped to `val`. If a mark for `key` already exists on this frame, it is replaced. | +| `current-continuation-marks` | `(current-continuation-marks)` | Capture the full set of continuation marks from the current continuation. | +| `continuation-mark-set->list` | `(continuation-mark-set->list mark-set key)` | Extract all values for `key` from the mark set as a list, innermost first. | +| `continuation-mark-set-first` | `(continuation-mark-set-first mark-set key)` | Return the first (innermost) value for `key` from the mark set, or `#f` if not found. `mark-set` can be `#f` to use the current continuation marks. | +| `continuation-marks?` | `(continuation-marks? x)` | Returns `#t` if `x` is a continuation mark set. | +| `call-with-immediate-continuation-mark` | `(call-with-immediate-continuation-mark key default proc)` | Call `proc` with the value of `key` in the immediately enclosing continuation frame, or `default` if no mark for `key` exists. Note the argument order: key, default, proc. | + +### Examples + +**Basic mark and lookup:** + +```scheme +(import (std misc cont-marks)) + +(with-continuation-mark 'key 'val + (continuation-mark-set->list + (current-continuation-marks) + 'key)) +;; => (val) +``` + +**Nested marks accumulate:** + +```scheme +(with-continuation-mark 'depth 0 + (with-continuation-mark 'depth 1 + (with-continuation-mark 'depth 2 + (continuation-mark-set->list + (current-continuation-marks) + 'depth)))) +;; => (2 1 0) +``` + +**Tail-call mark replacement:** + +When marks are set in tail position relative to an existing mark on the same +frame, the old value is replaced rather than accumulated: + +```scheme +(with-continuation-mark 'k 'a + (with-continuation-mark 'k 'b ;; replaces 'a on the same frame + (continuation-mark-set-first #f 'k))) +;; => b +``` + +**Implementing a call stack trace:** + +```scheme +(define (traced name thunk) + (with-continuation-mark 'trace name + (thunk))) + +(define (current-trace) + (continuation-mark-set->list + (current-continuation-marks) + 'trace)) + +(traced 'foo + (lambda () + (traced 'bar + (lambda () + (current-trace))))) +;; => (bar foo) +``` + +**Immediate continuation mark:** + +```scheme +(with-continuation-mark 'ctx "request-123" + (call-with-immediate-continuation-mark 'ctx #f + (lambda (v) v))) +;; => "request-123" +``` + +--- + +## Non-deterministic Backtracking + +**Module:** `(std misc amb)` +**File:** `lib/std/misc/amb.sls` + +```scheme +(import (std misc amb)) +``` + +### Overview + +The `amb` operator implements McCarthy's ambiguous choice, enabling +non-deterministic programming with automatic backtracking. `amb` picks one of +its alternatives; if a later assertion fails, execution backtracks to the most +recent `amb` and tries the next alternative. This is useful for constraint +solving, search problems, and logic programming. + +### API Reference + +| Procedure / Syntax | Signature | Description | +|--------------------|-----------|-------------| +| `amb` | `(amb expr ...)` | Choose one of the expressions. If the current choice leads to failure, backtrack and try the next. `(amb)` with no arguments is equivalent to `(amb-fail)`. `(amb x)` with a single argument returns `x` directly. | +| `amb-fail` | `(amb-fail)` | Explicitly trigger backtracking. Raises an error if there are no remaining choice points. | +| `amb-assert` | `(amb-assert condition)` | If `condition` is `#f`, call `(amb-fail)` to backtrack. | +| `with-amb` | `(with-amb body ...)` | Run an amb computation. Returns the first successful result, or `#f` if no solution exists. | +| `amb-collect` | `(amb-collect body ...)` | Run an amb computation and collect all successful results into a list. | + +### Examples + +**Finding a solution:** + +```scheme +(import (std misc amb)) + +(with-amb + (let ([x (amb 1 2 3 4 5)] + [y (amb 1 2 3 4 5)]) + (amb-assert (= (+ x y) 7)) + (cons x y))) +;; => (2 . 5) +``` + +**Collecting all solutions:** + +```scheme +(amb-collect + (let ([x (amb 1 2 3 4 5)] + [y (amb 1 2 3 4 5)]) + (amb-assert (= (+ x y) 6)) + (cons x y))) +;; => ((1 . 5) (2 . 4) (3 . 3) (4 . 2) (5 . 1)) +``` + +**No solution returns `#f`:** + +```scheme +(with-amb + (let ([x (amb 1 2 3)]) + (amb-assert (> x 10)) + x)) +;; => #f +``` + +**Pythagorean triples:** + +```scheme +(define (iota-from a b) + ;; Returns list (a a+1 ... b) + (if (> a b) '() (cons a (iota-from (+ a 1) b)))) + +(amb-collect + (let* ([a (apply amb (iota-from 1 20))] + [b (apply amb (iota-from a 20))] + [c (apply amb (iota-from b 20))]) + (amb-assert (= (+ (* a a) (* b b)) (* c c))) + (list a b c))) +;; => ((3 4 5) (5 12 13) (6 8 10) (8 15 17) (9 12 15) (12 16 20)) +``` + +**Map coloring (constraint satisfaction):** + +```scheme +(with-amb + (let ([wa (amb 'red 'green 'blue)] + [nt (amb 'red 'green 'blue)] + [sa (amb 'red 'green 'blue)] + [q (amb 'red 'green 'blue)] + [nsw (amb 'red 'green 'blue)] + [v (amb 'red 'green 'blue)] + [t (amb 'red 'green 'blue)]) + ;; Adjacent regions must differ + (amb-assert (not (eq? wa nt))) + (amb-assert (not (eq? wa sa))) + (amb-assert (not (eq? nt sa))) + (amb-assert (not (eq? nt q))) + (amb-assert (not (eq? sa q))) + (amb-assert (not (eq? sa nsw))) + (amb-assert (not (eq? sa v))) + (amb-assert (not (eq? q nsw))) + (amb-assert (not (eq? nsw v))) + (list (cons 'WA wa) (cons 'NT nt) (cons 'SA sa) + (cons 'Q q) (cons 'NSW nsw) (cons 'V v) (cons 'T t)))) +;; => ((WA . red) (NT . green) (SA . blue) (Q . red) (NSW . red) (V . green) (T . red)) +``` new file mode 100644 --- /dev/null +++ b/docs/data-structures.md @@ -0,0 +1,633 @@ +# Data Structures and Algorithms + +Advanced data structures and algorithms in the jerboa standard library. + +## Table of Contents + +- [Persistent Hash Maps](#persistent-hash-maps) -- `(std misc persistent)` +- [Lazy Sequences](#lazy-sequences) -- `(std misc lazy-seq)` +- [Weak Collections](#weak-collections) -- `(std misc weak)` +- [Generic Collection Protocol](#generic-collection-protocol) -- `(std misc collection)` +- [Relational Data Operations](#relational-data-operations) -- `(std misc relation)` +- [LCS-Based Diff](#lcs-based-diff) -- `(std misc diff)` +- [Cycle-Aware Equality](#cycle-aware-equality) -- `(std misc equiv)` + +--- + +## Persistent Hash Maps + +**Module:** `(std misc persistent)` +**File:** `lib/std/misc/persistent.sls` + +```scheme +(import (std misc persistent)) +``` + +Immutable hash maps implemented as Hash Array Mapped Tries (HAMT) with 32-way branching and structural sharing. All operations return new HAMTs; the original is never mutated. Keys are compared with `equal?` and hashed with `equal-hash`. + +### API Reference + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `hamt-empty` | value | The empty HAMT. | +| `hamt?` | `(hamt? x)` | Returns `#t` if `x` is a HAMT. | +| `hamt-set` | `(hamt-set h key value)` | Returns a new HAMT with `key` mapped to `value`. | +| `hamt-ref` | `(hamt-ref h key default)` | Returns the value for `key`, or `default` if not found. | +| `hamt-delete` | `(hamt-delete h key)` | Returns a new HAMT without `key`. Returns `h` unchanged if `key` is absent. | +| `hamt-contains?` | `(hamt-contains? h key)` | Returns `#t` if `key` is present in the HAMT. | +| `hamt-size` | `(hamt-size h)` | Returns the number of key-value pairs. | +| `hamt-fold` | `(hamt-fold proc seed h)` | Folds `proc` over all entries. `proc` receives `(key value accumulator)`. | +| `hamt-keys` | `(hamt-keys h)` | Returns a list of all keys. | +| `hamt-values` | `(hamt-values h)` | Returns a list of all values. | +| `hamt-map` | `(hamt-map f h)` | Returns a new HAMT with `f` applied to each value. Keys are unchanged. | +| `hamt->alist` | `(hamt->alist h)` | Returns the HAMT contents as an association list of `(key . value)` pairs. | +| `alist->hamt` | `(alist->hamt alist)` | Creates a HAMT from an association list. | + +### Examples + +```scheme +(import (std misc persistent)) + +;; Build up a map incrementally +(define h0 hamt-empty) +(define h1 (hamt-set h0 "name" "Alice")) +(define h2 (hamt-set h1 "age" 30)) +(define h3 (hamt-set h2 "city" "Portland")) + +;; Lookup +(hamt-ref h3 "name" #f) ; => "Alice" +(hamt-ref h3 "missing" 'nope) ; => nope +(hamt-contains? h3 "age") ; => #t +(hamt-size h3) ; => 3 + +;; The original is unchanged (persistent/immutable) +(hamt-size h1) ; => 1 + +;; Delete a key +(define h4 (hamt-delete h3 "age")) +(hamt-size h4) ; => 2 +(hamt-contains? h4 "age") ; => #f + +;; Enumerate +(hamt-keys h3) ; => ("city" "age" "name") (order may vary) +(hamt-values h3) ; => ("Portland" 30 "Alice") (order may vary) +(hamt->alist h3) +; => (("city" . "Portland") ("age" . 30) ("name" . "Alice")) + +;; Map over values +(define ages (alist->hamt '(("Alice" . 30) ("Bob" . 25)))) +(define next-year (hamt-map add1 ages)) +(hamt-ref next-year "Alice" #f) ; => 31 + +;; Fold to compute a total +(hamt-fold (lambda (k v acc) (+ v acc)) 0 ages) ; => 55 + +;; Round-trip through alist +(define h5 (alist->hamt '((x . 1) (y . 2) (z . 3)))) +(hamt->alist h5) ; => ((z . 3) (y . 2) (x . 1)) (order may vary) +``` + +--- + +## Lazy Sequences + +**Module:** `(std misc lazy-seq)` +**File:** `lib/std/misc/lazy-seq.sls` + +```scheme +(import (std misc lazy-seq)) +``` + +Clojure-style lazy sequences built from memoized thunks. A lazy sequence is a thunk that, when called, produces either `(cons head tail)` where tail is another lazy sequence, or `'()` for the end. Results are cached after the first force. + +### API Reference + +| Procedure / Macro | Signature | Description | +|-------------------|-----------|-------------| +| `lazy-seq` | `(lazy-seq body ...)` | Macro. Wraps body in a memoized thunk. Body should return `(cons head tail)` or `'()`. | +| `lazy-cons` | `(lazy-cons head tail-expr)` | Macro. Creates a lazy pair. `head` is evaluated eagerly; `tail-expr` is delayed. | +| `lazy-null` | value | The empty lazy sequence. | +| `lazy-null?` | `(lazy-null? lseq)` | Returns `#t` if the lazy sequence is empty. Forces the thunk. | +| `lazy-car` | `(lazy-car lseq)` | Returns the first element. Raises an error if empty. | +| `lazy-cdr` | `(lazy-cdr lseq)` | Returns the tail (another lazy sequence). Raises an error if empty. | +| `lazy-seq->list` | `(lazy-seq->list lseq)` | Forces the entire sequence and returns a list. | +| `list->lazy-seq` | `(list->lazy-seq lst)` | Converts a proper list to a lazy sequence. | +| `lazy-take` | `(lazy-take n lseq)` | Returns a lazy sequence of at most `n` elements. | +| `lazy-drop` | `(lazy-drop n lseq)` | Returns a lazy sequence with the first `n` elements removed. | +| `lazy-map` | `(lazy-map f lseq)` | Lazily applies `f` to each element. | +| `lazy-filter` | `(lazy-filter pred lseq)` | Lazily keeps only elements satisfying `pred`. | +| `lazy-append` | `(lazy-append lseq1 lseq2)` | Lazily concatenates two sequences. | +| `lazy-range` | `(lazy-range)` | Infinite sequence 0, 1, 2, ... | +| | `(lazy-range end)` | Range `[0, end)` with step 1. | +| | `(lazy-range start end)` | Range `[start, end)` with step 1. | +| | `(lazy-range start end step)` | Range `[start, end)` with the given step. | +| `lazy-iterate` | `(lazy-iterate f seed)` | Infinite sequence: `seed`, `(f seed)`, `(f (f seed))`, ... | +| `lazy-zip` | `(lazy-zip lseq1 lseq2)` | Lazily pairs elements from two sequences into cons pairs. Stops at the shorter. | + +### Examples + +```scheme +(import (std misc lazy-seq)) + +;; Finite ranges +(lazy-seq->list (lazy-range 5)) ; => (0 1 2 3 4) +(lazy-seq->list (lazy-range 2 7)) ; => (2 3 4 5 6) +(lazy-seq->list (lazy-range 0 10 3)) ; => (0 3 6 9) + +;; Infinite sequences with take +(lazy-seq->list (lazy-take 5 (lazy-iterate add1 0))) +; => (0 1 2 3 4) + +;; Powers of 2 +(lazy-seq->list (lazy-take 8 (lazy-iterate (lambda (x) (* x 2)) 1))) +; => (1 2 4 8 16 32 64 128) + +;; Filter and map +(lazy-seq->list (lazy-filter odd? (lazy-range 0 10))) +; => (1 3 5 7 9) + +(lazy-seq->list (lazy-map (lambda (x) (* x x)) (lazy-range 1 6))) +; => (1 4 9 16 25) + +;; Zip two sequences +(lazy-seq->list (lazy-zip (lazy-range 0 3) (list->lazy-seq '(a b c)))) +; => ((0 . a) (1 . b) (2 . c)) + +;; Append +(lazy-seq->list (lazy-append (lazy-range 0 3) (lazy-range 10 13))) +; => (0 1 2 10 11 12) + +;; Drop +(lazy-seq->list (lazy-take 3 (lazy-drop 5 (lazy-range)))) +; => (5 6 7) + +;; Build from scratch with lazy-cons +(define fibs + (let fib ([a 0] [b 1]) + (lazy-cons a (fib b (+ a b))))) +(lazy-seq->list (lazy-take 10 fibs)) +; => (0 1 1 2 3 5 8 13 21 34) +``` + +--- + +## Weak Collections + +**Module:** `(std misc weak)` +**File:** `lib/std/misc/weak.sls` + +```scheme +(import (std misc weak)) +``` + +Weak references and collections built on Chez Scheme's GC primitives. Weak pairs hold their car weakly -- when the GC reclaims the referenced object, the car becomes `#!bwp` (broken weak pointer). Weak hash tables hold keys weakly with `eq?` comparison; entries are automatically removed when keys are collected. + +### API Reference + +#### Weak Pairs + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `make-weak-pair` | `(make-weak-pair key value)` | Creates a weak pair. The car (`key`) is held weakly; the cdr (`value`) is held strongly. | +| `weak-pair?` | `(weak-pair? obj)` | Returns `#t` if `obj` is a weak pair. | +| `weak-car` | `(weak-car wp)` | Returns the car of the weak pair. May be `#!bwp` if reclaimed. | +| `weak-cdr` | `(weak-cdr wp)` | Returns the cdr of the weak pair. | +| `weak-pair-value` | `(weak-pair-value wp)` | Returns the car if still live, or `#f` if reclaimed. | + +#### Weak Lists + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `list->weak-list` | `(list->weak-list lst)` | Converts a list into a chain of weak pairs. Each element is held weakly. | +| `weak-list->list` | `(weak-list->list wl)` | Collects all live (non-reclaimed) elements into a regular list. | +| `weak-list-compact!` | `(weak-list-compact! wl)` | Destructively removes reclaimed entries. Returns the (possibly new) head. | + +#### Weak Hash Tables + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `make-weak-hashtable` | `(make-weak-hashtable)` | Creates a weak eq-hashtable. Keys are held weakly. | +| | `(make-weak-hashtable size)` | Creates a weak eq-hashtable with initial size hint. | +| `weak-hashtable-ref` | `(weak-hashtable-ref ht key default)` | Looks up `key`, returning `default` if absent or reclaimed. | +| `weak-hashtable-set!` | `(weak-hashtable-set! ht key value)` | Associates `key` with `value`. | +| `weak-hashtable-delete!` | `(weak-hashtable-delete! ht key)` | Removes the entry for `key`. | +| `weak-hashtable-keys` | `(weak-hashtable-keys ht)` | Returns a list of all live keys (filters out `#!bwp` entries). | + +### Examples + +```scheme +(import (std misc weak)) + +;; Weak pairs +(define wp (make-weak-pair 'hello 42)) +(weak-pair? wp) ; => #t +(weak-car wp) ; => hello +(weak-cdr wp) ; => 42 +(weak-pair-value wp) ; => hello (still live) + +;; Weak lists +(define wl (list->weak-list '(a b c d))) +(weak-list->list wl) ; => (a b c d) (all still live) +;; After GC may reclaim unreferenced objects, weak-list->list +;; returns only the surviving elements. + +;; Compact removes dead entries in-place +(weak-list-compact! wl) ; => new head of compacted list + +;; Weak hash tables for caches +(define cache (make-weak-hashtable)) +(let ([key1 (list 'data 1)] + [key2 (list 'data 2)]) + (weak-hashtable-set! cache key1 "result-1") + (weak-hashtable-set! cache key2 "result-2") + (weak-hashtable-ref cache key1 #f) ; => "result-1" + (weak-hashtable-keys cache)) ; => ((data 2) (data 1)) + +(weak-hashtable-delete! cache (list 'data 1)) +;; Note: weak-hashtable uses eq? comparison, so the delete above +;; would not find the key unless it is the same object (eq?). +``` + +**Important:** Weak hash tables use `eq?` comparison for keys, not `equal?`. This means only the exact same object (by identity) will match on lookup or deletion. This is appropriate for caching computed results keyed on object identity. + +--- + +## Generic Collection Protocol + +**Module:** `(std misc collection)` +**File:** `lib/std/misc/collection.sls` + +```scheme +(import (std misc collection)) +``` + +A protocol for writing algorithms that work across different data structures. The core abstraction is an **iterator**: a thunk that returns `(values element #t)` for each element, then `(values #f #f)` when exhausted. Built-in support for lists, vectors, strings, bytevectors, and hashtables. New types can be registered with `define-collection`. + +### API Reference + +| Procedure / Macro | Signature | Description | +|-------------------|-----------|-------------| +| `make-iterator` | `(make-iterator coll)` | Returns an iterator thunk for `coll`. Dispatches based on registered type predicates. | +| `define-collection` | `(define-collection pred make-iter)` | Registers a new collection type. `pred` is a type predicate; `make-iter` takes a collection and returns an iterator thunk. | +| `collection-fold` | `(collection-fold proc seed coll)` | Folds `proc` over elements. `proc` receives `(element accumulator)`. | +| `collection-map` | `(collection-map proc coll)` | Applies `proc` to each element, returns a list of results. | +| `collection-filter` | `(collection-filter pred coll)` | Returns a list of elements satisfying `pred`. | +| `collection-for-each` | `(collection-for-each proc coll)` | Calls `proc` on each element for side effects. | +| `collection-find` | `(collection-find pred coll)` | Returns the first element satisfying `pred`, or `#f`. | +| `collection-any` | `(collection-any pred coll)` | Returns `#t` if any element satisfies `pred`. | +| `collection-every` | `(collection-every pred coll)` | Returns `#t` if all elements satisfy `pred`. | +| `collection->list` | `(collection->list coll)` | Converts any collection to a list. | +| `collection-length` | `(collection-length coll)` | Returns the number of elements. | + +#### Built-in Collection Types + +| Type | Iterator Behavior | +|------|-------------------| +| List | Iterates over elements in order. | +| Vector | Iterates over elements by index. | +| String | Iterates over characters. | +| Bytevector | Iterates over bytes as exact integers. | +| Hashtable | Iterates over `(key . value)` pairs. | + +### Examples + +```scheme +(import (std misc collection)) + +;; Works uniformly across types +(collection->list '(1 2 3)) ; => (1 2 3) +(collection->list '#(4 5 6)) ; => (4 5 6) +(collection->list "abc") ; => (#\a #\b #\c) +(collection->list #vu8(10 20 30)) ; => (10 20 30) + +;; Fold +(collection-fold + 0 '(1 2 3 4)) ; => 10 +(collection-fold + 0 '#(1 2 3 4)) ; => 10 + +;; Map and filter +(collection-map add1 '#(1 2 3)) ; => (2 3 4) +(collection-filter even? '(1 2 3 4 5 6)) ; => (2 4 6) + +;; Search +(collection-find (lambda (x) (> x 3)) '#(1 2 3 4 5)) ; => 4 +(collection-any negative? '(1 -2 3)) ; => #t +(collection-every positive? '(1 2 3)) ; => #t + +;; Length +(collection-length "hello") ; => 5 +(collection-length '#(a b c)) ; => 3 + +;; Iterate over hashtable entries +(let ([ht (make-hashtable string-hash string=?)]) + (hashtable-set! ht "x" 1) + (hashtable-set! ht "y" 2) + (collection-map cdr ht)) ; => (1 2) (order may vary) + +;; Register a custom collection type +(define-record-type range-obj (fields start end)) + +(define-collection range-obj? + (lambda (r) + (let ([i (range-obj-start r)] + [end (range-obj-end r)]) + (let ([current i]) + (lambda () + (if (< current end) + (let ([v current]) + (set! current (+ current 1)) + (values v #t)) + (values #f #f))))))) + +(collection->list (make-range-obj 0 5)) ; => (0 1 2 3 4) +(collection-fold + 0 (make-range-obj 1 4)) ; => 6 +``` + +--- + +## Relational Data Operations + +**Module:** `(std misc relation)` +**File:** `lib/std/misc/relation.sls` + +```scheme +(import (std misc relation)) +``` + +In-memory relational data operations on tabular data. A relation is a set of rows with named columns (symbols). Rows are stored internally as association lists. Supports select, project, join, group-by, sort, extend, and aggregate. + +### API Reference + +| Procedure | Signature | Description | +|-----------|-----------|-------------| +| `make-relation` | `(make-relation columns rows)` | Creates a relation. `columns` is a list of symbols. `rows` is a list of lists (positional) or alists. | +| `relation?` | `(relation? x)` | Returns `#t` if `x` is a relation. | +| `relation-columns` | `(relation-columns r)` | Returns the list of column names (symbols). | +| `relation-rows` | `(relation-rows r)` | Returns the rows as a list of alists. | +| `relation-count` | `(relation-count r)` | Returns the number of rows. | +| `relation-ref` | `(relation-ref row col)` | Gets a column value from a row alist. Raises an error if the column is not found. | +| `relation-select` | `(relation-select r pred)` | Filters rows. `pred` receives the row alist. | +| `relation-project` | `(relation-project r cols)` | Selects specific columns. `cols` is a list of symbols. | +| `relation-extend` | `(relation-extend r col-name proc)` | Adds a computed column. `proc` receives the row alist and returns the new column's value. | +| `relation-sort` | `(relation-sort r col comparator)` | Sorts rows by a column using `comparator`. | +| `relation-group-by` | `(relation-group-by r col)` | Groups rows by a column. Returns an alist of `(key-value . sub-relation)`. | +| `relation-join` | `(relation-join r1 r2 key-col)` | Inner join on a shared key column. | +| `relation-aggregate` | `(relation-aggregate r col proc init)` | Folds `proc` over a column's values. `proc` receives `(accumulator column-value)`. | +| `relation->alist-list` | `(relation->alist-list r)` | Returns rows as a list of alists. | +| `alist-list->relation` | `(alist-list->relation alist-list)` | Creates a relation from a list of alists. Column names are taken from the first row. | + +### Examples + +```scheme +(import (std misc relation)) + +;; Create a relation with positional rows +(define people + (make-relation '(name age city) + '(("Alice" 30 "Portland") + ("Bob" 25 "Seattle") + ("Carol" 35 "Portland") + ("Dave" 28 "Seattle")))) + +(relation-count people) ; => 4 + +;; Select (filter) rows +(define portlanders + (relation-select people + (lambda (row) (string=? (relation-ref row 'city) "Portland")))) +(relation-count portlanders) ; => 2 + +;; Project (pick columns) +(define names-only (relation-project people '(name))) +(relation->alist-list names-only) +; => (((name . "Alice")) ((name . "Bob")) ((name . "Carol")) ((name . "Dave"))) + +;; Extend with a computed column +(define with-senior + (relation-extend people 'senior? + (lambda (row) (>= (relation-ref row 'age) 30)))) +(relation-ref (car (relation-rows with-senior)) 'senior?) ; => #t + +;; Sort by age