Add docs/TODO.md, remove completed newer.md and stale better*.md
ober
572082d615263debace266adecf3f8c9a98f68e4
deleted file mode 100644 --- a/better.md +++ /dev/null @@ -1,451 +0,0 @@ -# Better: 30 Features for Gerbil→Jerboa Translation - -Features identified from analysis of 43 gerbil-* repos, Chez Scheme 10.4.0 capabilities, -and real-world translation gaps. Each feature includes implementation plan, test strategy, -and documentation requirements. - ---- - -## Translator Enhancements (1–10) - -### 1. `translate-method-dispatch` — Curly-Brace Method Syntax -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil's `{method obj args ...}` syntax to jerboa's `(~ obj method args ...)`. -Currently the translator has zero support for method dispatch syntax. - -**Impact:** Blocks all OOP-heavy ports (gerbil-litehtml 100+ uses, gerbil-origin 200+). - -### 2. `translate-defrules` — Macro Definition Translation -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil's `(defrules name () (pat body) ...)` to jerboa's `(defrules name (pat body) ...)`. -Gerbil's defrules has an extra `()` literals list that jerboa's doesn't need. - -**Impact:** 15+ projects with macro libraries. - -### 3. `translate-defstruct` Enhancement — Parent & Mutable Fields -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Current `translate-defstruct` drops parent information and field mutability. -Enhance to emit `(parent ...)` clause and `(mutable field)` annotations. - -**Impact:** 50+ structs across gerbil projects with inheritance or mutable fields. - -### 4. `translate-hash-literal` — Hash Table Literal Syntax -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil's hash table construction patterns: -- `(hash (key val) ...)` already works (jerboa core has it) -- Translate `(hash-eq (key val) ...)` patterns -- Translate `(list->hash-table alist)` calls (same API, just verify) - -**Impact:** 30+ files in gerbil-utils, gerbil-postgres. - -### 5. `translate-try-catch` — Exception Handling Normalization -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Normalize Gerbil's exception handling forms: -- `(with-catch handler thunk)` → `(with-exception-catcher handler thunk)` (jerboa core) -- Verify `(try ... (catch (e) ...) (finally ...))` passes through unchanged - -**Impact:** Every project with error handling. - -### 6. `translate-export` — Export Form Translation -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil export forms: -- `(export ident ...)` → R6RS `(export ident ...)` -- `(export (struct-out name))` → expanded field accessor exports -- `(export (rename-out (old new) ...))` → R6RS `(rename (old new) ...)` - -**Impact:** Every file that exports anything. - -### 7. `translate-for-loops` — Iterator Syntax Translation -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil's `:std/iter` forms to jerboa equivalents: -- `(for ((x (in-list lst))) body)` → `(for ((x (in-list lst))) body)` (same API) -- `(for/collect ...)` → same (jerboa has it) -- Verify pass-through since jerboa's `(std iter)` matches Gerbil's API - -**Impact:** 228 imports across gerbil projects. - -### 8. `translate-match-patterns` — Match Clause Normalization -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Normalize Gerbil match patterns to jerboa's match: -- `(? pred)` guard patterns → verify same syntax -- `(and pat ...)` / `(or pat ...)` → verify pass-through -- `(struct-name field ...)` patterns → verify compatibility -- `[a b c]` in match patterns → `(list a b c)` (binding context) - -**Impact:** 500+ match expressions across gerbil projects. - -### 9. `translate-spawn-forms` — Concurrency Syntax -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Verify/translate concurrency forms: -- `(spawn thunk)` → pass-through (jerboa core has spawn) -- `(spawn/name name thunk)` → pass-through (jerboa core has it) -- `(<- expr)` actor receive → verify pass-through - -**Impact:** 460 call sites across gerbil projects. - -### 10. `translate-package-to-library` — Full File Structure Translation -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Transform a complete Gerbil file structure to R6RS library: -- `(package: :foo/bar)` + `(export ...)` + body → `(library (foo bar) (export ...) (import ...) body)` -- Auto-detect imports from `(import ...)` forms -- Handle `(namespace ...)` directives (strip them) - -**Impact:** Every Gerbil source file needs this for a full port. - ---- - -## Missing Standard Library Modules (11–20) - -### 11. `(std misc pqueue)` — Priority Queue -**Status:** DONE -**File:** `lib/std/misc/pqueue.sls` - -Binary heap priority queue with: -- `make-pqueue`, `pqueue?`, `pqueue-empty?`, `pqueue-length` -- `pqueue-push!`, `pqueue-pop!`, `pqueue-peek` -- Optional custom comparator -- `pqueue->list`, `pqueue-for-each` - -**Impact:** Used in scheduling, graph algorithms, event-driven systems. - -### 12. `(std misc barrier)` — Thread Barrier -**Status:** DONE -**File:** `lib/std/misc/barrier.sls` - -Cyclic barrier for thread synchronization: -- `make-barrier`, `barrier?`, `barrier-wait!` -- `barrier-reset!`, `barrier-parties` -- Reusable (cyclic) — automatically resets after all parties arrive - -**Impact:** Parallel algorithm coordination. - -### 13. `(std misc timeout)` — Timeout Operations -**Status:** DONE -**File:** `lib/std/misc/timeout.sls` - -Timeout-wrapped operations using Chez's engine system: -- `with-timeout` — run thunk with time limit, return default on timeout -- `timeout?` — predicate for timeout sentinel -- `make-timeout` — timeout value constructor - -Leverages Chez Scheme's `make-engine` for preemptive time-slicing — a -capability Gambit doesn't have. Engines provide tick-based fuel that the -compiler integrates at safe points, giving precise timeout control without -spawning extra threads. - -**Impact:** Network operations, database queries, any blocking operation. - -### 14. `(std misc func)` — Functional Combinators -**Status:** DONE -**File:** `lib/std/misc/func.sls` - -Core functional utilities scattered across Gerbil projects: -- `compose`, `compose1` — function composition -- `identity` — identity function -- `constantly` — constant-returning function -- `flip` — swap first two arguments -- `curry`, `curryn` — partial application -- `memo-proc` — simple memoization wrapper -- `negate` — predicate negation -- `conjoin`, `disjoin` — predicate AND/OR - -**Impact:** Foundational utilities used across every functional codebase. - -### 15. `(std misc repr)` — Object Representation Protocol -**Status:** DONE -**File:** `lib/std/misc/repr.sls` - -Custom print representations for user-defined types: -- `defmethod {write-repr obj port}` pattern -- `repr` — convert any object to readable string -- `display-repr` — write repr to port -- Default representations for records, hash tables, closures - -**Impact:** Debugging, logging, REPL output for custom types. - -### 16. `(std event)` — First-Class Events -**Status:** DONE -**File:** `lib/std/event.sls` - -Gerbil's event system (NOT the existing event-emitter pub/sub): -- `sync`, `select` — synchronize on first-ready event -- `choice` — combine events -- `wrap`, `handle` — transform event results -- `timeout-evt` — event that fires after delay -- `channel-recv-evt`, `channel-send-evt` — channel events -- `always-evt`, `never-evt` — constant events - -Leverages Chez's condition variables and the existing channel infrastructure. - -**Impact:** Concurrent programming patterns from gerbil-origin, gerbil-persist. - -### 17. `(std stxutil)` — Syntax Utilities -**Status:** DONE -**File:** `lib/std/stxutil.sls` - -Macro-writing helpers: -- `stx-car`, `stx-cdr`, `stx-null?`, `stx-pair?` — syntax object accessors -- `stx-map`, `stx-for-each` — iterate over syntax lists -- `stx->datum`, `datum->stx` — conversion (aliases for syntax->datum etc.) -- `with-syntax*` — sequential with-syntax bindings -- `genident` — generate unique identifier - -**Impact:** Any project defining macros. - -### 18. `(std contract)` — Design by Contract -**Status:** DONE -**File:** `lib/std/contract.sls` - -Contracts for defensive programming: -- `define/contract` — define with pre/post conditions -- `->` — function contract (domain → range) -- `->*` — function contract with optional/keyword args -- `contract-violation?` — predicate -- `check-argument` — argument validation with clear errors - -**Impact:** API boundary validation, library quality. - -### 19. `(std misc rwlock)` — Read-Write Lock -**Status:** DONE -**File:** `lib/std/misc/rwlock.sls` - -Multiple-reader/single-writer lock: -- `make-rwlock`, `rwlock?` -- `rwlock-read-lock!`, `rwlock-read-unlock!` -- `rwlock-write-lock!`, `rwlock-write-unlock!` -- `with-read-lock`, `with-write-lock` — RAII-style macros - -Leverages Chez's efficient mutex and condition variable primitives. - -**Impact:** Concurrent data structure access patterns. - -### 20. `(std misc symbol)` — Symbol Utilities -**Status:** DONE -**File:** `lib/std/misc/symbol.sls` - -Symbol manipulation matching Gerbil patterns: -- `symbol-append` — concatenate symbols: `(symbol-append 'make- 'point)` → `make-point` -- `symbol->keyword`, `keyword->symbol` — interconversion -- `make-symbol` — alias for `symbol-append` -- `interned-symbol?` — check if symbol is interned (uses Chez's gensym detection) - -**Impact:** Code generation, macro writing, serialization. - ---- - -## Chez Scheme Power Features (21–27) - -### 21. `(std engine)` — Preemptive Evaluation Engines -**Status:** DONE -**File:** `lib/std/engine.sls` - -Expose Chez's unique engine system (time-sliced evaluation): -- `make-engine` — create an engine from a thunk -- `engine-run` — run engine for N ticks -- `engine-result` — get result if completed -- `engine-expired?` — check if ticks exhausted -- `engine-map` — transform engine result - -Chez's engine system is unique among Scheme implementations — Gambit has -nothing comparable. It provides cooperative preemption at compiler-inserted -safe points, enabling timeout, resource limiting, and sandboxing without -threads. - -**Impact:** Sandboxed evaluation, resource-limited computation, REPL timeouts. - -### 22. `(std fasl)` — Fast-Load Serialization -**Status:** DONE -**File:** `lib/std/fasl.sls` - -Expose Chez's binary serialization for high-performance data exchange: -- `fasl-write` — serialize any Scheme datum to binary -- `fasl-read` — deserialize from binary -- `fasl-file-write`, `fasl-file-read` — file-level operations -- Handles: pairs, vectors, records, bytevectors, bignums, symbols, etc. - -Much faster than JSON/S-expr serialization for large data structures. -Chez FASL handles cycles and shared structure correctly. - -**Impact:** Cache files, IPC, persistent data structures. - -### 23. `(std inspect)` — Runtime Inspection -**Status:** DONE -**File:** `lib/std/inspect.sls` - -Expose Chez's inspector API for debugging: -- `inspect-object` — get type, fields, and values of any object -- `inspect-procedure` — get source, arity, free variables of a closure -- `inspect-condition` — extract all fields from a condition -- `inspect-code` — disassemble a compiled procedure -- `object-counts` — count live objects by type (GC statistics) - -**Impact:** REPL inspection, debugging tools, memory profiling. - -### 24. `(std ephemeron)` — Ephemeron Tables -**Status:** DONE -**File:** `lib/std/ephemeron.sls` - -Expose Chez's ephemeron support (GC-aware weak references): -- `make-ephemeron-eq-hashtable` — hash table where entries are GC'd when key is unreachable -- `ephemeron-pair`, `ephemeron-pair?` — raw ephemeron pairs -- `make-weak-eq-hashtable` — weak-key hash table - -Ephemerons are stronger than weak references: an ephemeron's value is -only traced if its key is reachable through non-ephemeron paths. -Perfect for caches and observer patterns. - -**Impact:** Memory-safe caching, observer patterns, interning tables. - -### 25. `(std ftype)` — Foreign Type Definitions -**Status:** DONE -**File:** `lib/std/ftype.sls` - -Expose Chez's ftype system for structured FFI: -- `define-ftype` — define C-compatible struct/union types -- `ftype-ref`, `ftype-set!` — field access -- `make-ftype-pointer` — allocate foreign memory -- `ftype-pointer?` — type predicate -- `ftype-sizeof` — size of foreign type - -Chez's ftype system is far more expressive than Gambit's c-define-type, -supporting bit fields, unions, endianness control, and nested structs. - -**Impact:** FFI-heavy projects, system programming. - -### 26. `(std compress lz4)` — LZ4 Compression -**Status:** DONE -**File:** `lib/std/compress/lz4.sls` - -LZ4 compression using Chez's built-in support or FFI: -- `lz4-compress` — compress bytevector -- `lz4-decompress` — decompress bytevector -- `make-lz4-compress-port` — streaming compression -- `make-lz4-decompress-port` — streaming decompression - -Chez has built-in port compression support; expose it at the bytevector level. - -**Impact:** Data storage, network protocols, log compression. - -### 27. `(std profile)` — Profiling Utilities -**Status:** DONE -**File:** `lib/std/profile.sls` - -Wrap Chez's profiling infrastructure: -- `with-profile` — profile a thunk, return timing/allocation stats -- `profile-dump` — dump profile data as alist -- `time-it` — simple wall-clock timing with display -- `allocation-count` — count bytes allocated during a thunk - -Chez has `(time expr)` but no programmatic API. This wraps the internal -`statistics` and profiling counters. - -**Impact:** Performance optimization, benchmarking. - ---- - -## Quality of Life (28–30) - -### 28. `(std misc hash-more)` — Extended Hash Table Operations -**Status:** DONE -**File:** `lib/std/misc/hash-more.sls` - -Hash operations missing from jerboa's runtime but common in Gerbil code: -- `hash-filter` — filter entries by predicate -- `hash-map/values` — map over values only -- `hash-ref/default` — explicit default (vs hash-ref's error) -- `hash-value-set!` — alias for hash-put! (Gerbil naming) -- `hash->alist` — alias for hash->list with explicit key-value pairs -- `hash-union` — merge with conflict resolution function -- `hash-intersect` — intersection of two hash tables - -**Impact:** Data manipulation in every project. - -### 29. `(std misc string-more)` — Extended String Operations -**Status:** DONE -**File:** `lib/std/misc/string-more.sls` - -String operations from Gerbil's `:std/misc/string` not yet in jerboa: -- `string-prefix?`, `string-suffix?` — test prefix/suffix -- `string-contains?` — substring search predicate -- `string-trim-both` — trim both ends (alias for string-trim in some contexts) -- `string-join` — join list of strings with separator -- `string-repeat` — repeat string N times -- `string-index` — find first occurrence of char/pred -- `string-pad-left`, `string-pad-right` — padding - -**Impact:** String processing in every project. - -### 30. `(std misc list-more)` — Extended List Operations -**Status:** DONE -**File:** `lib/std/misc/list-more.sls` - -List operations from Gerbil that aren't in jerboa core or SRFI-1: -- `flatten` — deep flatten nested lists -- `group-by` — group list elements by key function -- `partition-by` — partition based on predicate (returns two lists) -- `zip-with` — zip with combining function -- `interleave` — interleave two lists -- `chunk` — split list into sublists of size N -- `unique` — remove duplicates (with optional equality) -- `frequencies` — count occurrences as hash table - -**Impact:** Data transformation pipelines in every project. - ---- - -## Implementation Tracking - -All 30 features implemented. 202 tests passing in `tests/test-better.ss`. - -| # | Feature | Status | Tests | Docs | Committed | -|---|---------|--------|-------|------|-----------| -| 1 | translate-method-dispatch | DONE | 5 | inline | YES | -| 2 | translate-defrules | DONE | 3 | inline | YES | -| 3 | translate-defstruct enhanced | DONE | 6 | inline | YES | -| 4 | translate-hash-literal | DONE (pass-through) | 1 | inline | YES | -| 5 | translate-try-catch | DONE | 2 | inline | YES | -| 6 | translate-export | DONE | 5 | inline | YES | -| 7 | translate-for-loops | DONE (pass-through) | 1 | inline | YES | -| 8 | translate-match-patterns | DONE (pass-through) | 1 | inline | YES | -| 9 | translate-spawn-forms | DONE (pass-through) | 1 | inline | YES | -| 10 | translate-package-to-library | DONE | 3 | inline | YES | -| 11 | pqueue | DONE | 13 | inline | YES | -| 12 | barrier | DONE | 4 | inline | YES | -| 13 | timeout | DONE | 5 | inline | YES | -| 14 | func | DONE | 16 | inline | YES | -| 15 | repr | PRE-EXISTING | — | — | YES | -| 16 | event | DONE | 5 | inline | YES | -| 17 | stxutil | DONE | 10 | inline | YES | -| 18 | contract | DONE | 7 | inline | YES | -| 19 | rwlock | PRE-EXISTING | — | — | YES | -| 20 | symbol utils | DONE | 6 | inline | YES | -| 21 | engine | DONE | 5 | inline | YES | -| 22 | fasl | DONE | 4 | inline | YES | -| 23 | inspect | DONE | 9 | inline | YES | -| 24 | ephemeron | DONE | 5 | inline | YES | -| 25 | ftype | DONE | 7 | inline | YES | -| 26 | lz4 | DONE | 2 | inline | YES | -| 27 | profile | DONE | 6 | inline | YES | -| 28 | hash-more | DONE | 21 | inline | YES | -| 29 | string-more | DONE | 19 | inline | YES | -| 30 | list-more | DONE | 19 | inline | YES | deleted file mode 100644 --- a/better2.md +++ /dev/null @@ -1,436 +0,0 @@ -# Better2: 30 More Features for Gerbil→Jerboa Translation - -Second round of features identified from analysis of 45 gerbil-* repos, Chez Scheme 10.4.0, -and real translation gaps discovered during jerboa-shell and jerboa-emacs porting. - ---- - -## Translator Enhancements (1–5) - -### 1. `translate-using` — Method Dispatch with `using` -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Translate Gerbil's `using` operator (735 usage sites across gerbil-* repos): -- `(using obj Type method)` → `(Type-method obj)` accessor call -- Critical for gerbil-origin, gerbil-litehtml, gerbil-persist - -**Impact:** 735 usage sites; blocks most OOP-heavy ports. - -### 2. `translate-define-values` — Multiple Value Binding -**Status:** DONE -**File:** `lib/std/sugar.sls` + `lib/jerboa/translator.sls` - -Add `define-values` macro (223 usage sites): -- `(define-values (a b c) (values 1 2 3))` -- Sugar form for binding multiple return values at top level - -**Impact:** 223 call sites across gerbil projects. - -### 3. `translate-hash-operations` — Hash API Normalization -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Normalize remaining Gerbil hash operations to jerboa equivalents: -- `(hash-ref ht key)` (2-arg, errors) → passes through (jerboa has it) -- `(hash-set! ht key val)` → `(hash-put! ht key val)` (rename) -- `(hash-delete! ht key)` → `(hash-remove! ht key)` (rename) -- `(hash-contains? ht key)` → `(hash-key? ht key)` (rename) - -**Impact:** 300+ sites using Gerbil hash naming. - -### 4. `translate-gerbil-void` — Variadic void Compatibility -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Gerbil's `void` is variadic (accepts any args, returns void). Chez's `void` takes 0 args. -`(with-catch void thunk)` crashes in Chez because the handler calls `(void error)`. -- `(void)` → passes through -- `(void expr ...)` → `(begin expr ... (void))` or `(lambda _ (void))` in handler context - -**Impact:** Every project using `(with-catch void ...)` pattern. - -### 5. `translate-import-paths` — Module Path Normalization -**Status:** DONE -**File:** `lib/jerboa/translator.sls` - -Normalize Gerbil import paths to R6RS library names: -- `:std/sugar` → `(std sugar)` -- `:std/misc/string` → `(std misc string)` -- `:std/text/json` → `(std text json)` -- Handle `(only-in ...)`, `(except-in ...)`, `(rename-in ...)` - -**Impact:** Every Gerbil file needs this. - ---- - -## Missing Stdlib Completions (6–15) - -### 6. `(std misc hash-more)` Completion — fold, find, clear, copy -**Status:** DONE -**File:** `lib/std/misc/hash-more.sls` - -Add missing hash operations (184 usage sites): -- `hash-fold` — fold over entries -- `hash-find` — find first matching entry -- `hash-clear!` — clear all entries -- `hash-copy` — shallow copy -- `hash-merge` — merge (already in gambit-compat, need in hash-more) -- `hash-keys`, `hash-values` — extract keys/values as lists - -**Impact:** 184 usage sites across gerbil projects. - -### 7. `(std iter)` Completion — in-port, in-lines, in-chars, in-bytes -**Status:** DONE -**File:** `lib/std/iter.sls` - -Add I/O iterators missing from iter.sls: -- `in-port` — iterate over datums from a port (using read) -- `in-lines` — iterate over lines from a port (using read-line) -- `in-chars` — iterate over characters from a port -- `in-bytes` — iterate over bytes from a binary port -- `in-producer` — iterate over results of a thunk until EOF - -**Impact:** Common pattern in file-processing code. - -### 8. `(std source)` — Source Location Tracking -**Status:** DONE -**File:** `lib/std/source.sls` - -Compile-time source location macros (10 import sites): -- `this-source-file` — expands to current file path string -- `this-source-directory` — expands to directory of current file -- `this-source-location` — expands to `(file line column)` list -- Leverages Chez's `source-condition` and annotation system - -**Impact:** Used in logging, error reporting, and build systems. - -### 9. `(std misc wg)` — Wait Groups -**Status:** DONE -**File:** `lib/std/misc/wg.sls` - -Go-style wait group for thread coordination: -- `make-wg` — create wait group -- `wg-add` — increment pending count -- `wg-done` — decrement (signal completion) -- `wg-wait` — block until count reaches 0 -- Complements barriers (fixed N) with dynamic count - -**Impact:** Common concurrency pattern in gerbil-origin, gerbil-persist. - -### 10. `(std text/char-set)` — Character Sets -**Status:** DONE -**File:** `lib/std/text/char-set.sls` - -Character set operations for text processing: -- `char-set`, `char-set?`, `char-set-contains?` -- `char-set:letter`, `char-set:digit`, `char-set:whitespace` -- `char-set-union`, `char-set-intersection`, `char-set-complement` -- `char-set->list`, `string->char-set` -- Used by parsers, validators, tokenizers - -**Impact:** Foundation for text processing modules. - -### 11. `(std os/temp)` — Temporary Files/Directories -**Status:** DONE -**File:** `lib/std/os/temp.sls` - -Temporary file management: -- `make-temporary-file` — create temp file, return path -- `make-temporary-directory` — create temp dir, return path -- `call-with-temporary-file` — auto-cleanup on exit -- `call-with-temporary-directory` — auto-cleanup on exit -- Uses Chez's foreign-procedure for mkstemp/mkdtemp - -**Impact:** Test suites, build systems, data processing pipelines. - -### 12. `(std os/file-info)` — File Metadata via stat -**Status:** DONE -**File:** `lib/std/os/file-info.sls` - -File metadata access: -- `file-info` — returns record with size, mtime, mode, uid, gid -- `file-size`, `file-mtime`, `file-mode` — individual accessors -- `file-type` — regular, directory, symlink, pipe, socket -- `file-executable?`, `file-readable?`, `file-writable?` -- Uses Chez's foreign-procedure for stat(2) - -**Impact:** 200+ lines of FFI in jerboa-shell compat; every project touching files. - -### 13. `(std os/pipe)` — Pipe Operations -**Status:** DONE -**File:** `lib/std/os/pipe.sls` - -Unix pipe operations: -- `open-pipe` — create pipe, return (input-port . output-port) -- `pipe->ports` — convert pipe fds to Scheme ports -- Uses Chez's foreign-procedure for pipe(2) - -**Impact:** Process pipelines, IPC between threads. - -### 14. `(std os/tty)` — Terminal Control -**Status:** DONE -**File:** `lib/std/os/tty.sls` - -Terminal detection and raw mode: -- `tty?` — is port a terminal? -- `tty-size` — (values rows cols) -- `tty-raw-mode!` — set terminal to raw mode -- `tty-cooked-mode!` — restore cooked mode -- `with-raw-mode` — RAII wrapper -- Uses Chez FFI for isatty, ioctl TIOCGWINSZ, tcsetattr - -**Impact:** jerboa-shell and jerboa-emacs both need this. - -### 15. `(std text/ini)` — INI File Parsing -**Status:** DONE -**File:** `lib/std/text/ini.sls` - -Simple INI/config file parser: -- `ini-read` — parse INI file to nested alist -- `ini-write` — write alist as INI file -- `ini-ref` — lookup section.key -- Handles sections, comments (#, ;), key=value pairs - -**Impact:** Config files in jerboa-shell, various utilities. - ---- - -## Chez Scheme Power Features (16–23) - -### 16. `(std guardian)` — GC Guardians for Resource Cleanup -**Status:** DONE -**File:** `lib/std/guardian.sls` - -Expose Chez's guardian system (GC-triggered cleanup): -- `make-guardian` — create a guardian -- `guardian-register!` — register object for finalization -- `guardian-drain!` — collect all finalized objects -- Pattern for auto-closing file handles, freeing foreign memory - -**Impact:** Memory-safe resource management without explicit close. - -### 17. `(std trace)` — Function Tracing & Debugging -**Status:** DONE -**File:** `lib/std/trace.sls` - -Expose Chez's tracing system: -- `trace-define` — define with automatic call tracing -- `trace-lambda` — lambda with tracing -- `trace-let` — let with tracing -- `untrace` — remove tracing -- `trace-output-port` — control trace output destination - -**Impact:** Interactive debugging without external tools. - -### 18. `(std compile)` — Compilation Utilities -**Status:** DONE -**File:** `lib/std/compile.sls` - -Expose Chez's compilation infrastructure: -- `compile-file` — compile .sls to .so -- `compile-whole-program` — whole-program optimization -- `compile-to-port` — compile to binary port -- `optimize-level` — get/set optimization level (0-3) -- `generate-wpo-files` — enable whole-program optimization files - -**Impact:** Build systems, deployment, performance optimization. - -### 19. `(std symbol-property)` — Symbol Property Lists -**Status:** DONE -**File:** `lib/std/symbol-property.sls` - -Expose Chez's symbol property system: -- `putprop` — attach property to symbol -- `getprop` — retrieve property from symbol -- `remprop` — remove property -- `property-list` — get all properties of a symbol -- Unique to Chez: per-symbol key-value store without external hash table - -**Impact:** Code generation, macro metadata, DSL implementation. - -### 20. `(std fixnum)` — Extended Fixnum Operations -**Status:** DONE -**File:** `lib/std/fixnum.sls` - -Re-export Chez's fixnum-specific operations: -- `fx+`, `fx-`, `fx*`, `fxdiv`, `fxmod` — fixnum arithmetic -- `fxlogand`, `fxlogor`, `fxlogxor`, `fxlognot` — bitwise -- `fxsll`, `fxsrl`, `fxsra` — shifts -- `fx=`, `fx<`, `fx>`, `fx<=`, `fx>=` — comparisons -- `fixnum-width`, `greatest-fixnum`, `least-fixnum` - -**Impact:** Performance-critical inner loops, protocol parsing. - -### 21. `(std port-position)` — Port Position Tracking -**Status:** DONE -**File:** `lib/std/port-position.sls` - -Expose Chez's port position API: -- `port-position` — current position in port -- `set-port-position!` — seek to position -- `port-has-port-position?` — can this port report position? -- `port-has-set-port-position!?` — can this port seek? -- `port-length` — total length (for file ports) - -**Impact:** Binary protocol parsing, file format readers, seekable I/O. - -### 22. `(std record-meta)` — Advanced Record Features -**Status:** DONE -**File:** `lib/std/record-meta.sls` - -Expose Chez's advanced record type features: -- `record-type-descriptor` — get RTD from instance -- `record-constructor-descriptor` — get RCD -- `record-type-name`, `record-type-parent` — introspection -- `record-type-field-names` — list fields -- `nongenerative`, `sealed`, `opaque` — record type options -- `record-rtd` — RTD from instance (for dispatching) - -**Impact:** Serialization, debugging, generic programming. - -### 23. `(std cafe)` — REPL Customization -**Status:** DONE -**File:** `lib/std/cafe.sls` - -Expose Chez's REPL (cafe) customization: -- `waiter-prompt-string` — customize REPL prompt -- `waiter-prompt-and-read` — custom read hook -- `new-cafe` — launch nested REPL -- `cafe-eval` — evaluate in cafe context -- `reset-handler` — custom reset behavior - -**Impact:** Development tooling, embedded REPLs. - ---- - -## Quality of Life (24–30) - -### 24. `(std misc string-more)` Completion — split, replace, filter -**Status:** DONE -**File:** `lib/std/misc/string-more.sls` - -Add missing string operations: -- `string-split` — split string by delimiter (117 usage sites!) -- `string-replace` — replace substring occurrences -- `string-filter` — filter characters by predicate -- `string-upcase`, `string-downcase` — case conversion -- `string-reverse` — reverse a string - -**Impact:** 312 usage sites across gerbil projects. - -### 25. `(std misc vector-more)` — Extended Vector Operations -**Status:** DONE -**File:** `lib/std/misc/vector-more.sls` - -Vector operations matching Gerbil patterns: -- `vector-map` — already in Chez but not R6RS -- `vector-for-each` — iterate with index -- `vector-filter` — filter elements -- `vector-fold` — fold over vector -- `vector-append` — concatenate vectors -- `vector-copy` — with optional start/end - -**Impact:** Data processing with vectors instead of lists. - -### 26. `(std misc alist-more)` — Extended Alist Operations -**Status:** DONE -**File:** `lib/std/misc/alist-more.sls` - -Alist operations beyond what's in misc/alist.sls: -- `alist-ref/default` — lookup with default -- `alist-update` — functional update -- `alist-merge` — merge two alists -- `alist-filter` — filter entries -- `alist->hash` — convert to hash table -- `hash->alist` — already in hash-more, add reverse - -**Impact:** Config handling, lightweight key-value stores. - -### 27. `(std misc port-utils)` — Port Convenience Functions -**Status:** DONE -**File:** `lib/std/misc/port-utils.sls` - -Port utilities matching Gambit/Gerbil patterns: -- `read-all-as-string` — read entire port to string -- `read-all-as-bytes` — read entire port to bytevector -- `call-with-input-string` — open string port, call proc, close -- `call-with-output-string` — open string port, call proc, extract -- `with-output-to-string` — capture output to string -- `with-input-from-string` — read from string - -**Impact:** 270 usage sites for port I/O patterns. - -### 28. `(std misc numeric)` — Numeric Utilities -**Status:** DONE -**File:** `lib/std/misc/numeric.sls` - -Numeric utilities from Gerbil: -- `clamp` — clamp value to range -- `lerp` — linear interpolation -- `in-range?` — range check (different from in-range iterator) -- `integer->bytevector`, `bytevector->integer` — for protocol parsing -- `number->padded-string` — zero-padded number formatting - -**Impact:** Protocol implementations, data formatting. - -### 29. `(std debug/pp)` — Pretty Printer -**Status:** DONE -**File:** `lib/std/debug/pp.sls` - -Expose Chez's pretty printer with Gerbil-compatible API: -- `pp` — pretty-print to current output -- `pp-to-string` — pretty-print to string -- `pretty-print-columns` — control line width -- `pprint` — alias for pretty-print (Gerbil naming) - -**Impact:** Debugging, REPL output, code generation. - -### 30. `(std misc/with-destroy)` — Resource Management Macro -**Status:** DONE -**File:** `lib/std/misc/with-destroy.sls` - -RAII-style resource management (Gerbil pattern): -- `with-destroy` — ensure cleanup on exit (normal or exception) -- `defstruct` with `:destroy` method support -- Pattern: `(with-destroy (obj (make-resource)) body ...)` -- Calls `(destroy obj)` on scope exit - -**Impact:** File handles, network connections, FFI resources. - ---- - -## Implementation Tracking - -| # | Feature | Status | Tests | Docs | Committed | -|---|---------|--------|-------|------|-----------| -| 1 | translate-using | DONE | ✓ | ✓ | ✓ | -| 2 | define-values | DONE | ✓ | ✓ | ✓ | -| 3 | translate-hash-operations | DONE | ✓ | ✓ | ✓ | -| 4 | translate-gerbil-void | DONE | ✓ | ✓ | ✓ | -| 5 | translate-import-paths | DONE | ✓ | ✓ | ✓ | -| 6 | hash-more completion | DONE | ✓ | ✓ | ✓ | -| 7 | iter completion | DONE | ✓ | ✓ | ✓ | -| 8 | source location | DONE | ✓ | ✓ | ✓ | -| 9 | wait groups | DONE | ✓ | ✓ | ✓ | -| 10 | char-set | DONE | ✓ | ✓ | ✓ | -| 11 | temp files | DONE | ✓ | ✓ | ✓ | -| 12 | file-info | DONE | ✓ | ✓ | ✓ | -| 13 | pipe | DONE | ✓ | ✓ | ✓ | -| 14 | tty | DONE | ✓ | ✓ | ✓ | -| 15 | ini parser | DONE | ✓ | ✓ | ✓ | -| 16 | guardian | DONE | ✓ | ✓ | ✓ | -| 17 | trace | DONE | ✓ | ✓ | ✓ | -| 18 | compile | DONE | ✓ | ✓ | ✓ | -| 19 | symbol-property | DONE | ✓ | ✓ | ✓ | -| 20 | fixnum | DONE | ✓ | ✓ | ✓ | -| 21 | port-position | DONE | ✓ | ✓ | ✓ | -| 22 | record-meta | DONE | ✓ | ✓ | ✓ | -| 23 | cafe | DONE | ✓ | ✓ | ✓ | -| 24 | string-more completion | DONE | ✓ | ✓ | ✓ | -| 25 | vector-more | DONE | ✓ | ✓ | ✓ | -| 26 | alist-more | DONE | ✓ | ✓ | ✓ | -| 27 | port-utils | DONE | ✓ | ✓ | ✓ | -| 28 | numeric utils | DONE | ✓ | ✓ | ✓ | -| 29 | pretty printer | DONE | ✓ | ✓ | ✓ | -| 30 | with-destroy | DONE | ✓ | ✓ | ✓ | deleted file mode 100644 --- a/better3.md +++ /dev/null @@ -1,972 +0,0 @@ -# Better3: 30 World-Shattering Language Features - -Ambitious features inspired by Rust, Haskell, Elixir, Zig, Swift, Clojure, OCaml, Unison, -and research PLs — all exploiting Chez Scheme's unique capabilities (engines, continuations, -cp0 optimizer, guardians, ftypes, nanopass compiler). - -Jerboa already has: algebraic effects, gradual types, STM, actors, capabilities, lazy seqs, -pattern matching v2, transducers, delimited continuations, coroutines. These 30 features -build on that foundation to create something no other Scheme — or most languages — offer. - ---- - -## I. Ownership & Safety (1–5) - -### 1. `(std region)` — Region-Based Memory with Compile-Time Lifetimes -**Inspiration:** Rust lifetimes, Cyclone regions, Linear Haskell - -Chez has guardians and ftypes for C memory. Combine with jerboa's linear types -(`std/typed/linear.sls`) to create region-scoped allocations that are *provably* freed: - -```scheme -(with-region r - (let ([buf (region-alloc r 4096)]) ;; allocate in region r - (region-ref buf 0) ;; read — valid inside region - buf)) ;; ERROR: buf escapes region r -;; ALL memory in r freed here — no GC pressure, no leaks -``` - -**Why this is world-shattering:** No Scheme has region-based memory. Chez's ftype system -provides the raw allocation; linear types prevent escape. This gives Rust-like memory -safety *within a dynamic language* — zero-cost for FFI-heavy code (litehtml, Qt, crypto). - -**Chez leverage:** `ftype-pointer`, `foreign-alloc`/`foreign-free`, guardian fallback, -`define-ftype` for typed regions. - ---- - -### 2. `(std borrow)` — Borrow Checker for Mutable State -**Inspiration:** Rust borrow checker, Clean uniqueness types - -Build on linear types to enforce single-writer/multiple-reader discipline at the -*macro expansion* level: - -```scheme -(define-linear buf (make-bytevector 1024)) -(borrow buf reader ;; immutable borrow - (bytevector-u8-ref reader 0)) ;; OK: read access -(borrow-mut buf writer ;; mutable borrow - (bytevector-u8-set! writer 0 42)) ;; OK: exclusive write -;; buf still owned here -(consume buf) ;; linear resource consumed -``` - -**Why:** Eliminates data races at compile time for shared mutable state — something -even Clojure can't do (it uses STM at runtime). This is a *static* guarantee. - -**Chez leverage:** `syntax-case` for compile-time tracking, continuation marks for -borrow stack, cp0 for dead-borrow elimination. - ---- - -### 3. `(std move)` — Move Semantics for Zero-Copy Pipelines -**Inspiration:** Rust move semantics, Zig's comptime - -When data flows through a pipeline, copies are the enemy. Move semantics transfer -ownership without copying: - -```scheme -(define-move (process-request req) - (let ([body (move! (request-body req))]) ;; req.body invalidated - (let ([parsed (json-parse (move! body))]) ;; body invalidated - parsed))) ;; only parsed survives — zero copies -``` - -**Why:** Critical for jerboa-shell pipelines (zero-copy between stages), network -servers (request body → parser → handler), and FFI (C buffer ownership transfer). - -**Chez leverage:** Continuation marks track ownership, cp0 eliminates dead references, -guardian catches use-after-move at runtime as safety net. - ---- - -### 4. `(std phantom)` — Phantom Types for Type-Level State Machines -**Inspiration:** Haskell phantom types, Rust typestate pattern, OCaml GADTs - -Encode protocol states in the type system so invalid transitions are compile-time errors: - -```scheme -(define-phantom-states connection - [disconnected connected authenticated]) - -(define/phantom (connect host) : (Connection disconnected) -> (Connection connected) - (tcp-connect host 443)) - -(define/phantom (login conn creds) : (Connection connected) -> (Connection authenticated) - (send-auth conn creds)) - -(define/phantom (query conn sql) : (Connection authenticated) -> Result - (send-query conn sql)) - -;; (query (connect "db") "SELECT 1") ;; TYPE ERROR: connected ≠ authenticated -``` - -**Why:** Prevents impossible state transitions at compile time. Database connections -that query before login, files that write after close, TLS that sends before handshake — -all caught statically. No other Scheme has this.