updates
ober
2923ffe6d0ce5dc298cc4c2a180de5ccc4ea2933
--- a/README.md +++ b/README.md @@ -73,7 +73,7 @@ exports the everyday language; specialized libraries are imported from | Clojure compatibility | Sequences, reducers, transducers, atoms, agents, refs/STM, protocols, multimethods, metadata, futures/promises/deref, EDN, walkers, zippers, nested data helpers, datafy, core.async-style CSP channels. | | Concurrency | Native OS threads with no GIL, atomics, mutex/condition wrappers, M:N fibers, fiber-aware events/channels, async/await, structured concurrency, work pools, resource pools, barriers, wait groups, actor systems, supervisors, distributed actors, CRDTs, Raft, STM, CSP. | | Networking and web | HTTP client/server, secure rustls HTTPS daemon, fiber HTTP server, WebSocket/fiber WebSocket, HTTP/2, DNS, routers, rate limiting, connection pools, sendfile/zero-copy paths, TCP/UDP/TLS/rustls, SSH, S3, SMTP, SOCKS5, gRPC, JSON-RPC, 9P, FastCGI, Rack-style web adapters, event streams. | -| Text and protocols | JSON and JSON Schema, CSV, YAML, XML, HTML/SXML, TOML, INI, EDN, MessagePack, CBOR, Transit, protobuf, base58/base64/hex, UTF-8/16/32, globbing, diffs, templates, regex, native regex, PCRE2, `rx`, PEG parsers. | +| Text and protocols | JSON and JSON Schema, CSV, YAML, XML, HTML/SXML, TOML, INI, EDN, safe s-expression wire data, MessagePack, CBOR, Transit, protobuf, base58/base64/hex, UTF-8/16/32, globbing, diffs, templates, regex, native regex, PCRE2, `rx`, PEG parsers. | | Storage and databases | SQLite, PostgreSQL, DuckDB, LevelDB, DBI layer, query compilation, connection pools, mmap and mmap-btree helpers, content-addressed storage, image/closure persistence, package stores. | | Native Rust backend | Optional `libjerboa_native` bindings for ring crypto, rustls TLS, Rust HTTP request parsing, ReDoS-resistant regex, flate2 compression, rusqlite/postgres/duckdb, secure memory, epoll, inotify, Landlock, packet capture, and panic-contained FFI entry points. | | Security and capabilities | Audit logging, auth, cages, capability and capability-typed I/O, import audit, flow/taint tracking, IO interception, Landlock, seccomp, Capsicum, seatbelt, sandboxing, sanitizers, secret handling, privilege separation, safe/pure audit tools. | --- a/data/anti-patterns.sexp +++ b/data/anti-patterns.sexp @@ -3577,4 +3577,547 @@ "jerboa_security_scan" "live TLS lifecycle test" "ASan" - "fd/RSS soak"))) + "fd/RSS soak")) + (("advice" + . + "For static musl builds, compare foreign-procedure runtime failures against the generated Sforeign_symbol lists, confirm libc declarations with the target cross compiler, add related symbol families consistently to Linux generators, and rerun the container smoke target. Add required headers such as <sys/statvfs.h> when registering libc functions that need them.") + ("avoid" + . + "Fixing a single missing static libc symbol and rerunning without auditing adjacent POSIX/FFI symbols.") + ("id" . "static-musl-ffi-missing-symbol-drip") + ("kinds" "ffi" "debug-error") + ("pattern" + . + "Exception in foreign-procedure: no entry for \"(openat|fchmod|mkdirat|unlinkat|fstatvfs|shutdown|[^\"]+)\"") + ("severity" . "high") + ("tags" "static-build" "musl" "foreign-procedure" + "Sforeign_symbol" "libc" "jsh") + ("title" + . + "Do not chase static musl foreign-procedure symbols one at a time") + ("tools" + "jerboa_howto" + "jerboa_verify" + "jerboa_security_scan")) + (("advice" + . + "Route capture work through a bounded background queue owned by one worker thread. The UI thread should use non-blocking enqueue (`chan-try-put!` or equivalent), update visible state immediately, and tolerate dropped capture jobs if the queue is full. Confirm with trace/sample that the UI thread returns to event polling after send completion.") + ("avoid" + . + "Do not call encrypted logging, database writes, filesystem persistence, or capture normalization directly from a TUI event loop or send-completion handler. A send may complete quickly but the UI can still freeze in post-send capture, especially when pure Scheme database code spins or allocates heavily.") + ("id" . "tui-sync-capture-event-loop-freeze") + ("kinds" "module" "debug-error") + ("pattern" + . + "capture-(notification|outbound)!.*event-loop|logdb-put.*TUI event loop|handle-.*event.*capture-.*!") + ("severity" . "high") + ("tags" "tui" "event-loop" "logdb" "capture" "freeze" + "worker") + ("title" + . + "Do Not Write Logs Synchronously On TUI Event Loop") + ("tools" "sample or dtruss/strace" "tail trace" + "jerboa_howto tui-worker-mailbox-nonblocking-drain" + "make test" "git diff --check")) + (("advice" + . + "Use `fork-thread` or a local helper that explicitly starts an OS thread for blocking RPC/process/file work. Return results to the UI via a channel that the event loop drains non-blockingly. Add a regression with a deliberately slow worker proving the dispatch function returns before the slow call completes.") + ("avoid" + . + "Do not assume `(go ...)` always creates an OS thread suitable for blocking process I/O. In a fiber runtime it may spawn a cooperative fiber; a blocking RPC wait or process read can starve the TUI/event loop.") + ("id" . "csp-go-for-blocking-tui-io") + ("kinds" "module" "debug-error") + ("pattern" + . + "\\(go\\s+\\(lambda\\s*\\(\\).*actor-call|\\(go\\s+.*read-bounded-line|\\(go\\s+.*open-process") + ("severity" . "medium") + ("tags" "csp" "go" "fork-thread" "tui" "blocking-io" "rpc") + ("title" . "Do Not Use CSP go For Blocking TUI I/O") + ("tools" + "jerboa_howto tui-worker-mailbox-nonblocking-drain" + "rg \"(go\"" + "make test" + "sample")) + (("advice" + . + "Check the entry script's actual imports and either add a verified narrow import or implement the small traversal with Chez primitives. Run the WPO/binary build because lightweight verification may report no compile issues while whole-program compilation still finds the unbound identifier.") + ("avoid" + . + "Do not use a helper merely because it exists in the Jerboa prelude when the current entry script imports `(chezscheme)` and does not import that helper.") + ("id" . "chez-entry-assumes-prelude-helper") + ("kinds" "script" "debug-error") + ("pattern" + . + "attempt to reference unbound identifier (filter-map|[^ ]+)") + ("severity" . "medium") + ("tags" "chezscheme" "prelude" "unbound" "wpo" + "entry-script" "imports") + ("title" + . + "Do Not Assume Jerboa Prelude Helpers In Chez Entry Scripts") + ("tools" + "jerboa_module_exports" + "jerboa_function_signature" + "jerboa_verify" + "jerboa_make")) + (("advice" + . + "Make the build regenerate packaged seed assets from the authoritative source directory, then run a verifier that opens the final package and compares bundled entries against the source-of-truth. Fail the build on missing entries, extra stale entries, or contamination from unrelated generated directories.") + ("avoid" + . + "Do not assume a rebuilt APK or binary includes current generated data when the build copies a pre-existing packaged asset. Rebuilding code can still ship stale truth, seed data, model files, or fixture bundles.") + ("id" . "stale-packaged-seed-asset-without-source-verifier") + ("kinds" "all" "test" "debug-error") + ("pattern" + . + "seed_.*\\.zip|asset-dir|copy.*assets|APK seed|bundled truth") + ("severity" . "high") + ("tags" "android" "apk" "seed-data" "generated-assets" + "source-of-truth" "verification") + ("title" + . + "Do Not Ship Stale Packaged Seed Assets Without Source Verification") + ("tools" "rg" "zipinfo/unzip -l" "sha256sum" + "custom package verifier" "make target dependency")) + (("advice" + . + "Distinguish complete reviewed truth from area-only or partial cache metadata. Use area-only records only as detector boundaries, force fresh detection within the boundary, and reject/prune operations that would remove every group unless the user explicitly confirms destructive deletion.") + ("avoid" + . + "Do not let metadata-only truth or cache records with zero groups suppress fresh detection. This creates a blank-result trap where a saved boundary or partial cache is replayed as authoritative data.") + ("id" . "area-only-cache-treated-as-complete-detection") + ("kinds" "debug-error" "test" "module") + ("pattern" + . + "groups\\s*[:=]\\s*0|optJSONArray\\(\"groups\"\\).*length\\(\\).*0|ssd_area.*groups") + ("severity" . "high") + ("tags" "cache" "detector" "truth-data" "zero-results" + "android" "ssd-area") + ("title" + . + "Do Not Treat Area-Only Cache Records As Complete Detection Results") + ("tools" + "focused regression fixture" + "detector count telemetry" + "package/runtime logs" + "truth replay tests")) + (("advice" + . + "Emit normal-result telemetry with source id, input dimensions, detector backend, item counts, cache/truth source, and suppression/prune counts. Treat suspicious successful zero counts as diagnostic events or test failures for pipelines expected to find objects.") + ("avoid" + . + "Do not instrument detector, recognizer, parser, or sync pipelines only with crash/error logs. A zero-result success path can be the actual failure and will be invisible if only exceptions are reported.") + ("id" . "error-only-telemetry-hides-zero-result-pipelines") + ("kinds" "debug-error" "test" "all") + ("pattern" + . + "log.*error|status-error|catch.*log|Detect failed") + ("severity" . "medium") + ("tags" "telemetry" "detector" "parser" "zero-results" + "diagnostics" "android") + ("title" + . + "Do Not Log Only Errors For Detector Or Parser Pipelines") + ("tools" + "client log receiver" + "count telemetry" + "focused zero-result regression" + "health endpoint")) + (("advice" + . + "Preserve the current artifact, expose the literal failing evidence prominently, make one bounded edit with old_str/new_str, line_edit, replace_def, or replace_range, and automatically run the same verifier immediately afterward.") + ("avoid" + . + "Do not replace a verifier-visible Jerboa file wholesale when the verifier names one literal input, message, line, or localized behavior. A rewrite can destroy already passing behavior and consume the local model's output budget.") + ("id" . "full-rewrite-after-focused-verify-failure") + ("kinds" "debug-error" "script" "test") + ("pattern" + . + "full[- ]file rewrite.*(?:FAIL|verification failed)|(?:FAIL|verification failed).*full[- ]file rewrite") + ("severity" . "high") + ("tags" "local-model" "verified-repair" "full-rewrite" + "exact-edit" "jerboa") + ("title" . "Full rewrite after a focused verifier failure") + ("tools" "verify" "read" "edit" "line_edit" "replace_def" + "replace_range")) + (("advice" + . + "Put every field inside one field-list form: `(defstruct game (board score))`. When the compiler reports invalid syntax at defstruct, repair that declaration directly and rerun the verifier.") + ("avoid" + . + "Do not write Jerboa records as `(defstruct game board score)` or put each field directly after the struct name.") + ("id" . "defstruct-fields-outside-list") + ("kinds" "script" "module" "debug-error") + ("pattern" . "\\(defstruct\\s+[^\\s()]+\\s+[^()]") + ("severity" . "high") + ("tags" "jerboa" "defstruct" "record" "invalid-syntax" + "fields") + ("title" + . + "Jerboa defstruct fields must be one parenthesized list") + ("tools" "jerboa_howto" "jerboa_check_syntax" + "jerboa_error_fix_lookup" "verify" "replace_range")) + (("advice" + . + "Inspect the generated/enclosing for-each form, verify it has exactly a callback and iterable, and add a runtime assertion on the enclosing function's return type or length. Run jerboa_check_balance plus the focused runtime test or benchmark.") + ("avoid" + . + "Do not trust delimiter balance or compilation alone when a for-each callback may accidentally absorb the intended iterable expression; the next expression can become the iterable and for-each then returns void.") + ("id" . "for-each-iterable-inside-callback") + ("kinds" "module" "debug-error" "test") + ("pattern" + . + "\\(for-each\\s+\\(lambda[\\s\\S]*<iterable>\\s*\\)\\s*\\(<next-expression>\\)") + ("severity" . "medium") + ("tags" "for-each" "lambda" "parentheses" "void" "balanced" + "regression") + ("title" + . + "For-each Iterable Accidentally Captured In Callback") + ("tools" + "jerboa_check_balance" + "jerboa_verify" + "jerboa_eval" + "jerboa_run_tests")) + (("advice" + . + "Use domain-qualified local names such as document-length or byte-count when the module also calls common procedures. When an error says a number or string was applied, inspect lexical bindings before changing the call site, then add a focused runtime test because syntax and compilation will both accept the shadowing.") + ("avoid" + . + "Do not bind locals named length, map, filter, sort, string, or other imported procedures and then call the shadowed procedure within that lexical scope. The code compiles but attempts to apply the local data value at runtime.") + ("id" . "local-data-shadows-procedure") + ("kinds" "module" "script" "debug-error" "test") + ("pattern" + . + "(?:let|let\\*)[^\n]*(?:length|map|filter|sort)[^\n]*attempt to apply non-procedure") + ("severity" . "medium") + ("tags" "shadowing" "lexical-binding" "non-procedure" + "length" "runtime" "jerboa") + ("title" + . + "Local data binding shadows an imported procedure") + ("tools" + "jerboa_error_fix_lookup" + "jerboa_verify" + "jerboa_eval" + "focused runtime test")) + (("advice" + . + "Treat the manifest rename as the logical commit point. Return and install the new chain after publication; perform deletion of only old validated-manifest files as guarded best-effort cleanup. Test a cleanup failure or partial deletion and verify restart follows the new manifest.") + ("avoid" + . + "Do not treat obsolete-file deletion failure after atomic manifest publication as failure of the publication itself. Returning an error can leave memory on the old chain even though disk readers follow the new manifest, and a retry may republish references to already deleted files.") + ("id" . "post-publish-cleanup-rolls-back-logical-state") + ("kinds" "module" "debug-error" "security") + ("pattern" + . + "(?:write|rename).*manifest[\\s\\S]*(?:delete|cleanup)[\\s\\S]*(?:raise|old.chain|rollback)") + ("severity" . "high") + ("tags" "manifest" "atomic-publish" "compaction" "cleanup" + "crash-consistency" "segments") + ("title" + . + "Cleanup failure after manifest publication rolls logical state backward") + ("tools" + "jerboa_howto" + "jerboa_security_scan" + "jerboa_verify" + "fault-injection test")) + (("advice" + . + "Prepare immutable output without the publication lock, then acquire the owner lock and compare the current manifest generation with the worker target. Publish only on equality. Otherwise delete the prepared file, clear the running flag, and atomically reschedule against the newer chain when compaction is still required. Add a race test that checkpoints a sentinel generation while compaction runs and verifies it survives restart.") + ("avoid" + . + "Do not let a background compactor publish solely because it finished successfully. Checkpoints may have advanced the manifest while it was reading or serializing, so blind publication can roll durable state backward. Do not merely discard stale output without rescheduling when the current chain still exceeds the threshold.") + ("id" + . + "stale-background-compactor-publishes-old-generation") + ("kinds" "module" "debug-error" "security") + ("pattern" + . + "background.*compact[\\s\\S]*(?:write|rename).*manifest(?![\\s\\S]*(?:generation|epoch|version).*(?:check|compare))") + ("severity" . "high") + ("tags" "compaction" "generation" "manifest" + "background-worker" "race" "reschedule") + ("title" + . + "Stale background compactor publishes an older generation") + ("tools" + "jerboa_verify" + "jerboa_security_scan" + "concurrency race test" + "restart test")) + (("advice" + . + "Validate the sorted document table once when opening the mapping and cache only compact hot metadata such as document ID to length. Keep the much larger term bytes and posting lists mapped. Benchmark before and after with identical queries and report the resident-metadata tradeoff.") + ("avoid" + . + "Do not binary-search a mapped document table separately for every posting scored. The logarithmic lookup plus repeated foreign reads can make a memory-mapped query several times slower than the in-memory baseline even when postings themselves are zero-copy.") + ("id" . "binary-search-document-table-inside-posting-loop") + ("kinds" "module" "debug-error" "test") + ("pattern" + . + "posting.*(?:for|loop)[\\s\\S]*binary.search.*document") + ("severity" . "medium") + ("tags" "mmap" "postings" "binary-search" "metadata-cache" + "performance" "benchmark") + ("title" + . + "Binary-searching document metadata for every mapped posting") + ("tools" + "jerboa_howto" + "jerboa_verify" + "jerboa_security_scan" + "focused benchmark")) + (("advice" + . + "Build one logical corpus view. Exclude overridden and tombstoned base documents, include live overlay documents, compute global document count and total length, compute each query term document frequency across accepted base plus overlay postings, then score every accepted posting with those shared statistics. Regression-test IDs and scores against a fully materialized reference index across replacement, deletion, addition, stale-term, and multi-term cases.") + ("avoid" + . + "Do not score a mapped base and mutable delta with their own document counts, average lengths, and document frequencies and then merge their top-k lists. The scores are not comparable, replacements leave stale base statistics, and filtered candidates can change ranking.") + ("id" . "merge-independently-scored-bm25-shards") + ("kinds" "module" "debug-error" "test") + ("pattern" + . + "(?:base|segment|shard).*BM25[\\s\\S]*(?:delta|overlay).*BM25[\\s\\S]*(?:merge|top.k)") + ("severity" . "high") + ("tags" "bm25" "shards" "segments" "delta" "ranking" + "statistics") + ("title" + . + "Merging independently scored BM25 base and delta results") + ("tools" + "jerboa_verify" + "jerboa_security_scan" + "ranking-equivalence test" + "benchmark")) + (("advice" + . + "Inventory every owning reference, provide one release operation that clears all aliases, and expose an observable residency flag derived from the actual owners. Test activation, fallback rehydration, and release-after-recovery. Use RSS measurements separately before claiming quantitative memory savings.") + ("avoid" + . + "Do not claim a large core, cache, or buffer was released merely because its obvious owner field was set to #f. Wrapper records, segment chains, closures, registries, or callbacks may still hold aliases, preventing collection while status falsely reports success.") + ("id" + . + "clear-primary-reference-while-alias-retains-large-object") + ("kinds" "module" "debug-error" "test") + ("pattern" + . + "(?:core|cache|buffer).*(?:set!|set).*#f[\\s\\S]*(?:chain|descriptor|wrapper).*(?:core|cache|buffer)") + ("severity" . "medium") + ("tags" "memory" "aliasing" "residency" "cache" "ownership" + "gc") + ("title" + . + "Clearing the primary slot while an alias retains the large object") + ("tools" + "jerboa_verify" + "jerboa_security_scan" + "residency status test" + "RSS benchmark")) + (("advice" + . + "Compact or freeze a representative base before timing direct mmap startup. Report the active source plus base and delta document counts, and benchmark post-base delta replay separately so both costs remain visible.") + ("avoid" + . + "Do not claim direct mapped-base startup performance when the frozen base is empty or stale and the representative corpus exists entirely in later delta segments. That benchmark primarily measures overlay replay and can hide the intended startup behavior.") + ("id" + . + "benchmark-direct-mmap-startup-with-uncompacted-base") + ("kinds" "test" "docs" "module") + ("pattern" + . + "direct mmap startup benchmark.*(?:empty|uncompacted) base.*(?:full|large) delta") + ("severity" . "medium") + ("tags" "mmap" "benchmark" "startup" "compaction" "delta" + "overlay") + ("title" + . + "Benchmarking direct mmap startup against an uncompacted base") + ("tools" + "benchmark" + "status endpoint" + "jerboa_verify" + "jerboa_security_scan")) + (("advice" + . + "Use one work-needed predicate for scheduling and locked publication revalidation. Test each trigger independently.") + ("avoid" + . + "Do not add a scheduling condition while leaving publication-time admission on the old condition; valid prepared work will be discarded.") + ("id" + . + "background-work-trigger-and-commit-predicates-diverge") + ("kinds" "module" "test") + ("pattern" . "schedule.*new trigger.*commit.*old trigger") + ("severity" . "high") + ("tags" "background" "compaction" "scheduler" "publication" + "predicate") + ("title" + . + "Background work trigger and commit predicates diverge") + ("tools" + "jerboa_verify" + "jerboa_run_tests" + "concurrency regression")) + (("advice" + . + "Set the stop flag, join workers outside locks, clear worker state after all joins, then checkpoint. Keep natural completion running until the last worker exits.") + ("avoid" + . + "Do not clear a running flag and return from stop while mutation workers remain in flight; maintenance can race late writes.") + ("id" . "report-stopped-before-workers-join") + ("kinds" "module" "test" "security") + ("pattern" . "stop.*running.*false.*before.*join") + ("severity" . "high") + ("tags" "shutdown" "workers" "join" "quiescence" "race") + ("title" + . + "Reporting a service stopped before workers join") + ("tools" + "thread-join" + "dynamic-wind" + "jerboa_run_tests" + "status endpoint")) + (("advice" + . + "Keep a worst-first heap bounded by limit, replace its root only with better candidates, and sort only the retained entries. Define deterministic tie ordering and verify heap output against a full reference sort. Report separately that score accumulation may still be O(matches).") + ("avoid" + . + "Do not materialize and sort every scored candidate when the API requests a small result limit. This adds O(matches log matches) selection time and a full entry list.") + ("id" . "full-sort-for-bounded-top-k") + ("kinds" "module" "test") + ("pattern" . "take.*sort.*hash->list.*scores") + ("severity" . "medium") + ("tags" "top-k" "heap" "ranking" "search" "performance" + "memory") + ("title" . "Sorting every score for a bounded top-k result") + ("tools" + "jerboa_run_tests" + "ranking equivalence test" + "focused benchmark" + "jerboa_security_scan")) + (("advice" + . + "Read stable counts from validated metadata and inspect only bounded mutable overlays. If exactness would require a corpus scan, expose an explicit exactness flag and a safe bound instead. Benchmark status independently at representative corpus size.") + ("avoid" + . + "Do not compute operational status by walking every term, posting, row, or object on every poll. Monitoring traffic becomes corpus-sized application load.") + ("id" . "health-endpoint-scans-full-corpus") + ("kinds" "module" "test") + ("pattern" . "status.*for-each.*(?:term|posting)") + ("severity" . "high") + ("tags" "status" "health" "monitoring" "index" "latency" + "scalability") + ("title" + . + "Health or status endpoint scans the full corpus") + ("tools" + "status benchmark" + "integration test" + "jerboa_run_tests" + "jerboa_security_scan")) + (("advice" + . + "Acquire a per-index read lease while briefly holding the store mutex, release the store during scoring, release the read lease before database result hydration, and keep store-to-index as the only nested lock order. Use the write side for overlay mutation, publication, and mapping close.") + ("avoid" + . + "Do not hold a global database/store mutex while traversing an immutable search index. It serializes readers and blocks unrelated persistence behind query latency.") + ("id" . "posting-traversal-under-database-mutex") + ("kinds" "module" "test") + ("pattern" . "call-with-store.*(?:index-search|posting)") + ("severity" . "high") + ("tags" "search" "mutex" "rwlock" "concurrency" "database" + "postings") + ("title" . "Posting traversal under the database mutex") + ("tools" + "jerboa_check_balance" + "concurrency race test" + "throughput benchmark" + "jerboa_security_scan")) + (("advice" + . + "Inspect or benchmark the actual plan. For a bounded ID list, use a parameterized UNION ALL of id = ? branches when each branch receives the INTEGER PRIMARY KEY point-seek optimization. Reconstruct caller order explicitly and benchmark both forms.") + ("avoid" + . + "Do not assume one IN query is faster than several primary-key lookups. Without an IN-to-seek planner rule it can turn bounded point seeks into a full table scan.") + ("id" . "batch-in-query-without-plan-verification") + ("kinds" "module" "test") + ("pattern" . "WHERE.*(?:id|rowid).*IN \\(") + ("severity" . "high") + ("tags" "sqlite" "jsqlite" "query-plan" "batch" + "primary-key" "performance") + ("title" + . + "Replacing point seeks with an unverified IN batch") + ("tools" + "hydration benchmark" + "EXPLAIN QUERY PLAN" + "jerboa_run_tests" + "jerboa_security_scan")) + (("advice" + . + "Let the owner provide ordered close hooks. Under the normal lock order, detach derived state and close mapped handles before closing the backing store. Make close idempotent, reject operations after close, retain explicit unload only for deliberate cache restart, and regression-test hook execution exactly once.") + ("avoid" + . + "Do not close an authoritative store while leaving its cached mmap, process-global registry entry, worker handle, or other derived resource for callers to unload separately.") + ("id" . "owner-close-leaves-derived-registry-handles") + ("kinds" "module" "test" "security") + ("pattern" + . + "close.*owner.*(?:registry|mmap).*manual unload") + ("severity" . "high") + ("tags" "lifecycle" "mmap" "registry" "close" + "resource-leak" "ownership") + ("title" + . + "Owner close leaves derived registry handles alive") + ("tools" + "jerboa_security_scan" + "lifecycle regression" + "restart test" + "resource handle inspection")) + (("advice" + . + "Put page upsert, generation advance, outgoing-link replacement, and frontier enqueue in one store-owned transaction. Mutate disposable derived indexes only after commit, drop their cache on update failure, and force a late statement failure in a regression test to prove every authoritative table rolls back.") + ("avoid" + . + "Do not commit page metadata and generation in one transaction, then replace links or enqueue discovered frontier URLs in a second transaction. A crash between commits publishes a page without its crawl graph and adds an extra durable flush.") + ("id" . "split-transactions-for-one-logical-ingest") + ("kinds" "module" "test") + ("pattern" + . + "page.*transaction.*links|links.*second transaction|generation.*frontier") + ("severity" . "high") + ("tags" "transaction" "ingestion" "crawler" "frontier" "links" + "crash-consistency") + ("title" + . + "Do Not Split One Logical Ingest Across Durable Transactions") + ("tools" + "transaction rollback regression" + "fault injection" + "file-backed A/B benchmark" + "jerboa_security_scan")) + (("advice" + . + "Start from the validated dictionary document frequency and subtract only excluded document IDs found with binary search in the sorted postings. Bound the exclusion set with compaction policy, verify equivalence across replacements and tombstones, benchmark the frequency operation separately, and state that scoring still visits accepted postings.") + ("avoid" + . + "Do not rescan an entire immutable sorted posting list merely to adjust a stored document frequency for a small bounded set of replacements or tombstones. This can double common-term posting traversal on every query.") + ("id" . "rescanning-postings-for-bounded-exclusions") + ("kinds" "module" "test") + ("pattern" + . + "for-each-posting.*(?:override|tombstone|excluded).*(?:count|frequency)") + ("severity" . "high") + ("tags" "mmap" "postings" "document-frequency" "overrides" + "binary-search" "performance") + ("title" + . + "Do Not Rescan Immutable Postings for Bounded Exclusions") + ("tools" + "jerboa_verify" + "hybrid equivalence test" + "focused A/B benchmark" + "jerboa_security_scan"))) --- a/data/api-signatures.sexp +++ b/data/api-signatures.sexp @@ -1,4 +1,4 @@ -(("errors") ("generated" . "2026-07-11") +(("errors") ("generated" . "2026-07-15") ("modules" ("(jerboa build musl)" ("exports" "build-musl-binary" "make-musl-cross-target" @@ -660,6 +660,72 @@ "typed-ir-var-source" "typed-ir-var-type" "typed-ir-var?") ("file" . "lib/jerboa/typed/core.ss") ("tier" . "core")) + ("(jerboa typed kotlin ast)" + ("exports" "kt-assign-expr" "kt-assign-target" "kt-assign?" + "kt-binary-left" "kt-binary-op" "kt-binary-right" + "kt-binary?" "kt-block-result" "kt-block-statements" + "kt-block?" "kt-call-args" "kt-call-callee" "kt-call?" + "kt-class-annotations" "kt-class-body" "kt-class-kind" + "kt-class-name" "kt-class-params" "kt-class-super-types" + "kt-class-visibility" "kt-class?" "kt-expr-stmt-expr" + "kt-expr-stmt?" "kt-file-declarations" "kt-file-imports" + "kt-file-package" "kt-file?" "kt-for-range-body" + "kt-for-range-end" "kt-for-range-start" "kt-for-range-var" + "kt-for-range?" "kt-function-annotations" "kt-function-body" + "kt-function-name" "kt-function-params" + "kt-function-return-type" "kt-function-visibility" + "kt-function?" "kt-if-else" "kt-if-test" "kt-if-then" + "kt-if?" "kt-import-alias" "kt-import-path" "kt-import?" + "kt-lit-type" "kt-lit-value" "kt-lit?" "kt-member-call-args" + "kt-member-call-name" "kt-member-call-target" + "kt-member-call?" "kt-member-get-name" + "kt-member-get-target" "kt-member-get?" "kt-name-parts" + "kt-name?" "kt-new-args" "kt-new-type" "kt-new?" + "kt-param-default" "kt-param-mutable?" "kt-param-name" + "kt-param-property" "kt-param-type" "kt-param-visibility" + "kt-param?" "kt-property-annotations" "kt-property-init" + "kt-property-mutable?" "kt-property-name" "kt-property-type" + "kt-property-visibility" "kt-property?" "kt-return-expr" + "kt-return?" "kt-type-args" "kt-type-name" + "kt-type-nullable?" "kt-type?" "kt-unary-expr" "kt-unary-op" + "kt-unary?" "kt-val-init" "kt-val-mutable?" "kt-val-name" + "kt-val-type" "kt-val?" "kt-when-branch-body" + "kt-when-branch-pattern" "kt-when-branch?" + "kt-when-branches" "kt-when-subject" "kt-when?" + "make-kt-assign" "make-kt-binary" "make-kt-block" + "make-kt-call" "make-kt-class" "make-kt-expr-stmt" + "make-kt-file" "make-kt-for-range" "make-kt-function" + "make-kt-if" "make-kt-import" "make-kt-lit" + "make-kt-member-call" "make-kt-member-get" "make-kt-name" + "make-kt-new" "make-kt-param" "make-kt-property" + "make-kt-return" "make-kt-type" "make-kt-unary" + "make-kt-val" "make-kt-when" "make-kt-when-branch") + ("file" . "lib/jerboa/typed/kotlin/ast.ss") + ("tier" . "core")) + ("(jerboa typed kotlin lower)" + ("exports" + "typed-library-form->kotlin-string" + "typed-module->kotlin-file" + "typed-module->kotlin-string" + "typed-type->kotlin-type") + ("file" . "lib/jerboa/typed/kotlin/lower.ss") + ("tier" . "core")) + ("(jerboa typed kotlin print)" + ("exports" "kotlin-char-literal" "kotlin-expr->string" + "kotlin-file->string" "kotlin-package-name" + "kotlin-string-literal" "kotlin-symbol-name" + "kotlin-type->string") + ("file" . "lib/jerboa/typed/kotlin/print.ss") + ("tier" . "core")) + ("(jerboa typed kotlin)" + ("exports" "kotlin-char-literal" "kotlin-expr->string" + "kotlin-file->string" "kotlin-package-name" + "kotlin-string-literal" "kotlin-symbol-name" + "kotlin-type->string" "typed-library-form->kotlin-string" + "typed-module->kotlin-file" "typed-module->kotlin-string" + "typed-type->kotlin-type") + ("file" . "lib/jerboa/typed/kotlin.ss") + ("tier" . "core")) ("(jerboa typed llvmir)" ("exports" "llvm-float-literal" "llvm-function-symbol" "llvm-module-mangle" "llvm-symbol-name" @@ -6683,6 +6749,10 @@ "regex-split" "regex-star-inner" "regex-star?") ("file" . "lib/std/text/regex-compile.ss") ("tier" . "stable")) + ("(std text sexpr)" + ("exports" "sexpr->string" "string->sexpr") + ("file" . "lib/std/text/sexpr.ss") + ("tier" . "stable")) ("(std text template)" ("exports" "make-template-env" "template-compile" "template-env-ref" "template-env-set!" "template-escape-html" "template-render" @@ -7005,13 +7075,13 @@ ("file" . "lib/std/zipper.ss") ("tier" . "stable"))) ("source_root" . "/Users/user/mine/jerboa") - ("stats" ("modules" . 713) ("parse_errors" . 0) ("symbols" . 10344) + ("stats" ("modules" . 718) ("parse_errors" . 0) ("symbols" . 10476) ("tiers" ("compat" . 47) - ("core" . 34) - ("stable" . 554) + ("core" . 38) + ("stable" . 555) ("unstable" . 78)) - ("total_exports" . 13560)) + ("total_exports" . 13703)) ("symbol_index" ("%chan-enqueue-raw!" "(std csp)") ("&actor-dead" "(std error conditions)") ("&actor-timeout" "(std error conditions)") @@ -12013,6 +12083,27 @@ "(jerboa runtime)" "(std gambit-compat)" "(std prelude)") ("keywordize-keys" "(std clojure walk)") ("kill" "(std os signal)") + ("kotlin-char-literal" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-expr->string" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-file->string" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-package-name" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-string-literal" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-symbol-name" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") + ("kotlin-type->string" + "(jerboa typed kotlin print)" + "(jerboa typed kotlin)") ("kqueue-add-read" "(std os kqueue)") ("kqueue-add-signal" "(std os kqueue)") ("kqueue-add-timer" "(std os kqueue)") @@ -12025,6 +12116,101 @@ ("kqueue-event?" "(std os kqueue)") ("kqueue-remove" "(std os kqueue)") ("kqueue-wait" "(std os kqueue)") + ("kt-assign-expr" "(jerboa typed kotlin ast)") + ("kt-assign-target" "(jerboa typed kotlin ast)") + ("kt-assign?" "(jerboa typed kotlin ast)") + ("kt-binary-left" "(jerboa typed kotlin ast)") + ("kt-binary-op" "(jerboa typed kotlin ast)") + ("kt-binary-right" "(jerboa typed kotlin ast)") + ("kt-binary?" "(jerboa typed kotlin ast)") + ("kt-block-result" "(jerboa typed kotlin ast)") + ("kt-block-statements" "(jerboa typed kotlin ast)") + ("kt-block?" "(jerboa typed kotlin ast)") + ("kt-call-args" "(jerboa typed kotlin ast)") + ("kt-call-callee" "(jerboa typed kotlin ast)") + ("kt-call?" "(jerboa typed kotlin ast)") + ("kt-class-annotations" "(jerboa typed kotlin ast)") + ("kt-class-body" "(jerboa typed kotlin ast)") + ("kt-class-kind" "(jerboa typed kotlin ast)") + ("kt-class-name" "(jerboa typed kotlin ast)") + ("kt-class-params" "(jerboa typed kotlin ast)") + ("kt-class-super-types" "(jerboa typed kotlin ast)") + ("kt-class-visibility" "(jerboa typed kotlin ast)") + ("kt-class?" "(jerboa typed kotlin ast)") + ("kt-expr-stmt-expr" "(jerboa typed kotlin ast)") + ("kt-expr-stmt?" "(jerboa typed kotlin ast)") + ("kt-file-declarations" "(jerboa typed kotlin ast)") + ("kt-file-imports" "(jerboa typed kotlin ast)") + ("kt-file-package" "(jerboa typed kotlin ast)") + ("kt-file?" "(jerboa typed kotlin ast)") + ("kt-for-range-body" "(jerboa typed kotlin ast)") + ("kt-for-range-end" "(jerboa typed kotlin ast)") + ("kt-for-range-start" "(jerboa typed kotlin ast)") + ("kt-for-range-var" "(jerboa typed kotlin ast)") + ("kt-for-range?" "(jerboa typed kotlin ast)") + ("kt-function-annotations" "(jerboa typed kotlin ast)") + ("kt-function-body" "(jerboa typed kotlin ast)") + ("kt-function-name" "(jerboa typed kotlin ast)") + ("kt-function-params" "(jerboa typed kotlin ast)") + ("kt-function-return-type" "(jerboa typed kotlin ast)") + ("kt-function-visibility" "(jerboa typed kotlin ast)") + ("kt-function?" "(jerboa typed kotlin ast)") + ("kt-if-else" "(jerboa typed kotlin ast)") + ("kt-if-test" "(jerboa typed kotlin ast)") + ("kt-if-then" "(jerboa typed kotlin ast)") + ("kt-if?" "(jerboa typed kotlin ast)") + ("kt-import-alias" "(jerboa typed kotlin ast)") + ("kt-import-path" "(jerboa typed kotlin ast)") + ("kt-import?" "(jerboa typed kotlin ast)") + ("kt-lit-type" "(jerboa typed kotlin ast)") + ("kt-lit-value" "(jerboa typed kotlin ast)") + ("kt-lit?" "(jerboa typed kotlin ast)") + ("kt-member-call-args" "(jerboa typed kotlin ast)") + ("kt-member-call-name" "(jerboa typed kotlin ast)") + ("kt-member-call-target" "(jerboa typed kotlin ast)") + ("kt-member-call?" "(jerboa typed kotlin ast)") + ("kt-member-get-name" "(jerboa typed kotlin ast)") + ("kt-member-get-target" "(jerboa typed kotlin ast)") + ("kt-member-get?" "(jerboa typed kotlin ast)") + ("kt-name-parts" "(jerboa typed kotlin ast)") + ("kt-name?" "(jerboa typed kotlin ast)") + ("kt-new-args" "(jerboa typed kotlin ast)") + ("kt-new-type" "(jerboa typed kotlin ast)") + ("kt-new?" "(jerboa typed kotlin ast)") + ("kt-param-default" "(jerboa typed kotlin ast)") + ("kt-param-mutable?" "(jerboa typed kotlin ast)") + ("kt-param-name" "(jerboa typed kotlin ast)") + ("kt-param-property" "(jerboa typed kotlin ast)") + ("kt-param-type" "(jerboa typed kotlin ast)") + ("kt-param-visibility" "(jerboa typed kotlin ast)") + ("kt-param?" "(jerboa typed kotlin ast)") + ("kt-property-annotations" "(jerboa typed kotlin ast)") + ("kt-property-init" "(jerboa typed kotlin ast)") + ("kt-property-mutable?" "(jerboa typed kotlin ast)") + ("kt-property-name" "(jerboa typed kotlin ast)") + ("kt-property-type" "(jerboa typed kotlin ast)") + ("kt-property-visibility" "(jerboa typed kotlin ast)") + ("kt-property?" "(jerboa typed kotlin ast)") + ("kt-return-expr" "(jerboa typed kotlin ast)") + ("kt-return?" "(jerboa typed kotlin ast)") + ("kt-type-args" "(jerboa typed kotlin ast)") + ("kt-type-name" "(jerboa typed kotlin ast)") + ("kt-type-nullable?" "(jerboa typed kotlin ast)") + ("kt-type?" "(jerboa typed kotlin ast)") + ("kt-unary-expr" "(jerboa typed kotlin ast)") + ("kt-unary-op" "(jerboa typed kotlin ast)") + ("kt-unary?" "(jerboa typed kotlin ast)") + ("kt-val-init" "(jerboa typed kotlin ast)") + ("kt-val-mutable?" "(jerboa typed kotlin ast)") + ("kt-val-name" "(jerboa typed kotlin ast)") + ("kt-val-type" "(jerboa typed kotlin ast)") + ("kt-val?" "(jerboa typed kotlin ast)") + ("kt-when-branch-body" "(jerboa typed kotlin ast)") + ("kt-when-branch-pattern" "(jerboa typed kotlin ast)") + ("kt-when-branch?" "(jerboa typed kotlin ast)") + ("kt-when-branches" "(jerboa typed kotlin ast)") + ("kt-when-subject" "(jerboa typed kotlin ast)") + ("kt-when?" "(jerboa typed kotlin ast)") ("label->pred-datum" "(std regex-ct-impl)") ("lambda-lift" "(jerboa wasm closure)") ("lambda-params" "(jerboa wasm closure)") @@ -12887,6 +13073,30 @@ ("make-keyword" "(jerboa clojure)" "(jerboa core)" "(jerboa prelude clean)" "(jerboa prelude safe)" "(jerboa prelude)" "(jerboa runtime)" "(std gambit-compat)" "(std prelude)") + ("make-kt-assign" "(jerboa typed kotlin ast)") + ("make-kt-binary" "(jerboa typed kotlin ast)") + ("make-kt-block" "(jerboa typed kotlin ast)") + ("make-kt-call" "(jerboa typed kotlin ast)") + ("make-kt-class" "(jerboa typed kotlin ast)") + ("make-kt-expr-stmt" "(jerboa typed kotlin ast)") + ("make-kt-file" "(jerboa typed kotlin ast)") + ("make-kt-for-range" "(jerboa typed kotlin ast)") + ("make-kt-function" "(jerboa typed kotlin ast)") + ("make-kt-if" "(jerboa typed kotlin ast)") + ("make-kt-import" "(jerboa typed kotlin ast)") + ("make-kt-lit" "(jerboa typed kotlin ast)") + ("make-kt-member-call" "(jerboa typed kotlin ast)") + ("make-kt-member-get" "(jerboa typed kotlin ast)") + ("make-kt-name" "(jerboa typed kotlin ast)") + ("make-kt-new" "(jerboa typed kotlin ast)") + ("make-kt-param" "(jerboa typed kotlin ast)") + ("make-kt-property" "(jerboa typed kotlin ast)") + ("make-kt-return" "(jerboa typed kotlin ast)") + ("make-kt-type" "(jerboa typed kotlin ast)") + ("make-kt-unary" "(jerboa typed kotlin ast)") + ("make-kt-val" "(jerboa typed kotlin ast)") + ("make-kt-when" "(jerboa typed kotlin ast)") + ("make-kt-when-branch" "(jerboa typed kotlin ast)") ("make-landlock-error" "(std os landlock)" "(std os landlock-native)") @@ -16438,6 +16648,7 @@ ("sexp->lockfile" "(jerboa lock)") ("sexp->provenance" "(std build reproducible)") ("sexp->sbom" "(std build sbom)") + ("sexpr->string" "(std text sexpr)") ("sftp-attrs-atime" "(std net ssh sftp)" "(std net ssh)") ("sftp-attrs-gid" "(std net ssh sftp)" "(std net ssh)") ("sftp-attrs-mtime" "(std net ssh sftp)" "(std net ssh)") @@ -17261,6 +17472,7 @@ "(jerboa prelude safe)" "(jerboa prelude)" "(jerboa runtime)" "(std gambit-compat)" "(std prelude)") ("string->platform" "(std build cross)") + ("string->sexpr" "(std text sexpr)") ("string->text" "(std srfi srfi-135)") ("string->toml" "(std text toml)") ("string->transit" "(jerboa clojure)" "(std transit)") @@ -18313,6 +18525,9 @@ "(jerboa typed wrapper)") ("typed-library-form->jerboa-wrapper-string" "(jerboa typed wrapper)") + ("typed-library-form->kotlin-string" + "(jerboa typed kotlin lower)" + "(jerboa typed kotlin)") ("typed-library-form->llvmir-string" "(jerboa typed llvmir)") ("typed-library-form->rust-string" "(jerboa typed rust)") @@ -18333,6 +18548,12 @@ "(jerboa typed wrapper)") ("typed-module->jerboa-wrapper-string" "(jerboa typed wrapper)") + ("typed-module->kotlin-file" + "(jerboa typed kotlin lower)" + "(jerboa typed kotlin)") + ("typed-module->kotlin-string" + "(jerboa typed kotlin lower)" + "(jerboa typed kotlin)") ("typed-module->llvmir-string" "(jerboa typed llvmir)") ("typed-module->rust-string" "(jerboa typed rust)") ("typed-module-declarations" "(jerboa typed parser)") @@ -18364,6 +18585,9 @@ ("typed-resource-name" "(jerboa typed parser)") ("typed-resource-source" "(jerboa typed parser)") ("typed-resource?" "(jerboa typed parser)") + ("typed-type->kotlin-type" + "(jerboa typed kotlin lower)" + "(jerboa typed kotlin)") ("typed-type-decl-name" "(jerboa typed parser)") ("typed-type-decl-source" "(jerboa typed parser)") ("typed-type-decl?" "(jerboa typed parser)") --- a/data/cookbooks.sexp +++ b/data/cookbooks.sexp @@ -6670,4 +6670,75 @@ "job-control") ("title" . - "Pair a SIGCHLD-blocking Fork/Exec Wrapper With Unwind-Safe Parent Cleanup"))) + "Pair a SIGCHLD-blocking Fork/Exec Wrapper With Unwind-Safe Parent Cleanup")) + (("code" + . + "(import (jerboa prelude)\n (jsh ffi))\n\n(def (tty-port? port)\n (with-catch (lambda (e) #f)\n (lambda ()\n (let ([fd (with-catch (lambda (e) #f)\n (lambda () (port-file-descriptor port)))])\n (cond\n [(fixnum? fd) (= (ffi-isatty fd) 1)]\n [(eq? port (current-input-port)) (= (ffi-isatty 0) 1)]\n [else #f])))))") ("id" . "side-effect-free-tty-predicate") + ("imports" "(jsh ffi)") + ("notes" + . + "Use this in line editors and REPL prompts. Do not use tty-mode-set! as a predicate: it changes terminal state and can flush queued typeahead, causing characters typed while a foreground command is busy to disappear before the next prompt.") + ("tags" "terminal" "tty" "isatty" "port-file-descriptor" + "lineedit" "typeahead") + ("title" + . + "Check terminal status without mutating tty mode")) + (("code" + . + "(import (jerboa prelude))\n\n(def (consume-sgr params)\n (let ([fg 'default] [bg 'default] [attrs '()])\n (let loop ([ps (if (null? params) '(0) params)])\n (if (null? ps)\n (alist (fg fg) (bg bg) (attrs (reverse attrs)))\n (let ([p (car ps)])\n (cond\n [(= p 0)\n (set! fg 'default)\n (set! bg 'default)\n (set! attrs '())\n (loop (cdr ps))]\n [(= p 1)\n (set! attrs (cons 'bold attrs))\n (loop (cdr ps))]\n [(= p 38)\n (cond\n [(and (>= (length ps) 3) (= (cadr ps) 5))\n (set! fg (caddr ps))\n (loop (cdddr ps))]\n [(and (>= (length ps) 5) (= (cadr ps) 2))\n (set! fg (list 'rgb (list-ref ps 2) (list-ref ps 3) (list-ref ps 4)))\n (loop (list-tail ps 5))]\n [else (loop (cdr ps))])]\n [(= p 48)\n (cond\n [(and (>= (length ps) 3) (= (cadr ps) 5))\n (set! bg (caddr ps))\n (loop (cdddr ps))]\n [(and (>= (length ps) 5) (= (cadr ps) 2))\n (set! bg (list 'rgb (list-ref ps 2) (list-ref ps 3) (list-ref ps 4)))\n (loop (list-tail ps 5))]\n [else (loop (cdr ps))])]\n [else (loop (cdr ps))]))))))\n\n(assert! (equal? (consume-sgr '(38 5 214 48 5 235 1))\n '((fg . 214) (bg . 235) (attrs . (bold)))))") ("id" . "vt-sgr-consume-extended-colors") + ("imports" "(jerboa prelude)") + ("notes" + . + "When handling SGR 38/48 extended color groups, recurse with the remaining list immediately. Do not mutate the loop cursor and then also recurse on (cdr ps), or a compound sequence such as 38;5;214;48;5;235;1 will skip the 48 background marker. For truecolor, consume five entries: 38/48, 2, R, G, B.") + ("tags" "vt" "sgr" "terminal" "256-color" "truecolor" + "parser") + ("title" + . + "Parse compound SGR extended colors without skipping the next code")) + (("code" + . + "(import (jerboa typed kotlin))\n\n(define source\n '(typed-library (sample typed calc)\n (export add-one)\n (def (add-one (x : Nat)) : Nat\n (+ x 1))))\n\n(display (typed-library-form->kotlin-string source))\n") ("id" . "typed-kotlin-structured-ast-backend") + ("imports" "(jerboa typed kotlin)") + ("notes" + . + "Use `(jerboa typed kotlin ast)` for declarations, statements, expressions, and types; render only through `(jerboa typed kotlin print)`. Do not add raw Kotlin/Java text nodes. Unsupported typed IR should fail closed until a structured KAST node, printer case, and test exist.") + ("tags" "typed" "kotlin" "backend" "compiler" "ast" + "printer") + ("title" + . + "Typed Jerboa Kotlin backend with structured KAST and no raw source node")) + (("code" + . + "(import (jerboa prelude)\n (only (std safe-fasl)\n safe-fasl-write-bytevector *fasl-max-byte-size*\n *fasl-max-object-count*)\n (only (std crypto sha256-pure) sha256-hex)\n (only (std security taint) safe-delete-file))\n\n(def (encode-checked value)\n (let ([payload\n (parameterize ([*fasl-max-byte-size* (* 64 1024 1024)]\n [*fasl-max-object-count* 1000000])\n (safe-fasl-write-bytevector value))])\n (safe-fasl-write-bytevector\n (list \"checked-file-v1\" (sha256-hex payload) payload))))\n\n(def (write-bytes! path bytes)\n (let ([port (open-file-output-port path (file-options no-fail))])\n (dynamic-wind void\n (lambda () (put-bytevector port bytes) (flush-output-port port))\n (lambda () (close-port port)))))\n\n(def (atomic-write! path bytes)\n (let ([temporary (string-append path \".tmp\")])\n (when (file-exists? temporary) (safe-delete-file temporary))\n (guard (error [else\n (when (file-exists? temporary)\n (safe-delete-file temporary))\n (raise error)])\n (write-bytes! temporary bytes)\n (rename-file temporary path))))\n\n(def (publish-chain! manifest-path segment-path\n segment-value manifest-value obsolete-paths)\n ;; Publish data first. Never let the manifest name a missing segment.\n (unless (file-exists? segment-path)\n (atomic-write! segment-path (encode-checked segment-value)))\n ;; This rename is the logical commit point.\n (atomic-write! manifest-path (encode-checked manifest-value))\n ;; Cleanup after publication is best-effort and cannot roll state back.\n (for-each\n (lambda (path)\n (guard (ignored [else (void)])\n (when (file-exists? path) (safe-delete-file path))))\n obsolete-paths)\n manifest-path)") ("id" . "atomic-immutable-segment-manifest-publication") + ("imports" + "(jerboa prelude)" + "(std safe-fasl)" + "(std crypto sha256-pure)" + "(std security taint)") + ("notes" + . + "Write and checksum every immutable data file before atomically replacing the manifest. The manifest rename is the logical commit point. After it succeeds, obsolete-file cleanup must be best-effort: a deletion error must not make the caller retain or republish the old chain. Keep temporary files beside their targets and validate existing immutable files before reuse.") + ("tags" "segments" "manifest" "atomic-write" "safe-fasl" + "compaction" "crash-consistency") + ("title" . "Publish an Immutable Segment Chain Atomically")) + (("code" + . + "(import (jerboa prelude)\n (only (std concur util) make-rwlock with-read-lock with-write-lock))\n(def lock (make-rwlock))\n(def value 0)\n(def (read-value) (with-read-lock lock value))\n(def (increment!)\n (with-write-lock lock (set! value (+ value 1)) value))\n(def threads\n (for/collect ([i (in-range 4)])\n (fork-thread (lambda () (for ([j (in-range 100)]) (read-value))))))\n(increment!)\n(for-each thread-join threads)") ("id" . "writer-preferring-rwlock-read-mostly-state") + ("imports" + "(only (std concur util) make-rwlock with-read-lock with-write-lock)") + ("notes" + . + "make-rwlock takes zero arguments. These are body-style macros from (std concur util), release via dynamic-wind, and prefer waiting writers. Keep one global lock order.") + ("tags" "rwlock" "concurrency" "read-lock" "write-lock") + ("title" . "Use Jerboa rwlocks for read-mostly state")) + (("code" + . + "(import (except (jerboa prelude) sqlite-query)\n (only (jsqlite api) sqlite-query))\n(def (pages-by-id db ids)\n (if (null? ids) '()\n (let ([branch \"SELECT id,title FROM page WHERE id = ?\"])\n (apply sqlite-query db\n (string-join (make-list (length ids) branch) \" UNION ALL \")\n ids))))") ("id" . "jsqlite-bounded-batch-primary-key-seeks") + ("imports" + "(except (jerboa prelude) sqlite-query)" + "(only (jsqlite api) sqlite-query)") + ("notes" + . + "Keep count bounded and values parameterized. Each id = ? branch can use the jsqlite rowid point seek; IN may scan. Map returned rows by ID when caller order matters.") + ("tags" "jsqlite" "union-all" "primary-key" "batch") + ("title" . "Batch bounded jsqlite primary-key seeks"))) --- a/data/error-fixes.sexp +++ b/data/error-fixes.sexp @@ -2692,4 +2692,13 @@ "If the symbol belongs to an optional applet set that is statically linked only into release binaries, probe each entry with foreign-entry? before constructing foreign-procedure and register only the available bindings. Do not invent a bare-name development cdylib fallback.") ("id" . "ffi-no-entry-optional-static-applet") ("pattern" . "no entry for \\\"jsh_[^\\\"]+\\\"") - ("type" . "FFI Entry Not Found"))) + ("type" . "FFI Entry Not Found")) + (("explanation" + . + "The compiler reports the next definition even though the actual cause is an earlier unclosed form.") + ("fix" + . + "Run jerboa_check_balance on the hand-authored source. A preceding form is usually missing a closing delimiter, so the reported top-level def became nested. Fix the first imbalance, rebuild generated output, and do not patch generated files.") + ("id" . "invalid-context-definition-from-unclosed-form") + ("pattern" . "invalid context for definition.*line [0-9]+") + ("type" . "syntax"))) --- a/data/features.sexp +++ b/data/features.sexp @@ -3307,4 +3307,44 @@ ("use_case" . "Before accepting a source or packaged binary that imports FFI-heavy modules such as OpenSSL wrappers.") + ("votes" . 0)) + (("description" + . + "jerboa_verify on jsh lineedit.ss fails before expansion with invalid character name #\\escape, while the project jerbuild path accepts the file. module_exports/module_reference can locate generated modules with module_exists but then fail to import local libraries such as (jsh ffi). Align verifier/introspection reader and libdir handling with jerbuild so source checks work on repos using Chez character names and generated src libraries.") ("estimated_token_reduction" . 1200) + ("example_scenario" + . + "In /Users/user/mine/jerboa-shell, lineedit.ss contains (def ESC #\\escape). jerboa_verify reports invalid character name #\\escape, but make jerboa succeeds. module_exists finds /src/jsh/ffi.sls, but module_exports reports library (jsh ffi) not found.") + ("id" + . + "verifier-read-chez-character-names-and-local-libdirs") + ("impact" + . + "Reduces false verifier failures and fallback shell inspection for Jerboa shell modules.") + ("tags" "verifier" "reader" "chez" "libdirs" + "module-exports" "jsh") + ("title" + . + "Make verifier handle Chez character names and generated local libdirs") + ("use_case" + . + "A coding agent fixing line editor code should be able to run jerboa_verify and module_exports on lineedit.ss instead of falling back to make jerboa and generated .sls inspection.")