Wire MCP advisor guidance tools
ober
f19da0b25615cacef5e62c8091227b05c6a6917e
new file mode 100644 --- /dev/null +++ b/docs/mcp-advisor-plan.md @@ -0,0 +1,287 @@ +# Jerboa MCP Advisor Plan + +This plan makes `jmcp` a stronger domain brain for local coding agents without +moving Jerboa-specific expertise into those agents. The intended boundary is: + +```text +jerboa-mcp Jerboa knowledge, recipes, error diagnosis, verify advice +jcode verified edit/verify/repair harness and execution gate +local model constrained code generator using MCP guidance +``` + +The current MCP already has useful primitives: cookbook search, task bundles, +workflow advice, syntax/compile verification, error-fix lookup, security scans, +feature tracking, and compact tool discovery. The gap is that weak local models +often fail to choose the right primitive at the right time. They need the MCP to +return a small, opinionated plan up front and a small, opinionated repair path +after failures. + +## Goals + +- Give clients one preflight call that answers: "What should the model do for + this request?" +- Give clients one failure call that answers: "Given this verifier error, what + repair move should the model make next?" +- Recommend a verify command and write scope before generation starts. +- Package cookbook recipes and anti-patterns as task-specific battle cards, not + broad tutorials. +- Keep outputs compact enough for small local models and harness prompts. +- Keep all Jerboa facts in `jmcp`; jcode should only call the advisor and inject + the returned guidance. + +## Non-Goals + +- Do not embed an LLM inside `jmcp`. +- Do not make `jmcp` edit project files on behalf of the harness. +- Do not replace `jcode verified`; the verify gate remains the execution + authority. +- Do not dump the whole cookbook into the prompt. + +## Current Baseline + +Existing relevant tools: + +- `jerboa_task_workflow_advisor` +- `jerboa_cookbook_task_bundle` +- `jerboa_howto` +- `jerboa_howto_get` +- `jerboa_verify` +- `jerboa_verify_changes` +- `jerboa_check_balance` +- `jerboa_explain_error` +- `jerboa_error_fix_lookup` +- `jerboa_compact_tool_manifest` + +These tools are useful, but they are mostly independent. A weak model must +still decide which one to call first, how to combine results, and when to stop +inspecting. + +## Feature Set + +### 1. Request Advisor + +Tool: + +```text +jerboa_request_advisor(task, project_path?, file_path?, max_recipes?) +``` + +Purpose: + +Return a compact preflight battle card for the task. + +Output should include: + +- inferred task type, +- recommended verify command, +- recommended write scope, +- relevant cookbook recipe IDs and titles, +- first actions, +- anti-patterns to avoid, +- follow-up MCP tools. + +This is the tool jcode should call before a `jcode verified` Jerboa run. + +### 2. Verify Plan + +Tool: + +```text +jerboa_verify_plan(task, project_path?, file_path?) +``` + +Purpose: + +Recommend a verify command that actually checks the requested task. This should +discourage weak checks such as "file loads" when the task asks for observable +behavior. + +Examples: + +- Standalone script: `scheme --script life.ss` +- CLI script with expected output: `scheme --script tool.ss args...` +- Repo change: focused `make` target or `jerboa_verify_changes` +- Docs task: `jerboa_doc_verify` + +### 3. Failure Advisor + +Tool: + +```text +jerboa_failure_advisor(verify_output | error_message, file_path?, snippet?) +``` + +Purpose: + +Map verifier/compiler/test output to the next repair action. It should return a +direct recommendation such as: + +- call `jerboa_check_balance`, +- read a line span, +- use `replace_range`, +- call `jerboa_function_signature`, +- call `jerboa_error_fix_lookup`, +- stop broad inspection and patch the known failing span. + +### 4. Cookbook Bundle Upgrade + +Extend `jerboa_cookbook_task_bundle` over time so it includes: + +- selected recipes, +- task-specific anti-patterns, +- verify plan, +- failure repair hints, +- "do not do this" examples for local models. + +The current bundle is useful but should become more prescriptive. + +### 5. Anti-Pattern Library + +Track recurring local-model mistakes as reusable patterns: + +- using Gerbil imports or commands for Jerboa tasks, +- assuming unavailable SRFIs, +- writing `.sls` libraries for simple user scripts, +- defining `main` but never invoking it, +- tiny paren edits on unbalanced code, +- continuing reads after verifier already identified the broken span, +- weak verify commands that only load the file. + +These patterns can initially live in code; later they can move to data files +if they need live updates like cookbooks. + +Tool: + +```text +jerboa_anti_pattern_lookup(task?, kind?, file_path?, max_results?) +``` + +Purpose: + +Return ranked mistakes to avoid for a task or task kind without requiring the +full request advisor output. + +### 6. jcode Integration + +Target jcode flow: + +```text +jcode verified starts +-> detects Jerboa-looking task or .ss/.sls file +-> calls jerboa_request_advisor +-> injects returned battle card as task guidance +-> exposes MCP tools during the workflow +-> on verify failure, optionally calls jerboa_failure_advisor +-> model repairs under verified gate +``` + +This keeps jcode generic. Jerboa-specific judgment remains in `jmcp`. + +## Implementation Phases + +### Phase 1: Add Advisor Tools + +- Implement `jerboa_request_advisor`. +- Implement `jerboa_verify_plan`. +- Implement `jerboa_failure_advisor`. +- Register tools as critical/high-value MCP tools. +- Add protocol tests that confirm each tool is visible and returns expected + fields for script, module, docs, and failure scenarios. + +### Phase 2: Improve Cookbook Bundles + +- Add anti-pattern and verify-plan sections to + `jerboa_cookbook_task_bundle`. +- Add task-kind-specific recipe query expansion. +- Add concise "first actions" and "stop condition" sections. + +### Phase 3: Add Data-Backed Anti-Patterns + +- Add `data/anti-patterns.sexp` or extend `data/error-fixes.sexp`. +- Add search/rank helpers for anti-patterns. +- Add writer tools only after the read path is stable. + +### Phase 4: jcode Preflight + +- In `jcode verified`, detect Jerboa tasks and call + `jerboa_request_advisor` when MCP is available. +- Inject the returned text into `task-guidance`. +- Keep `--guidance-file` as an explicit override/addition. +- Preserve `--no-mcp` and `--no-run-aliases` controls. + +### Phase 5: Failure Feedback Loop + +- Let jcode call `jerboa_failure_advisor` after failed verification when the MCP + server is available. +- Append the failure advice to the next tool result or task guidance. +- Track whether it reduces repeated read/inspect loops. + +## Testing + +Focused MCP tests should cover: + +- tools appear in `tools/list`, +- request advisor classifies script/module/docs/debug tasks, +- request advisor includes verify command and anti-patterns, +- verify plan uses file-specific commands, +- failure advisor maps common syntax/arity/unbound/import errors to repair + actions, +- compact manifest includes the new advisor tools, +- no existing cookbook or workflow tools regress. + +Manual harness tests should cover: + +```bash +jcode verified "write a text Conway's Game of Life in Jerboa" \ + --verify "scheme life.ss" \ + --write-scope "life.ss" \ + --guidance-file <advisor-output> +``` + +Success metric: + +- fewer first-turn hallucinated imports, +- fewer pointless shell/tool probes, +- fewer repeated reads after clear verifier output, +- more local-model tasks reaching a passing verify. + +## Implementation Status + +Current landed slice: + +- `jerboa_request_advisor` is implemented and registered as a critical MCP tool. +- `jerboa_verify_plan` is implemented and registered as a critical MCP tool. +- `jerboa_failure_advisor` is implemented and registered as a critical MCP tool. +- `jerboa_compact_tool_manifest` now promotes the advisor tools in its core + tool list. +- Dispatcher search aliases include request, verify, and failure advisor terms. +- `jerboa_cookbook_task_bundle` now includes verify plan, write scope, + anti-patterns, and failure-loop guidance. +- `data/anti-patterns.sexp` stores reusable local-model mistakes with avoid/do + guidance and follow-up tools. +- `jerboa_anti_pattern_lookup` exposes the anti-pattern data directly. +- Recipe selection now expands task queries by inferred kind so tasks can find + useful script/module/docs/debug recipes even when the exact user wording is + not in the cookbook. +- Protocol tests cover tool visibility, advisor output, failure diagnosis, and + the upgraded cookbook bundle. +- `jcode verified` now calls a registered MCP `jerboa_request_advisor` before + Jerboa-looking runs and appends `jerboa_failure_advisor` output to failed + verifier details when that MCP tool is available. + +Remaining work: + +- Add writer tooling for anti-patterns only if the read path proves useful + enough to need live updates. + +## Open Questions + +- Should advisor outputs be pure text, pure JSON, or text plus a JSON block? +- Should `jmcp` own a data-backed anti-pattern registry from the start? +- Should jcode call the failure advisor automatically, or only inject the + request advisor up front? +- How much cookbook code is safe to include by default for very small context + windows? + +Phase 1 should keep the outputs text-first and compact. If jcode needs machine +fields later, add a `format=json` option rather than breaking the default. --- a/mcp/server.ss +++ b/mcp/server.ss @@ -41,6 +41,10 @@ "jerboa_error_fix_add" "jerboa_check_balance" "jerboa_balanced_replace" + "jerboa_request_advisor" + "jerboa_verify_plan" + "jerboa_failure_advisor" + "jerboa_anti_pattern_lookup" "jerboa_task_workflow_advisor" "jerboa_compact_tool_manifest" "jerboa_script_scaffold_verify" @@ -1061,6 +1065,10 @@ ("error_fix_add" "add error fix recipe diagnostic workaround") ("error_fix_lookup" "lookup error fix diagnostic workaround") ("security_pattern_add" "add security rule pattern scanner finding") + ("request_advisor" "request advisor preflight task battle card local model cookbook verify scope") + ("verify_plan" "verify command plan oracle jcode verified write scope") + ("failure_advisor" "failure advisor repair error verify output compiler diagnosis next action") + ("anti_pattern_lookup" "anti pattern lookup local model mistakes avoid guidance failure") ("task_workflow_advisor" "workflow plan advisor local model agent mcp next tools") ("compact_tool_manifest" "manifest guide local model small context tool order") ("script_scaffold_verify" "script command scaffold cli args verify write file"))) @@ -1874,35 +1882,44 @@ (recipe-text recipe #f) (recipe-summary-line recipe))) -(def (cookbook-bundle-workflow file project) - (string-append - "Generic workflow:\n" - "1. Read the target files and project layout before editing.\n" - "2. Use the recipe imports, API names, and code shape as verified examples; do not copy unrelated logic blindly.\n" - "3. Confirm unfamiliar APIs with jerboa_module_exports or jerboa_function_signature.\n" - "4. Write the smallest complete artifact that should pass the verifier.\n" - "5. Run jerboa_verify on the changed .ss file" - (if file (string-append " (" file ")") "") - ", then the smallest runnable command or focused test" - (if project (string-append " in " project) "") - ".\n" - "6. If verification fails, use jerboa_explain_error or jerboa_error_fix_lookup before the next edit.\n" - "7. Save a non-trivial new working pattern with jerboa_howto_add.")) +(def (cookbook-bundle-workflow task kind file project anti-patterns-path anti-patterns-limit) + (let ([verify (shell-verify-command kind (string-downcase task) file project)] + [scope (recommended-write-scope kind file)]) + (string-append + "Generic workflow:\n" + "1. Read the target files and project layout before editing.\n" + "2. Use the recipe imports, API names, and code shape as verified examples; do not copy unrelated logic blindly.\n" + "3. Confirm unfamiliar APIs with jerboa_module_exports or jerboa_function_signature.\n" + "4. Write the smallest complete artifact that should pass the verifier.\n" + "5. Run the recommended verifier, then broaden to focused repo tests only if needed.\n" + "6. If verification fails, call jerboa_failure_advisor with the exact verifier output before the next edit.\n" + "7. Save a non-trivial new working pattern with jerboa_howto_add.\n\n" + "Verify plan:\n" + "- Shell verify: " verify "\n" + "- MCP verify: " (mcp-verify-tool-line kind file project) "\n" + "- Write scope: " scope "\n\n" + "Anti-patterns:\n" + (bullet-lines (advisor-anti-patterns kind task anti-patterns-path anti-patterns-limit)) "\n\n" + "Failure loop:\n" + "- Paste the exact failed command output into jerboa_failure_advisor.\n" + "- Patch the smallest span it identifies.\n" + "- Re-run the same verifier before reporting success."))) (def (tool-cookbook-task-bundle args) (let* ([task (cookbook-task-text args)] + [kind (advisor-kind args)] [path (hash-get* args "cookbook_path" (data-path "cookbooks.sexp"))] [project (hash-get* args "project_path" #f)] [file (hash-get* args "file_path" #f)] [limit (positive-integer-arg args "max_recipes" 5)] + [anti-patterns-path (hash-get* args "anti_patterns_path" (data-path "anti-patterns.sexp"))] + [anti-patterns-limit (positive-integer-arg args "max_anti_patterns" 5)] [include-code? (hash-get* args "include_code" #f)] [recipes (load-cookbook path)]) (if (= (string-length task) 0) (text-result "task, query, or description is required." #t) (let* ([matches (bounded-take - (rank-recipes - (filter (lambda (r) (recipe-matches? r task)) recipes) - task) + (rank-matching-recipes recipes task kind) limit)] [entries (map (lambda (r) (recipe-bundle-entry r include-code?)) matches)]) (if (null? matches) @@ -1910,11 +1927,13 @@ (string-append "Cookbook task bundle\n" "Task: " task "\n\n" - "No matching cookbook recipes were found. Use jerboa_task_workflow_advisor, jerboa_stdlib_search, jerboa_module_exports, and jerboa_function_signature to derive the implementation, then save the discovered pattern with jerboa_howto_add.")) + "No matching cookbook recipes were found. Use jerboa_request_advisor, jerboa_stdlib_search, jerboa_module_exports, and jerboa_function_signature to derive the implementation, then save the discovered pattern with jerboa_howto_add.\n\n" + (cookbook-bundle-workflow task kind file project anti-patterns-path anti-patterns-limit))) (text-result (string-append "Cookbook task bundle\n" "Task: " task "\n" + "Kind: " kind "\n" (maybe-line "Project: " project) (maybe-line "File: " file) "Source: " path "\n" @@ -1922,7 +1941,7 @@ "Recipe context:\n" (string-join entries "\n\n") "\n\n" - (cookbook-bundle-workflow file project)))))))) + (cookbook-bundle-workflow task kind file project anti-patterns-path anti-patterns-limit)))))))) (def (line-comment-index line) (string-index line #\;)) @@ -4995,19 +5014,22 @@ (def model-workflow-baseline (string-append "Baseline for constrained local models:\n" - "1. Call jerboa_task_workflow_advisor with the user task.\n" + "1. Call jerboa_request_advisor or jerboa_task_workflow_advisor with the user task.\n" "2. Call jerboa_cookbook_task_bundle or jerboa_howto before writing Jerboa code.\n" "3. Confirm unknown APIs with jerboa_module_exports or jerboa_function_signature.\n" "4. Write the smallest complete .ss file or patch.\n" "5. Call jerboa_verify on changed .ss code, then jerboa_security_scan for file/process/FFI code.\n" "6. Run the exact command or focused tests. If it fails, call jerboa_explain_error or jerboa_error_fix_lookup before editing again.")) +(def (task-text-raw args) + (string-trim + (hash-get* args "task" + (hash-get* args "query" + (hash-get* args "description" ""))))) + (def (task-text args) (string-downcase - (string-trim - (hash-get* args "task" - (hash-get* args "query" - (hash-get* args "description" "")))))) + (task-text-raw args))) (def (task-kind-from-text text) (cond @@ -5020,11 +5042,451 @@ [(contains-any? text '("module" "library" "export" "import")) "module"] [else "code"])) +(def (file-extension-kind file) + (cond + [(not (and file (string? file))) #f] + [(or (string-suffix? ".md" file) (string-suffix? ".markdown" file)) "docs"] + [(or (string-suffix? ".c" file) (string-suffix? ".h" file) + (string-suffix? ".rs" file)) + "ffi"] + [(or (string-suffix? ".ss" file) (string-suffix? ".scm" file)) "script"] + [(string-suffix? ".sls" file) "module"] + [else #f])) + +(def (advisor-kind args) + (let* ([explicit (hash-get* args "kind" #f)] + [task (task-text args)] + [file (hash-get* args "file_path" #f)] + [text-kind (task-kind-from-text task)] + [file-kind (file-extension-kind file)]) + (cond + [(and explicit (string? explicit) (> (string-length explicit) 0)) + (string-downcase explicit)] + [(and file-kind (string=? text-kind "code")) file-kind] + [else text-kind]))) + +(def (advisor-file args fallback) + (let ([file (hash-get* args "file_path" #f)]) + (if (and file (string? file) (> (string-length file) 0)) file fallback))) + +(def (script-behavior-task? text) + (contains-any? + text + '("print" "output" "display" "show" "demo" "game of life" "conway" + "hello" "text based" "text-based" "cli" "command line" "command-line"))) + +(def (recipe-kind-query kind) + (cond + [(string=? kind "script") + "script cli command-line args arguments prelude display output main"] + [(string=? kind "debug-error") + "error fix unbound arity syntax balance verify"] + [(string=? kind "module") + "module import export compile project libdirs"] + [(string=? kind "docs") + "markdown docs doc verify fenced code examples"] + [(string=? kind "test") + "test scaffold run focused verify"] + [(string=? kind "ffi") + "ffi foreign-procedure c shim null safety type check"] + [(string=? kind "security") + "security scan unsafe file process shell ffi audit"] + [else + "prelude verify module exports function signature"])) + +(def (recipe-expanded-query task kind) + (string-trim + (string-append task " " (recipe-kind-query kind)))) + +(def (rank-matching-recipes recipes task kind) + (let* ([query (recipe-expanded-query task kind)] + [matches (filter (lambda (r) (recipe-matches? r query)) recipes)]) + (rank-recipes matches query))) + +(def (shell-verify-command kind task file project) + (let* ([scheme (recommended-scheme-path #f)] + [target (advisor-file (jhash "file_path" file) "main.ss")]) + (cond + [(string=? kind "docs") + "make check-docs"] + [(string=? kind "test") + (if project "make test" "scheme --script test.ss")] + [(string=? kind "module") + (if file + (string-append "jerbuild compile " file) + "make build")] + [(string=? kind "script") + (if (script-behavior-task? task) + (string-append scheme " --script " target) + (string-append scheme " --script " target))] + [(string=? kind "debug-error") + (if file + (string-append scheme " --script " file) + "make test")] + [else + (if file + (string-append scheme " --script " file) + "make build")]))) + +(def (mcp-verify-tool-line kind file project) + (cond + [(string=? kind "docs") + (if file + (string-append "jerboa_doc_verify {\"file_path\":\"" file "\"}") + "jerboa_doc_verify {\"content\":\"<markdown>\"}")] + [(string=? kind "module") + (if file + (string-append "jerboa_verify_changes {\"file_path\":\"" file "\"" + (if project (string-append ", \"project_path\":\"" project "\"") "") + "}") + "jerboa_verify_changes {\"project_path\":\"<project>\"}")] + [else + (if file + (string-append "jerboa_verify {\"file_path\":\"" file "\"" + (if project (string-append ", \"project_path\":\"" project "\"") "") + "}") + "jerboa_verify {\"code\":\"<code>\"}")])) + +(def (recommended-write-scope kind file) + (cond + [(and file (> (string-length file) 0)) file] + [(string=? kind "docs") "docs/"] + [(string=? kind "test") "tests/"] + [(string=? kind "module") "src/,lib/,tests/"] + [else "all"])) + +(def (advisor-first-actions kind file) + (cond + [(string=? kind "script") + (list + "Read or create one complete .ss script; avoid partial snippets." + "Use `(import (jerboa prelude))` unless a specific recipe says otherwise." + "Run verification immediately after the first complete version.")] + [(string=? kind "debug-error") + (list + "Classify the exact error with jerboa_explain_error." + "Look up known repairs with jerboa_error_fix_lookup." + "Patch the smallest failing span, then verify again.")] + [(string=? kind "module") + (list + "Inspect the existing module imports/exports before editing." + "Confirm unfamiliar calls with jerboa_function_signature." + "Prefer jerboa_verify_changes or the project build over ad hoc eval.")] + [(string=? kind "docs") + (list + "Verify fenced Scheme/Jerboa snippets with jerboa_doc_verify." + "Keep claims tied to existing files/modules." + "Do not mark status complete without running the doc verifier.")] + [else + (list + "Fetch a cookbook task bundle before writing code." + "Confirm unfamiliar APIs with module_exports/signature tools." + "Verify the smallest changed artifact before broader tests.")])) + +(def (load-anti-patterns path) + (load-json-file path '())) + +(def (anti-pattern-field entry key) + (hash-get* entry key "")) + +(def (anti-pattern-list-field entry key) + (let ([value (hash-get* entry key '())]) + (cond + [(list? value) (filter string? value)] + [(string? value) (list value)] + [else '()]))) + +(def (anti-pattern-applies-to-kind? entry kind) + (let ([kinds (anti-pattern-list-field entry "kinds")]) + (or (null? kinds) + (any (lambda (k) (or (string=? k "all") (string=? k kind))) kinds)))) + +(def (anti-pattern-score entry query kind) + (let* ([terms (search-terms query)] + [tags (anti-pattern-list-field entry "tags")] + [tools (anti-pattern-list-field entry "tools")] + [kind-score (if (anti-pattern-applies-to-kind? entry kind) 50 0)] + [literal (string-trim query)] + [literal-score (if (and (> (string-length literal) 0) + (or (string-ci-contains? (anti-pattern-field entry "id") literal) + (string-ci-contains? (anti-pattern-field entry "title") literal) + (string-ci-contains? (anti-pattern-field entry "avoid") literal) + (string-ci-contains? (anti-pattern-field entry "advice") literal) + (string-ci-contains? (anti-pattern-field entry "pattern") literal))) + 25 + 0)]) + (+ kind-score + literal-score + (sum-numbers + (map (lambda (term) + (+ (score-text (anti-pattern-field entry "id") term 18) + (score-text (anti-pattern-field entry "title") term 18) + (score-tags tags term 16) + (score-tags tools term 10) + (score-text (anti-pattern-field entry "avoid") term 8) + (score-text (anti-pattern-field entry "advice") term 8) + (score-text (anti-pattern-field entry "pattern") term 6))) + terms))))) + +(def (rank-anti-patterns entries query kind) + (sort entries + (lambda (a b) + (let ([as (anti-pattern-score a query kind)] + [bs (anti-pattern-score b query kind)]) + (cond + [(> as bs) #t] + [(< as bs) #f] + [else (string<? (anti-pattern-field a "id") + (anti-pattern-field b "id"))]))))) + +(def (anti-pattern-line entry) + (string-append + (anti-pattern-field entry "id") " - " (anti-pattern-field entry "title") + "\n avoid: " (anti-pattern-field entry "avoid") + "\n do: " (anti-pattern-field entry "advice") + "\n tools: " (string-join (anti-pattern-list-field entry "tools") ", "))) + +(def (fallback-anti-patterns kind) + (append + '("Do not guess imports or SRFI availability; confirm modules or use a recipe." + "Do not use Gerbil commands or assumptions for Jerboa tasks." + "Do not declare success before a verifier or focused command passes." + "Do not keep broad-reading after an error identifies a concrete span.") + (cond + [(string=? kind "script") + '("Do not write a `.sls` library for a simple user-facing script." + "Do not only define `main`; make the script execute at top level or call the entry point." + "Do not use a verify command that only loads the file when the task asks for visible output.")] + [(string=? kind "debug-error") + '("Do not repair unbalanced code with tiny `old_str` edits; replace the whole broken span." + "Do not ignore arity or unbound-variable errors; confirm the exact signature/export.")] + [(string=? kind "module") + '("Do not rewrite generated lib output when the source module is under src/." + "Do not assume direct eval matches the project build environment.")] + [else '()]))) + +(def (advisor-anti-patterns kind task . maybe-path-limit) + (let* ([path (if (and (pair? maybe-path-limit) (car maybe-path-limit)) + (car maybe-path-limit) + (data-path "anti-patterns.sexp"))] + [limit (if (and (pair? maybe-path-limit) + (pair? (cdr maybe-path-limit)) + (positive-integer? (cadr maybe-path-limit))) + (cadr maybe-path-limit) + 5)] + [entries (load-anti-patterns path)] + [matches (bounded-take + (filter (lambda (entry) + (or (anti-pattern-applies-to-kind? entry kind) + (> (anti-pattern-score entry task kind) 0))) + (rank-anti-patterns entries task kind)) + limit)]) + (if (null? matches) + (fallback-anti-patterns kind) + (map anti-pattern-line matches)))) + +(def (advisor-followup-tools kind) + (append + '("jerboa_cookbook_task_bundle" + "jerboa_howto" + "jerboa_module_exports" + "jerboa_function_signature" + "jerboa_verify") + (cond + [(string=? kind "debug-error") + '("jerboa_explain_error" "jerboa_error_fix_lookup" "jerboa_check_balance")] + [(string=? kind "docs") + '("jerboa_doc_verify" "jerboa_doc_status_audit")] + [(or (string=? kind "security") (string=? kind "ffi")) + '("jerboa_security_scan" "jerboa_ffi_type_check" "jerboa_resource_leak_check")] + [else '("jerboa_verify_changes")]))) + +(def (advisor-recipe-lines task kind path limit) + (let* ([recipes (load-cookbook path)] + [matches (bounded-take + (rank-matching-recipes recipes task kind) + limit)]) + (if (null? matches) + '("No direct cookbook match; call jerboa_howto with the main API/task term.") + (map recipe-summary-line matches)))) + +(def (bullet-lines xs) + (if (null? xs) + "- none" + (string-append "- " (string-join xs "\n- ")))) + +(def (advisor-header title task kind project file) + (string-append + title "\n" + "Task: " task "\n" + "Kind: " kind "\n" + (maybe-line "Project: " project) + (maybe-line "File: " file))) + (def (maybe-line label value) (if (and value (string? value) (> (string-length value) 0)) (string-append label value "\n") "")) +(def (tool-request-advisor args) + (let* ([task-raw (task-text-raw args)] + [task (string-downcase task-raw)] + [kind (advisor-kind args)] + [project (hash-get* args "project_path" #f)] + [file (hash-get* args "file_path" #f)] + [cookbook-path (hash-get* args "cookbook_path" (data-path "cookbooks.sexp"))] + [limit (positive-integer-arg args "max_recipes" 3)] + [anti-patterns-path (hash-get* args "anti_patterns_path" (data-path "anti-patterns.sexp"))] + [anti-patterns-limit (positive-integer-arg args "max_anti_patterns" 5)] + [verify (shell-verify-command kind task file project)] + [scope (recommended-write-scope kind file)]) + (if (= (string-length task-raw) 0) + (text-result "task, query, or description is required." #t) + (text-result + (string-append + (advisor-header "Jerboa request advisor" task-raw kind project file) + "Recommended jcode verify: " verify "\n" + "Recommended write scope: " scope "\n" + "MCP verify tool: " (mcp-verify-tool-line kind file project) "\n\n" + "First actions:\n" (bullet-lines (advisor-first-actions kind file)) "\n\n" + "Cookbook candidates:\n" (bullet-lines (advisor-recipe-lines task kind cookbook-path limit)) "\n\n" + "Anti-patterns:\n" (bullet-lines (advisor-anti-patterns kind task anti-patterns-path anti-patterns-limit)) "\n\n" + "Follow-up MCP tools:\n" (bullet-lines (advisor-followup-tools kind)) "\n\n" + "Stop condition: only report success after the recommended verify command or MCP verifier passes."))))) + +(def (tool-anti-pattern-lookup args) + (let* ([task-raw (task-text-raw args)] + [kind (advisor-kind args)] + [path (hash-get* args "anti_patterns_path" (data-path "anti-patterns.sexp"))] + [limit (positive-integer-arg args "max_results" 5)] + [query (if (> (string-length task-raw) 0) (string-downcase task-raw) kind)] + [entries (load-anti-patterns path)] + [matches (bounded-take + (filter (lambda (entry) + (or (anti-pattern-applies-to-kind? entry kind) + (> (anti-pattern-score entry query kind) 0))) + (rank-anti-patterns entries query kind)) + limit)]) + (if (and (= (string-length task-raw) 0) + (not (hash-get* args "kind" #f))) + (text-result "task, query, description, or kind is required." #t) + (text-result + (string-append + "Jerboa anti-pattern lookup\n" + (maybe-line "Task: " task-raw) + "Kind: " kind "\n" + "Source: " path "\n" + "Matches: " (number->string (length matches)) "\n\n" + (if (null? matches) + (bullet-lines (fallback-anti-patterns kind)) + (bullet-lines (map anti-pattern-line matches)))))))) + +(def (tool-verify-plan args) + (let* ([task-raw (task-text-raw args)] + [task (string-downcase task-raw)] + [kind (advisor-kind args)] + [project (hash-get* args "project_path" #f)] + [file (hash-get* args "file_path" #f)] + [verify (shell-verify-command kind task file project)] + [scope (recommended-write-scope kind file)]) + (if (= (string-length task-raw) 0) + (text-result "task, query, or description is required." #t) + (text-result + (string-append + (advisor-header "Jerboa verify plan" task-raw kind project file) + "Recommended shell verify: " verify "\n" + "Recommended jcode flags: --verify " (shell-quote verify) + " --write-scope " (shell-quote scope) "\n" + "MCP verify tool: " (mcp-verify-tool-line kind file project) "\n\n" + "Rationale:\n" + (cond + [(string=? kind "script") + "- Standalone scripts should be executed, not only syntax-checked.\n- If the task requires visible behavior, make the script print or assert that behavior.\n"] + [(string=? kind "module") + "- Project modules often depend on project libdirs/build layout; verify through jerboa_verify_changes or the focused build target.\n"] + [(string=? kind "docs") + "- Documentation changes need fenced-code verification and claim auditing, not a generic build.\n"] + [else + "- Use the smallest command that proves the requested behavior, then broaden to repo tests if needed.\n"]) + "Weak verify commands to avoid:\n" + "- `true`\n- commands that only check file existence\n- commands that load code but do not exercise requested output/behavior\n"))))) + +(def (failure-message args) + (string-trim + (hash-get* args "verify_output" + (hash-get* args "error_message" + (hash-get* args "message" ""))))) + +(def (failure-diagnosis lower) + (cond + [(contains-any? lower '("unexpected close" "unexpected )" "unexpected close parenthesis" + "unexpected end" "eof" "unmatched" "invalid syntax")) + (list "delimiter-or-syntax" + "Run jerboa_check_balance on the file or snippet, then replace the whole broken span with replace_range." + "replace_range" + '("jerboa_check_balance" "jerboa_explain_error"))] + [(contains-any? lower '("invalid context for definition" "definition in expression context")) + (list "definition-context" + "A previous form likely swallowed a top-level definition. Read the enclosing top-level span and replace the whole span." + "replace_range" + '("jerboa_check_balance" "jerboa_read_forms"))] + [(contains-any? lower '("wrong number of arguments" "arity" "incorrect number of arguments")) + (list "arity" + "Confirm the exact procedure signature, then patch the call site." + "line_edit-or-replace_def" + '("jerboa_function_signature" "jerboa_module_catalog"))] + [(contains-any? lower '("not bound" "unbound" "undefined variable" "unknown identifier")) + (list "unbound-identifier" + "Confirm the symbol exists and which module exports it; import the module or define the missing helper." + "edit-import-or-definition" + '("jerboa_suggest_imports" "jerboa_module_exports" "jerboa_symbol_exists_batch"))] + [(contains-any? lower '("library" "module" "not found" "cannot find" "import")) + (list "module-resolution" + "Check module spelling and project libdirs before editing code." + "fix-import-or-libdirs" + '("jerboa_module_exists" "jerboa_project_info" "jerboa_compile_check"))] + [(contains-any? lower '("export form outside" "outside of a module")) + (list "project-source-layout" + "Direct file verification may not match the project build layout. Use project_path/libdirs or the focused project build." + "verify_changes-or-build" + '("jerboa_verify_changes" "jerboa_make"))] + [else + (list "unknown" + "Classify the raw error and look up known fixes before another edit." + "explain-error-first" + '("jerboa_explain_error" "jerboa_error_fix_lookup"))])) + +(def (tool-failure-advisor args) + (let* ([message (failure-message args)] + [lower (string-downcase message)] + [file (hash-get* args "file_path" #f)] + [project (hash-get* args "project_path" #f)]) + (if (= (string-length message) 0) + (text-result "verify_output, error_message, or message is required." #t) + (let* ([diag (failure-diagnosis lower)] + [kind (list-ref diag 0)] + [advice (list-ref diag 1)] + [repair-tool (list-ref diag 2)] + [tools (list-ref diag 3)]) + (text-result + (string-append + "Jerboa failure advisor\n" + "Diagnosis: " kind "\n" + (maybe-line "Project: " project) + (maybe-line "File: " file) + "Recommended repair move: " repair-tool "\n" + "Advice: " advice "\n\n" + "Next MCP tools:\n" (bullet-lines tools) "\n\n" + "Anti-patterns:\n" + (bullet-lines + '("Do not keep broad-reading when the verifier names a concrete error." + "Do not make random paren pokes; replace the balanced/enclosing span." + "Do not declare success until the same verifier command passes.")) + "\n\n" + "Verifier excerpt:\n```\n" + (shorten message 1200) + "\n```")))))) + (def (workflow-tool-lines kind project file) (cond [(string=? kind "script") @@ -5111,7 +5573,11 @@ "\n\nStop condition: report success only after jerboa_verify and the smallest runnable command/test both pass.")))) (def manifest-core-tool-names - '("jerboa_task_workflow_advisor" + '("jerboa_request_advisor" + "jerboa_verify_plan" + "jerboa_failure_advisor" + "jerboa_anti_pattern_lookup" + "jerboa_task_workflow_advisor" "jerboa_compact_tool_manifest" "jerboa_cookbook_task_bundle" "jerboa_howto" @@ -5137,6 +5603,10 @@ (def (tool-usage-hint name) (cond + [(string=? name "jerboa_request_advisor") "preflight battle card for a task"] + [(string=? name "jerboa_verify_plan") "recommend verifier command and write scope"] + [(string=? name "jerboa_failure_advisor") "diagnose verifier failure and next repair move"] + [(string=? name "jerboa_anti_pattern_lookup") "task-specific mistakes to avoid"] [(string=? name "jerboa_task_workflow_advisor") "first call for a task-specific plan"] [(string=? name "jerboa_compact_tool_manifest") "small-context catalog and call order"] [(string=? name "jerboa_cookbook_task_bundle") "package relevant cookbook recipes for any task"] @@ -7267,10 +7737,59 @@ (list "project_path" (property "string" "Project path")) (list "cookbook_path" (property "string" "Cookbook path")) (list "max_recipes" (property "number" "Maximum selected recipes")) + (list "anti_patterns_path" (property "string" "Anti-pattern data path")) + (list "max_anti_patterns" (property "number" "Maximum anti-pattern entries")) (list "include_code" (property "boolean" "Include full recipe code examples"))) '()) tool-cookbook-task-bundle #t '("cookbook_task_bundle" "task_bundle" "recipe_bundle")) + (register-tool "jerboa_request_advisor" "Request Advisor" + "Return a compact preflight battle card with verify advice, write scope, cookbook candidates, and anti-patterns." + (schema (list (list "task" (property "string" "User task or goal")) + (list "query" (property "string" "Alias for task")) + (list "description" (property "string" "Alias for task")) + (list "project_path" (property "string" "Project path")) + (list "file_path" (property "string" "Relevant file path")) + (list "cookbook_path" (property "string" "Cookbook path")) + (list "max_recipes" (property "number" "Maximum selected recipes")) + (list "anti_patterns_path" (property "string" "Anti-pattern data path")) + (list "max_anti_patterns" (property "number" "Maximum anti-pattern entries"))) + '()) + tool-request-advisor #t + '("request_advisor" "advisor" "preflight_advisor")) + (register-tool "jerboa_verify_plan" "Verify Plan" + "Recommend a verifier command, jcode verified flags, and MCP verifier for a task." + (schema (list (list "task" (property "string" "User task or goal")) + (list "query" (property "string" "Alias for task")) + (list "description" (property "string" "Alias for task")) + (list "project_path" (property "string" "Project path")) + (list "file_path" (property "string" "Relevant file path"))) + '()) + tool-verify-plan #t + '("verify_plan" "verification_plan")) + (register-tool "jerboa_failure_advisor" "Failure Advisor" + "Map verifier, compiler, or test output to the next Jerboa repair action." + (schema (list (list "verify_output" (property "string" "Verifier output")) + (list "error_message" (property "string" "Error message alias")) + (list "message" (property "string" "Message alias")) + (list "project_path" (property "string" "Project path")) + (list "file_path" (property "string" "Relevant file path")) + (list "snippet" (property "string" "Relevant source snippet"))) + '()) + tool-failure-advisor #t + '("failure_advisor" "error_advisor" "repair_advisor")) + (register-tool "jerboa_anti_pattern_lookup" "Anti-Pattern Lookup" + "Return task-specific local-model mistakes to avoid, with repair guidance and follow-up tools." + (schema (list (list "task" (property "string" "User task or goal")) + (list "query" (property "string" "Alias for task")) + (list "description" (property "string" "Alias for task")) + (list "kind" (property "string" "Task kind override")) + (list "file_path" (property "string" "Relevant file path")) + (list "anti_patterns_path" (property "string" "Anti-pattern data path")) + (list "max_results" (property "number" "Maximum matching entries"))) + '()) + tool-anti-pattern-lookup #t + '("anti_pattern_lookup" "anti_patterns" "antipatterns")) (register-tool "jerboa_security_scan" "Security Scanner" "Static security scanner for Jerboa, C, and Rust code." (schema (list (list "file_path" (property "string" "Single file")) --- a/mcp/test/protocol-test.ss +++ b/mcp/test/protocol-test.ss @@ -599,9 +599,32 @@ (cons "limit" 1)))) (call-tool 118 "jerboa_eval" (alist->hash-table - (list (cons "expression" "(exit 0)"))))))) + (list (cons "expression" "(exit 0)")))) + (call-tool 119 "jerboa_request_advisor" + (alist->hash-table + (list (cons "task" "write a text Conway's Game of Life in Jerboa") + (cons "file_path" "life.ss") + (cons "max_recipes" 2)))) + (call-tool 120 "jerboa_verify_plan" + (alist->hash-table + (list (cons "task" "write a command line script") + (cons "file_path" "tool.ss")))) + (call-tool 121 "jerboa_failure_advisor" + (alist->hash-table + (list (cons "verify_output" "Exception in read: unexpected close parenthesis at line 2") + (cons "file_path" "life.ss")))) + (call-tool 122 "jerboa_cookbook_task_bundle" + (alist->hash-table + (list (cons "task" "hash cookbook recipe") + (cons "file_path" "hash-demo.ss") + (cons "max_recipes" 1)))) + (call-tool 123 "jerboa_anti_pattern_lookup" + (alist->hash-table + (list (cons "task" "write a command line script in Jerboa") + (cons "file_path" "tool.ss") + (cons "max_results" 3))))))) -(check "expected 118 responses" (= (length responses) 118)) +(check "expected 123 responses" (= (length responses) 123)) (def init-result (result (car responses))) (check "initialize server name" @@ -619,6 +642,10 @@