Update handoff: PHP migrated onto structural matcher, validated 437/0
ober
af653d3beddc549d4b0a949068566bd76170a2da
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -4,9 +4,42 @@ Date: 2026-06-01 (continuation) Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` -Branch: `go-structural-migration` (off `main`) - -## Summary: ALL nine taint dirs pass (0 mismatched) — Go on a real AST matcher +Branch: `php-structural-migration` (off `main`); Go migration already merged. + +## Summary: Go AND PHP now run on the real tree-sitter structural matcher + +Both Go and PHP have been migrated off the regex generic-matcher approximation +onto the real tree-sitter structural matcher; `generic-language?` (scan.ss ~809) +no longer lists `go` or `php`. tree-sitter-go and tree-sitter-php (ABI 15) are +vendored + compiled into `jerboa-treesitter` (committed + pushed). All nine +taint dirs are 0-mismatched (go 9/9, php 6/6, python 12/12, js 11/11, +ruby/dart/java/scala/ts clean); `make test` 321/321. + +### PHP migration (this branch) + +PHP routes through the structural matcher; `$X` is recognised at the AST level +(uppercase `variable_name`) since `$` is PHP's variable sigil, rather than the +text-rewrite used for go/py/js. Three structural/taint gaps the migration +exposed were closed with real, general features (commits on this branch): + +- `assume_safe_numbers` now also treats numeric casts `(int)/(float)/…` as safe + (the structural taint engine traces `$i = (int) source(); sink($i)`, which the + regex path silently under-matched). — taint_assume_safe_numbers2 +- a metavariable-pattern's `pattern-not` re-parses the bound text as a target; + PHP fragments now get a `<?php …;` prelude so the sub-pattern can match. — + metavar_pattern_fake_toks1 +- `$FUNC(...)` matches include/require/include_once/require_once (PHP language + constructs that parse as `*_expression`, not `function_call_expression`), + binding `$FUNC` to the keyword token. — metavar_regex_include + +Validated (authoritative normalize-based comparison): broad `tests/rules` +sweep `437 passed, 0 mismatched, 3 current errors, 440 compared`; PHP 29/29 and +Go 16/16 fixtures clean; `make test` 321/321. Merged to `main`. (Note: the raw +start-line comparison is unreliable — jerboa emits intermediate taint trace-step +findings that `normalize-findings.ss` collapses to the sink, so always validate +through the normalize pipeline, not a bare line grep.) + +## Earlier: Go on a real AST matcher (merged to main) The two last frontier fixtures (go/`command-injection`, go/`zip-traversal`) are CLOSED. Every taint dir is now 0-mismatched: go 9/9, php 6/6, python 12/12, js @@ -70,9 +103,9 @@ Remaining frontier (2): The two remaining fixtures are blocked by the same gap that limits ~12 languages, so closing them is the tip of the larger work: -1. **AST matching for the "generic" languages (the big one).** Only - python/js/ts use the tree-sitter structural matcher. Go, PHP, Java, Ruby, - Rust, Scala, C#, Swift, Dart, Kotlin, C, C++ are `generic-language?` +1. **AST matching for the "generic" languages (the big one).** + python/js/ts/go/php now use the tree-sitter structural matcher. Java, Ruby, + Rust, Scala, C#, Swift, Dart, Kotlin, C, C++ remain `generic-language?` (scan.ss ~809) and use the regex-based `scan-generic-pattern`. That matcher approximates patterns with regexes and cannot express several Semgrep constructs. Real tree-sitter grammars for these languages would replace the