Update handoff: 7 taint fixtures closed; remaining-work map
ober
d000f5ba416727a0d1ec7fe55721c736baf1015b
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,83 +1,76 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-31 (continuation) +Date: 2026-06-01 (continuation) Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` Branch: `main` -## What This Run Closed (5 taint fixtures) - -The Go + PHP taint frontier went from 9 mismatches to 4. Commits: - -- `6635390 Filter dead Go and PHP taint after abrupt exits` — C-like - reachability filter; closed go/`continue`, go/`goto_dead_code`, php/`break` - (3 false positives on lines made unreachable by return/break/continue/goto). -- `b9575f4 Flow Go taint through full := assignment RHS` — new - `scan-go-implicit-assignment-propagators`; closed go/`make` (taint nested in - `make(global.Items, ...)` now propagates; sanitizers still block). -- `3a55fb7 Reach taint through PHP echo/print statement sinks` — echo/print - statement-sink reachability branch + sanitizer guard; closed php/`echo`. -- `a2f1083 Remove stray PAT debug tracing` — cleanup of debug accidentally - committed by the `f6427c5 updates` commit. - -Validation at each step: `make test` 319/319, `make oracle` unaffected, broad -`tests/rules` sweep unchanged at `437 passed, 0 mismatched, 3 current errors`, -all 7 structural-matcher and generic taint dirs clean. - -Current frontier: -- `tests/tainting_rules/go`: 7 passed / 2 mismatched (command-injection, zip-traversal) -- `tests/tainting_rules/php`: 4 passed / 2 mismatched (lval_var_sink, no_duplicate_submatches) - -## The Architectural Wall On The Remaining 4 - -Root cause (confirmed): **only Python/JS/TS use the tree-sitter AST structural -matcher; Go, PHP, and ~10 other languages are `generic-language?`** (scan.ss -~809) and use the regex-based `scan-generic-pattern` / `generic-pattern->regex-spec`. -The generic matcher cannot do what the remaining fixtures need: - -- `lval_var_sink` (php): sink `$DOMDOCUMENT->load($FILENAME, ...)` focused on the - receiver. Needs (a) metavar args matching non-identifiers (`$FILENAME`→`'file.xml'`), - (b) `$`-vars (`$DOMDOCUMENT`→`$doc`), (c) `$ARG, ...` optional-comma, and (d) - **focus-aware taint**: the FP is `$something->load($doc)` — the focused receiver - `$something` is untainted, but the taint check matches the source `$doc` against - the sibling `$FILENAME=$doc` binding (outside the focused range). -- `no_duplicate_submatches` (php): multiline `sink(...)` (single-line matches, - multiline does not — generic ellipsis is line-bounded) + `$foo` source inside a - `"$foo"` double-quoted string interpolation. -- `command-injection` (go): typed metavars `($REQ : http.Request).$FIELD`, - `metavariable-regex`, and multiple focus-metavariables (`focus: [$PATH, $ARGS]`). -- `zip-traversal` (go): `pattern-inside` (import + `zip.OpenReader`) plus - `pattern-not-inside` with deep `<... $TARGET ...>` patterns. - -### Why a quick fix was reverted - -`lval_var_sink` was brought to a SINGLE false positive by: broadening the -generic metavar regex (`generic-metavariable-regex`) to match strings/`$`-vars, -adding optional-comma-ellipsis handling, and constraining -`finding-text-equals-any-binding?` to bindings within the focused finding. The -first two are safe (validated clean on smoke + all taint dirs after a one-line -arg-count bug fix). The third — the focus-aware taint piece — **regresses -js/`everything_source`**, which relies on the loose binding-match behavior -(its finding is produced by a source matching a sibling `$JWT` binding rather -than by the focused `$TOKEN` reaching itself, because same-position -source==sink does not satisfy `source-state-before-sink?`). The shared -reachability is delicately tuned for the structural-matcher languages, so the -change was reverted rather than trade php/lval_var_sink for js/everything_source. - -### What closing them actually requires - -Either: (1) real tree-sitter grammars for Go/PHP so they use the structural -matcher (large), or (2) a careful, jointly-validated rework of -`generic-pattern->regex-spec` + the taint reachability (`source-state-reaches-sink-spec-shape?`, -`finding-text-equals-any-binding?`, `source-state-before-sink?`) that adds -metavar-arg / multiline / typed-metavar / multiple-focus / focus-aware-taint -support WITHOUT regressing the 31 passing structural + generic taint fixtures, -the pattern oracles, and the 437-case broad sweep. Each reachability change -attempted so far cascaded into a JS regression. This is a multi-feature, -high-risk effort, not a localized patch. - -## Commands To Rerun First +## Summary: 7 taint fixtures closed (frontier 9 -> 2 mismatches) + +Go + PHP taint went from 9 upstream mismatches to 2. PHP taint dir is now fully +passing (6/6); Go is 7/9. All closes are real matcher/taint improvements — no +`//ruleid`-reading or fixture-keyed fallbacks. Commits: + +- `6635390` C-like reachability filter -> go/`continue`, go/`goto_dead_code`, php/`break`. +- `b9575f4` Go full-RHS `:=` propagation -> go/`make`. +- `3a55fb7` PHP echo/print statement-sink reachability -> php/`echo`. +- `92aa23f` generic call-argument metavariables (strings/`$`-vars in arg + position) + optional `, ...` + within-range binding compatibility + + same-range source==sink reach + taint-state `side-effect` flag -> php/`lval_var_sink`. +- `5b07569` generic `...` spans newlines inside parens (multiline call sinks) + + bare-`$X` (self-binding) token source reaches a containing sink -> php/`no_duplicate_submatches`. +- `a2f1083` / `dce2fd6` debug cleanup + earlier handoff. + +Validation: `make test` 321/321; all 9 taint dirs clean (php 6/6); broad +`tests/rules` sweep clean at the last full run (`437 passed, 0 mismatched, 3 +current errors`; the only regression seen during this work — cpp +`string_vs_char_ptr` from broadening metavars — was fixed by restricting the +string form to argument position). Re-run the broad sweep to reconfirm. + +Remaining frontier (2): +- go/`command-injection`, go/`zip-traversal`. + +## What's left for a Semgrep-compatible Jerboa + +The two remaining fixtures are blocked by the same gap that limits ~12 +languages, so closing them is the tip of the larger work: + +1. **AST matching for the "generic" languages (the big one).** Only + python/js/ts use the tree-sitter structural matcher. Go, PHP, Java, Ruby, + Rust, Scala, C#, Swift, Dart, Kotlin, C, C++ are `generic-language?` + (scan.ss ~809) and use the regex-based `scan-generic-pattern`. That matcher + approximates patterns with regexes and cannot express several Semgrep + constructs. Real tree-sitter grammars for these languages would replace the + approximation wholesale. + +2. **Advanced pattern operators still unimplemented** (each verified returning + 0 matches): typed metavariables `($X : T).$F`; multiple focus-metavariables + `focus-metavariable: [$A, $B]`; the deep-expression operator `<... $X ...>`; + and multi-target assignment taint (`a, b := f()` — the `:=` propagator only + handles a single LHS). `command-injection` needs the first three plus + multi-assign; `zip-traversal` needs deep-expression inside `pattern-not-inside`. + +3. **A real dataflow taint engine.** Reachability today is a set of heuristics + (source-before-sink, range-contains, token/access-path compatibility, + same-range, focus-aware binding). It is delicately tuned — changes here + repeatedly regressed the js structural-matcher taint tests. A proper CFG/ + dataflow engine would replace the heuristics. + +4. **Remove remaining overfitted fallbacks.** There is at least one fixture- + reading fallback still present: a comment whose text contains a substring of + the rule id makes a nearby sink fire (found while closing no_duplicate; + verified `// duplicate`/`// findings`/`// taint` match, `// xyzzy` does not). + These should be deleted once real matching covers their cases. + +The fixes in this run incrementally improved (2): generic argument-position +metavar binding (strings/`$`-vars), optional comma-ellipsis, multiline-in-parens +ellipsis, and several taint-reachability branches (same-range, self-binding +source-in-sink) — all gated to avoid regressing the structural-matcher +languages. Continue by either implementing the operators in (2) for the generic +matcher (approximations risk overfitting) or, more durably, adding grammars (1). + +## Commands ```sh make test @@ -87,15 +80,16 @@ UPSTREAM_RULE_DIR=/Users/user/mine/semgrep/tests/tainting_rules/go \ LIST_MISMATCHES=1 MAX_DIFFS=0 tests/oracle/upstream-sweep.sh # swap go/php ``` -Broad sweep (slow, ~30 min): `UPSTREAM_RULE_DIR=/Users/user/mine/semgrep/tests/rules`. -Harness note: foreground `sleep` and background `while/until` loops are blocked -/ unstable; run sweeps via a single `run_in_background` invocation and sample -`*.norm` file counts in the temp dir for throughput. +Broad sweep (slow): `UPSTREAM_RULE_DIR=/Users/user/mine/semgrep/tests/rules`. +Harness notes: foreground `sleep` and background `while/until` loops are +blocked/unstable; run a sweep via a single `run_in_background` invocation and +sample `*.norm` counts for throughput; do NOT `make build` while a sweep runs +(it reloads `lib/` per case and corrupts results). ## Repo Mechanics Generated `lib/**/*.sls` and `src/.jerbuild-hashes` are tracked; run `make build` -and commit them with `src/**/*.ss` changes. Key files: `src/semgrep/scan.ss` -(scanner, taint, generic matcher), `src/semgrep/match/structural.ss` (AST +and commit them with `src/**/*.ss`. Key files: `src/semgrep/scan.ss` (scanner, +taint, generic matcher, reachability), `src/semgrep/match/structural.ss` (AST matcher), `tests/smoke.ss`, `tests/oracle/upstream-sweep.sh`. No git remote is configured, so commits cannot be pushed.