Advance lambda and source taint parity
ober
19f45f712a97f21d449108cd5e91c7b84cd0af25
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,6 +1,6 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-28 23:30 MDT local workspace time +Date: 2026-05-28 23:57 MDT local workspace time Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` @@ -31,6 +31,7 @@ taint/dataflow, path and target semantics, autofix, and output schemas. Recent checkpoints before this handoff commit: ```text +7cfb751 Clear lambda taint label reach 31db293 Clear taint label branch frontier 34227c0 Clear final global taint frontier 268a616 Clear field-sensitive taint frontier @@ -72,7 +73,7 @@ make test Result: ```text -189 tests, 189 passed, 0 failed +191 tests, 191 passed, 0 failed ``` Local oracle: @@ -87,25 +88,21 @@ Result: oracle: 42 passed, 0 failed ``` -Focused lambda and nearby taint guardrail for this checkpoint: +Focused lambda, source-focus, and nearby taint guardrail for this checkpoint: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(taint_lambda2|taint_lambda4|taint_typestate|taint_side_effect|taint_source)$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(taint_lambda2|taint_lambda4|taint_match_on_source|taint_typestate|taint_side_effect|taint_source)$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh ``` Result: ```text -MISMATCH taint_lambda4 -@@ -1,2 +0,0 @@ --(finding "test" "/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts" 3 8 55 3 26 73 "ERROR" "Test" "") --(finding "test" "/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts" 9 10 156 9 28 174 "ERROR" "Test" "") -upstream-sweep: 2 passed, 1 mismatched, 0 jerboa errors, 0 current errors, 3 compared +upstream-sweep: 4 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 4 compared ``` -This confirms `taint_lambda2`, `taint_typestate`, `taint_side_effect`, and -`taint_source` are clean in the current tree. `taint_lambda4` is the next -frontier. +This confirms `taint_lambda2`, `taint_lambda4`, `taint_match_on_source`, +`taint_typestate`, `taint_side_effect`, and `taint_source` are clean in the +current tree. Focused upstream guardrail containing the field-sensitive cases fixed so far: @@ -152,7 +149,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep MAX_CASES=220 LIST_MISMATCHES=1 M Result: ```text -upstream-sweep: 199 passed, 19 mismatched, 0 jerboa errors, 2 current errors, 220 compared +upstream-sweep: 201 passed, 17 mismatched, 0 jerboa errors, 2 current errors, 220 compared ``` The full 241-case sweep was not rerun after this checkpoint. Do not treat the @@ -175,27 +172,29 @@ errors. ## What Changed In This Checkpoint -This checkpoint clears `taint_lambda2`, building on `31db293` which cleared -`taint_labels2` and `taint_labels3`. +This checkpoint clears `taint_lambda4` and `taint_match_on_source`, building +on `7cfb751` which cleared `taint_lambda2`. Implementation changes in `src/semgrep/scan.ss`: -- `source-state-reaches-sink-spec-shape?` now lets an ordinary non-token - access-path source reach a wider containing sink when - `source-access-path-compatible-with-sink?` says the source and sink paths are - compatible. -- This specifically covers a source like `req.query` inside the sink argument - `req.query.id` at `db.sink(req.query.id)`. -- The branch is intentionally restricted to non-token source states. A broader - first attempt regressed same-call typestate and side-effect guardrails such - as `lock(l)` and `f.close()`, where token source self-hits must not report. -- The new reachability path still requires physical range containment and - access-path compatibility. It does not make arbitrary nested source - expressions taint arbitrary enclosing sinks. +- JS/TS `pattern-inside: function ... ($REQ) {...}` now has a targeted + fallback that recognizes ordinary function declarations and arrow functions + with typed parameters such as `req: Request`. +- JS/TS positive pattern matching now handles the expression source shape + `$X.$Y. ... + foobar` against member-chain additions such as + `req.query + foobar`, preserving `$X` and `$Y` bindings for later focus and + source `requires`. +- Taint states now preserve their original source finding through propagation. + This lets `options: taint_focus_on: source` report the source match even when + the sink is reached through assignments. +- Taint output selection now uses that source origin only when the rule option + explicitly requests source focus; ordinary taint rules still report sinks. New smoke coverage in `tests/smoke.ss`: ```text +scan TypeScript taint combines typed lambda source labels +scan taint focus on source reports original source scan JavaScript taint combines lambda db and request labels scan JavaScript taint focused header value under fetch scan taint labels keep sibling if else assignments separate @@ -215,6 +214,72 @@ subtrees. The ancestor-clean test mirrors `taint_field_sensitive4`: ## Resolved Recent Cases +`taint_lambda4` is now cleared: + +```sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_lambda4$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh +``` + +Current result: + +```text +upstream-sweep: 1 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 1 compared +``` + +What changed: + +- Target file: + `/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts`. +- Shape: + `pattern-inside: function ... ($REQ) {...}` plus `pattern: $REQ.query`, + and a second source requiring `__SOURCE__` with + `pattern: $X.$Y. ... + foobar` focused to `$X`. +- Jerboa previously found the sink and simple `$REQ.query` source, but not the + two full source formulas. +- The new JS/TS inside fallback validates that the focused `$REQ` occurrence is + inside a function or arrow body whose parameter list contains the same + identifier, including typed params such as `req: Request`. +- The new member-chain-plus fallback matches `req.query + foobar`, binds + `$X=req` and `$Y=query`, then focus narrows the source to `req` so the + `EDGECASE` label can combine with `__SOURCE__`. + +Expected matched lines for this case are now exactly: + +```text +3, 9 +``` + +`taint_match_on_source` is now cleared: + +```sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_match_on_source$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh +``` + +Current result: + +```text +upstream-sweep: 1 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 1 compared +``` + +What changed: + +- Target file: + `/Users/user/mine/semgrep/tests/rules/taint_match_on_source.py`. +- Rule option: `taint_focus_on: source`. +- Jerboa already detected the tainted flow but reported the sink at line 8. + Packaged Semgrep reports the original `source()` call at line 3. +- Taint states now keep an `origin` finding. Propagated states inherit the + origin from the source state that produced them. +- Sink output now checks `taint_focus_on: source`; when enabled and the sink's + label requirements are satisfied, it returns the origin source findings + instead of the sink finding. + +Expected matched line for this case is now exactly: + +```text +3 +``` + `taint_lambda2` is now cleared: ```sh @@ -562,11 +627,9 @@ What changed: ## Current First-220 Frontier -The current first-220 sweep has these 19 mismatches: +The current first-220 sweep has these 17 mismatches: ```text -taint_lambda4 -taint_match_on_source taint_nested_record_pattern taint_no_builtin_props taint_no_builtin_props1 @@ -603,21 +666,20 @@ vardef_assign_true1 vardef_assign_true2 ``` -## Immediate Next Case: taint_lambda4 +## Immediate Next Case: taint_nested_record_pattern Focused command: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_lambda4$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_nested_record_pattern$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh ``` Current result: ```text -MISMATCH taint_lambda4 -@@ -1,2 +0,0 @@ --(finding "test" "/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts" 3 8 55 3 26 73 "ERROR" "Test" "") --(finding "test" "/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts" 9 10 156 9 28 174 "ERROR" "Test" "") +MISMATCH taint_nested_record_pattern +@@ -1 +0,0 @@ +-(finding "test" "/Users/user/mine/semgrep/tests/rules/taint_nested_record_pattern.js" 4 3 76 4 14 87 "ERROR" "Test" "") upstream-sweep: 0 passed, 1 mismatched, 0 jerboa errors, 0 current errors, 1 compared ``` @@ -627,112 +689,57 @@ Rule: rules: - id: test message: Test + mode: taint + pattern-sources: + - pattern: tainted + pattern-sinks: + - pattern: sink(...) severity: ERROR languages: - javascript - typescript - mode: taint - options: - interfile: true - pattern-sources: - - pattern-either: - - patterns: - - pattern-either: - - pattern-inside: function ... ($REQ) {...} - - pattern: $REQ.query - - focus-metavariable: $REQ - - label: EDGECASE - patterns: - - patterns: - - pattern: $X.$Y. ... + foobar - - focus-metavariable: $X - requires: __SOURCE__ - pattern-sinks: - - patterns: - - pattern: sink($URL) - - focus-metavariable: $URL - requires: __SOURCE__ and EDGECASE ``` Target: ```text -/Users/user/mine/semgrep/tests/rules/taint_lambda4.ts +/Users/user/mine/semgrep/tests/rules/taint_nested_record_pattern.js ``` Relevant target: -```typescript -function test1(req: Request) { - sink(req.query + foobar); -}; - -function test2() { - return (req: Request) => { - sink(req.query + foobar); - }; -}; +```javascript +function test() { + const { body: { param } } = tainted + sink(param) +} ``` Packaged Semgrep expects: ```text -3, 9 +4 ``` Jerboa currently reports no findings: ```diff -@@ -1,2 +0,0 @@ --(finding "test" ".../taint_lambda4.ts" 3 8 ... "ERROR" "Test" "") --(finding "test" ".../taint_lambda4.ts" 9 10 ... "ERROR" "Test" "") +@@ -1 +0,0 @@ +-(finding "test" ".../taint_nested_record_pattern.js" 4 3 ... "ERROR" "Test" "") ``` -Isolation already done: - -- A sink-only rule with `pattern: sink($URL)` and `focus-metavariable: $URL` - finds both sinks, binding `$URL` to `req.query + foobar`. -- A simple source-only rule with `pattern: $REQ.query` finds both occurrences - and binds `$REQ` to `req`. -- The first real source formula returns no findings: - - ```yaml - patterns: - - pattern-either: - - pattern-inside: function ... ($REQ) {...} - - pattern: $REQ.query - - focus-metavariable: $REQ - ``` - -- The second source formula also returns no findings: - - ```yaml - patterns: - - patterns: - - pattern: $X.$Y. ... + foobar - - focus-metavariable: $X - requires: __SOURCE__ - ``` - Interpretation for the next fix: -- The first source formula probably needs `pattern-inside: function ... ($REQ) - {...}` to recognize TypeScript typed parameters in both ordinary function - declarations and arrow functions. It must bind or validate `$REQ=req` for - `function test1(req: Request) { ... }` and `return (req: Request) => { ... }`. -- The second source formula needs structural support for the JS/TS expression - pattern `$X.$Y. ... + foobar` against `req.query + foobar`, with - `focus-metavariable: $X` narrowing the source to `req`. -- `requires: __SOURCE__` on the `EDGECASE` source depends on the first source - being available at the focused `$X=req`. Do not treat `EDGECASE` as an - unconditional label. -- Start inspection around `src/semgrep/scan.ss` pattern clause handling, - especially `pattern-inside`, `inside-clause-apply`, JS sequence/object - helpers, and any JS recursive function-inside helpers near the clause - application code. The taint label plumbing appears to be working once source - specs produce the expected focused sources. -- Add a smoke test for the `taint_lambda4` shape before changing behavior, then - rerun the focused lambda guardrail above plus `make test` and `make oracle`. +- This is likely missing JavaScript nested object destructuring propagation: + `const { body: { param } } = tainted` should taint `param`. +- Existing JavaScript destructuring support already handles simpler object + destructuring shapes; inspect `scan-javascript-destructuring` and implicit + assignment propagator construction around `src/semgrep/scan.ss`. +- Add a smoke test that uses `const { body: { param } } = tainted` with source + `tainted` and sink `sink(...)`, then make the destructuring alias collector + descend through nested object patterns. +- Guard against over-tainting sibling destructured names; the source is the RHS + object, and the propagated binding should be the nested leaf identifier. ## Resolved Earlier Frontier --- a/lib/semgrep/scan.sls +++ b/lib/semgrep/scan.sls @@ -4227,6 +4227,12 @@ source (cdr entry))) (and (javascript-like-language? language) + (scan-javascript-member-ellipsis-plus-pattern + rule + path + source + (cdr entry))) + (and (javascript-like-language? language) (scan-javascript-new-function-sequence-pattern rule path @@ -4411,6 +4417,8 @@ source (cdr entry))) (and (javascript-like-language? language) + (scan-javascript-member-ellipsis-plus-pattern-with-bindings rule path source (cdr entry) initial-bindings)) + (and (javascript-like-language? language) (scan-javascript-new-function-sequence-pattern rule path @@ -5130,6 +5138,300 @@ (if (or (not best-start) (> start best-start)) (loop (cdr xs) (car xs) start) (loop (cdr xs) best best-start)))])))) + (def (js-function-param-inside-metavariable entry) + (and (eq? (car entry) 'pattern) + (let* ([pattern (cdr entry)] + [match (re-search + (re "\\([ \t\n]*\\$([A-Za-z_][A-Za-z0-9_]*)") + pattern + 0)]) + (and match + (string-find-substring pattern "function") + (string-find-substring pattern "...") + (string-find-substring pattern "{") + (re-match-group match 1))))) + (def (js-identifier-token-char? ch) + (or (char-alphabetic? ch) + (char-numeric? ch) + (char=? ch #\_) + (char=? ch #\$))) + (def (js-token-start-boundary? source index) + (or (= index 0) + (not (js-identifier-token-char? + (string-ref source (- index 1)))))) + (def (js-token-end-boundary? source index) + (or (= index (string-length source)) + (not (js-identifier-token-char? + (string-ref source index))))) + (def (js-parameter-name-binding-in-range source start end + metavariable name) + (let ([name-len (string-length name)]) + (let loop ([offset start]) + (let ([index (string-find-substring-from + source + name + offset)]) + (and index + (< index end) + (let ([name-end (+ index name-len)]) + (if (and (<= name-end end) + (js-token-start-boundary? source index) + (js-token-end-boundary? source name-end)) + (make-regex-capture-binding metavariable name + source index name-end) + (loop (+ index 1))))))))) + (def (js-first-parameter-binding-in-range + source + start + end + metavariable) + (let loop ([i start]) + (cond + [(>= i end) #f] + [(or (char-alphabetic? (string-ref source i)) + (char=? (string-ref source i) #\_) + (char=? (string-ref source i) #\$)) + (let name-loop ([j (+ i 1)]) + (if (and (< j end) + (js-identifier-token-char? (string-ref source j))) + (name-loop (+ j 1)) + (make-regex-capture-binding metavariable + (substring source i j) source i j)))] + [else (loop (+ i 1))]))) + (def (js-parameter-binding-in-range source start end + metavariable desired) + (if desired + (js-parameter-name-binding-in-range source start end + metavariable desired) + (js-first-parameter-binding-in-range + source + start + end + metavariable))) + (def (js-function-param-binding-for-context source function-index candidate-start metavariable desired) + (and (js-token-start-boundary? source function-index) + (js-token-end-boundary? + source + (+ function-index (string-length "function"))) + (let* ([param-open (string-find-substring-from + source + "(" + function-index)] + [body-open (string-find-substring-from + source + "{" + function-index)] + [param-close (and param-open + (find-matching-close-paren + source + param-open))] + [body-close (and body-open + (find-matching-close-brace + source + body-open))]) + (and param-open + param-close + body-open + body-close + (< param-open body-open) + (<= param-close body-open) + (< body-open candidate-start) + (< candidate-start body-close) + (js-parameter-binding-in-range source (+ param-open 1) (- param-close 1) metavariable + desired))))) + (def (find-matching-open-paren source close-index) + (let loop ([i close-index] [depth 0]) + (cond + [(< i 0) #f] + [(char=? (string-ref source i) #\)) + (loop (- i 1) (+ depth 1))] + [(char=? (string-ref source i) #\() + (if (= depth 1) i (loop (- i 1) (max 0 (- depth 1))))] + [else (loop (- i 1) depth)]))) + (def (js-arrow-param-range-before source arrow-index) + (let ([end-index (skip-horizontal-backward + source + (- arrow-index 1))]) + (and (>= end-index 0) + (if (char=? (string-ref source end-index) #\)) + (let ([open-index (find-matching-open-paren + source + end-index)]) + (and open-index (cons (+ open-index 1) end-index))) + (let ([start-index (let loop ([i end-index]) + (if (and (>= i 0) + (js-identifier-token-char? + (string-ref source i))) + (loop (- i 1)) + (+ i 1)))]) + (and (<= start-index end-index) + (cons start-index (+ end-index 1)))))))) + (def (js-arrow-param-binding-for-context source arrow-index candidate-start metavariable desired) + (let* ([body-open (skip-whitespace + source + (+ arrow-index 2))] + [body-close (and (< body-open (string-length source)) + (char=? (string-ref source body-open) #\{) + (find-matching-close-brace + source + body-open))] + [param-range (js-arrow-param-range-before + source + arrow-index)]) + (and body-close + param-range + (< body-open candidate-start) + (< candidate-start body-close) + (js-parameter-binding-in-range source (car param-range) + (cdr param-range) metavariable desired)))) + (def (js-function-param-inside-binding + source + candidate + metavariable) + (let* ([existing (finding-metavariable-binding + candidate + (string-append "$" metavariable))] + [desired (and existing + (metavariable-binding-text existing))] + [candidate-start (finding-start-offset candidate)]) + (or (let loop ([offset 0]) + (let ([index (string-find-substring-from + source + "function" + offset)]) + (and index + (< index candidate-start) + (or (js-function-param-binding-for-context source index candidate-start metavariable + desired) + (loop (+ index 1)))))) + (let loop ([offset 0]) + (let ([index (string-find-substring-from + source + "=>" + offset)]) + (and index + (< index candidate-start) + (or (js-arrow-param-binding-for-context source index candidate-start metavariable + desired) + (loop (+ index 2))))))))) + (def (js-function-param-inside-apply + rule + candidate + entry + source) + (let ([metavariable (js-function-param-inside-metavariable + entry)]) + (and metavariable + (let ([binding (js-function-param-inside-binding + source + candidate + metavariable)]) + (and binding + (let ([merged (merge-binding-list + (finding-metavars candidate) + (list (cons metavariable binding)))]) + (and merged + (finding-with-bindings + rule + candidate + merged + source)))))))) + (def (js-member-ellipsis-plus-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [match (re-search + (re "^\\$([A-Za-z_][A-Za-z0-9_]*)[ \t]*\\.[ \t]*\\$([A-Za-z_][A-Za-z0-9_]*)[ \t]*\\.[ \t]*\\.\\.\\.[ \t]*\\+[ \t]*([A-Za-z_$][A-Za-z0-9_$]*)$") + trimmed + 0)]) + (and match + (list + (cons 'base-metavariable (re-match-group match 1)) + (cons 'property-metavariable (re-match-group match 2)) + (cons 'right (re-match-group match 3)))))) + (def (js-member-ellipsis-plus-target-regex right) + (re (string-append + "([A-Za-z_$][A-Za-z0-9_$]*)\\.([A-Za-z_$][A-Za-z0-9_$]*)(?:\\.[A-Za-z_$][A-Za-z0-9_$]*)*[ \t\n]*\\+[ \t\n]*" + (regex-escape-string right)))) + (def (js-member-ellipsis-plus-bindings + source + match + spec + initial-bindings) + (let* ([full (re-match-full match)] + [start (re-match-start match)] + [base (re-match-group match 1)] + [property (re-match-group match 2)] + [base-metavariable (alist-ref/default + spec + 'base-metavariable + #f)] + [property-metavariable (alist-ref/default + spec + 'property-metavariable + #f)] + [base-rel (or (string-find-substring-from full base 0) 0)] + [property-rel (or (string-find-substring-from + full + property + (+ base-rel (string-length base))) + 0)] + [base-start (+ start base-rel)] + [base-end (+ base-start (string-length base))] + [property-start (+ start property-rel)] + [property-end (+ property-start (string-length property))]) + (merge-binding-list + initial-bindings + (list + (cons + base-metavariable + (make-regex-capture-binding base-metavariable base source + base-start base-end)) + (cons + property-metavariable + (make-regex-capture-binding property-metavariable property + source property-start property-end)))))) + (def (scan-javascript-member-ellipsis-plus-pattern-with-bindings rule path source pattern initial-bindings) + (let ([spec (js-member-ellipsis-plus-pattern-spec pattern)]) + (and spec + (let* ([right (alist-ref/default spec 'right "")] + [rx (js-member-ellipsis-plus-target-regex right)] + [right-len (string-length right)] + [len (string-length source)]) + (let loop ([offset 0] [acc '()]) + (if (> offset len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source offset)]) + (if match + (let* ([start (re-match-start match)] + [end (re-match-end match)] + [right-start (- end right-len)] + [bindings (and (js-token-start-boundary? + source + start) + (js-token-start-boundary? + source + right-start) + (js-token-end-boundary? + source + end) + (js-member-ellipsis-plus-bindings + source + match + spec + initial-bindings))] + [finding (and bindings + (finding-for-range-with-bindings rule path source start + end bindings))] + [next (max (+ start 1) end)]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + (def (scan-javascript-member-ellipsis-plus-pattern + rule + path + source + pattern) + (scan-javascript-member-ellipsis-plus-pattern-with-bindings rule path source pattern '())) (def (js-nearest-recursive-context-start source offset) (let loop ([xs (js-recursive-contexts-before source offset)] [best #f]) @@ -7440,6 +7742,12 @@ (cdr clause) source)) (and (javascript-like-language? language) + (js-function-param-inside-apply + rule + candidate + (cdr clause) + source)) + (and (javascript-like-language? language) (js-recursive-function-inside-apply rule candidate @@ -7737,11 +8045,16 @@ target-root) (scan-patterns-clauses rule (rule-pattern rule) language path source target-root)) - (def (taint-state finding labels exact? token? contained? - control?) + (def (taint-state* finding labels exact? token? contained? + control? origin) (list (cons 'finding finding) (cons 'labels labels) (cons 'exact exact?) (cons 'token token?) - (cons 'contained contained?) (cons 'control control?))) + (cons 'contained contained?) (cons 'control control?) + (cons 'origin origin))) + (def (taint-state finding labels exact? token? contained? + control?) + (taint-state* finding labels exact? token? contained? + control? finding)) (def (taint-state-finding state) (alist-ref/default state 'finding #f)) (def (taint-state-labels state) @@ -7754,11 +8067,16 @@ (and (alist-ref/default state 'contained #f) #t)) (def (taint-state-control? state) (and (alist-ref/default state 'control #f) #t)) + (def (taint-state-origin state) + (alist-ref/default + state + 'origin + (taint-state-finding state))) (def (taint-state-with-labels state labels) - (taint-state (taint-state-finding state) labels + (taint-state* (taint-state-finding state) labels (taint-state-exact? state) (taint-state-token? state) - (taint-state-contained? state) - (taint-state-control? state))) + (taint-state-contained? state) (taint-state-control? state) + (taint-state-origin state))) (def (taint-by-side-effect? value) (cond [(not value) #f] @@ -8437,6 +8755,10 @@ (rule-option-enabled? rule "taint_assume_safe_indexes")) (def (taint-assume-safe-functions? rule) (rule-option-enabled? rule "taint_assume_safe_functions")) + (def (taint-focus-on-source? rule) + (let ([value (rule-option-enabled? rule "taint_focus_on")]) + (or (eq? value 'source) + (and (string? value) (string=? value "source"))))) (def (text-contains-comparison? text) (or (string-find-substring text "==") (string-find-substring text "!=") @@ -9257,6 +9579,24 @@ sanitizers assignment-kills source-text) (source-state-reaches-sink-spec-shape? rule source-state sink-spec sanitizers assignment-kills source-text)) + (def (source-states-reaching-sink rule sink-spec sources sanitizers assignment-kills + source-text) + (let loop ([remaining sources] [acc '()]) + (if (null? remaining) + (reverse acc) + (loop + (cdr remaining) + (if (source-reaches-sink? rule (car remaining) sink-spec + sanitizers assignment-kills source-text) + (cons (car remaining) acc) + acc))))) + (def (label-set-for-source-states source-states) + (let loop ([remaining source-states] [acc '()]) + (if (null? remaining) + acc + (loop + (cdr remaining) + (add-labels (taint-state-labels (car remaining)) acc))))) (def (tainted-sink? rule sink-spec sources sanitizers assignment-kills source-text) (let* ([requires (alist-ref/default sink-spec 'requires #f)] @@ -9266,6 +9606,20 @@ (requires-satisfied? labels requires)))) (def (sink-finding sink-spec) (alist-ref/default sink-spec 'finding #f)) + (def (sink-output-findings rule sink-spec sources sanitizers + assignment-kills source-text) + (let* ([requires (alist-ref/default sink-spec 'requires #f)] + [reaching (source-states-reaching-sink rule sink-spec sources sanitizers + assignment-kills source-text)] + [labels (label-set-for-source-states reaching)]) + (if (and (not (null? labels)) + (requires-satisfied? labels requires)) + (if (taint-focus-on-source? rule) + (sg-filter + (lambda (finding) finding) + (map taint-state-origin reaching)) + (list (sink-finding sink-spec))) + '()))) (def (finding-text finding source) (source-slice source @@ -9497,9 +9851,10 @@ replace-names) propagated-bindings)]) (and merged - (taint-state (finding-with-bindings rule focused merged source) + (taint-state* (finding-with-bindings rule focused merged source) (propagator-output-labels source-state propagator) #f - side-effect? (not side-effect?) #f)))) + side-effect? (not side-effect?) #f + (taint-state-origin source-state))))) (def (new-required-sources source-matches known source) (let loop ([matches source-matches] [acc '()]) (if (null? matches) @@ -9628,10 +9983,10 @@ rule source sinks0)]) - (map sink-finding - (filter - (lambda (sink) - (tainted-sink? rule sink sources sanitizers + (apply + append + (map (lambda (sink) + (sink-output-findings rule sink sources sanitizers implicit-propagators source)) sinks)))) (def (scan-rule rule language path source target-root) --- a/src/.jerbuild-hashes +++ b/src/.jerbuild-hashes @@ -3,7 +3,7 @@ ("src/semgrep/output/json.ss" . "293881CFA2ADB7BC") ("src/semgrep/lang.ss" . "7E5441BD00A7F1D4") ("src/semgrep/parse/parse-target.ss" . "E74854DDDACF6BA") - ("src/semgrep/scan.ss" . "AAD636B00C0E8C45") + ("src/semgrep/scan.ss" . "998D1E472CAB5367") ("src/semgrep/output/text.ss" . "BE476CB84B807FBA") ("src/semgrep/fix.ss" . "2E5B65B1FEF3B2B1") ("src/semgrep/schema/lang.ss" . "CAE2CA859C9A9FD0") --- a/src/semgrep/scan.ss +++ b/src/semgrep/scan.ss @@ -4398,6 +4398,12 @@ source (cdr entry))) (and (javascript-like-language? language) + (scan-javascript-member-ellipsis-plus-pattern + rule + path + source + (cdr entry))) + (and (javascript-like-language? language) (scan-javascript-new-function-sequence-pattern rule path @@ -4633,6 +4639,13 @@ source (cdr entry))) (and (javascript-like-language? language) + (scan-javascript-member-ellipsis-plus-pattern-with-bindings + rule + path + source + (cdr entry) + initial-bindings)) + (and (javascript-like-language? language) (scan-javascript-new-function-sequence-pattern rule path @@ -5426,6 +5439,327 @@ (loop (cdr xs) (car xs) start) (loop (cdr xs) best best-start)))])))) +(def (js-function-param-inside-metavariable entry) + (and (eq? (car entry) 'pattern) + (let* ([pattern (cdr entry)] + [match (re-search + (re "\\([ \t\n]*\\$([A-Za-z_][A-Za-z0-9_]*)") + pattern + 0)]) + (and match + (string-find-substring pattern "function") + (string-find-substring pattern "...") + (string-find-substring pattern "{") + (re-match-group match 1))))) + +(def (js-identifier-token-char? ch) + (or (char-alphabetic? ch) + (char-numeric? ch) + (char=? ch #\_) + (char=? ch #\$))) + +(def (js-token-start-boundary? source index) + (or (= index 0) + (not (js-identifier-token-char? (string-ref source (- index 1)))))) + +(def (js-token-end-boundary? source index) + (or (= index (string-length source)) + (not (js-identifier-token-char? (string-ref source index))))) + +(def (js-parameter-name-binding-in-range source start end metavariable name) + (let ([name-len (string-length name)]) + (let loop ([offset start]) + (let ([index (string-find-substring-from source name offset)]) + (and index + (< index end) + (let ([name-end (+ index name-len)]) + (if (and (<= name-end end) + (js-token-start-boundary? source index) + (js-token-end-boundary? source name-end)) + (make-regex-capture-binding + metavariable + name + source + index + name-end) + (loop (+ index 1))))))))) + +(def (js-first-parameter-binding-in-range source start end metavariable) + (let loop ([i start]) + (cond + [(>= i end) #f] + [(or (char-alphabetic? (string-ref source i)) + (char=? (string-ref source i) #\_) + (char=? (string-ref source i) #\$)) + (let name-loop ([j (+ i 1)]) + (if (and (< j end) + (js-identifier-token-char? (string-ref source j))) + (name-loop (+ j 1)) + (make-regex-capture-binding + metavariable + (substring source i j) + source + i + j)))] + [else (loop (+ i 1))]))) + +(def (js-parameter-binding-in-range source start end metavariable desired) + (if desired + (js-parameter-name-binding-in-range + source + start + end + metavariable + desired) + (js-first-parameter-binding-in-range source start end metavariable))) + +(def (js-function-param-binding-for-context + source + function-index + candidate-start + metavariable + desired) + (and (js-token-start-boundary? source function-index) + (js-token-end-boundary? + source + (+ function-index (string-length "function"))) + (let* ([param-open + (string-find-substring-from source "(" function-index)] + [body-open + (string-find-substring-from source "{" function-index)] + [param-close + (and param-open + (find-matching-close-paren source param-open))] + [body-close + (and body-open + (find-matching-close-brace source body-open))]) + (and param-open + param-close + body-open + body-close + (< param-open body-open) + (<= param-close body-open) + (< body-open candidate-start) + (< candidate-start body-close) + (js-parameter-binding-in-range + source + (+ param-open 1) + (- param-close 1) + metavariable + desired))))) + +(def (find-matching-open-paren source close-index) + (let loop ([i close-index] [depth 0]) + (cond + [(< i 0) #f] + [(char=? (string-ref source i) #\)) + (loop (- i 1) (+ depth 1))] + [(char=? (string-ref source i) #\() + (if (= depth 1) + i + (loop (- i 1) (max 0 (- depth 1))))] + [else (loop (- i 1) depth)]))) + +(def (js-arrow-param-range-before source arrow-index) + (let ([end-index (skip-horizontal-backward source (- arrow-index 1))]) + (and (>= end-index 0) + (if (char=? (string-ref source end-index) #\)) + (let ([open-index + (find-matching-open-paren source end-index)]) + (and open-index + (cons (+ open-index 1) end-index))) + (let ([start-index + (let loop ([i end-index]) + (if (and (>= i 0) + (js-identifier-token-char? + (string-ref source i)))