Clear parameter source taint frontier
ober
efdf40637ceae2f0b03b05531d1f66762d9568ad
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,10 +1,10 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-29 01:22 MDT +Date: 2026-05-29 01:42 MDT Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` -Current pre-handoff HEAD: `a1201fb Clear typed request taint frontier` +Current pre-handoff HEAD: `370470d Clear default-parameter taint frontier` The user wants this project carried forward until the pure Jerboa port reaches Semgrep parity. Continue from this state. Do not restart broad discovery from @@ -52,7 +52,7 @@ make test Result: ```text -200 tests, 200 passed, 0 failed +203 tests, 203 passed, 0 failed ``` Local oracle: @@ -67,30 +67,33 @@ Result: oracle: 42 passed, 0 failed ``` -Focused default-parameter family: +Focused parameter-source family: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_param_default[0-9]*$' LIST_MISMATCHES=1 MAX_DIFFS=320 tests/oracle/upstream-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_param_source[123]$' LIST_MISMATCHES=1 MAX_DIFFS=5 tests/oracle/upstream-sweep.sh ``` Result: ```text -upstream-sweep: 4 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 4 compared +upstream-sweep: 3 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 3 compared ``` -Focused spread-record-small guardrail: +Nearby taint guardrail: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_spread_record_small$' LIST_MISMATCHES=1 MAX_DIFFS=220 tests/oracle/upstream-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(taint_param_source1|taint_param_source2|taint_param_source3|taint_typestate|taint_spread_record_small|taint_spread_record_big)$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh ``` Result: ```text -upstream-sweep: 1 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 1 compared +upstream-sweep: 5 passed, 1 mismatched, 0 jerboa errors, 0 current errors, 6 compared ``` +The remaining mismatch in that guardrail is the known frontier +`taint_spread_record_big`. + Full upstream sweep: ```sh @@ -100,7 +103,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep LIST_MISMATCHES=1 MAX_DIFFS=0 tes Result: ```text -upstream-sweep: 223 passed, 16 mismatched, 0 jerboa errors, 2 current errors, 241 compared +upstream-sweep: 226 passed, 13 mismatched, 0 jerboa errors, 2 current errors, 241 compared ``` The two current errors are packaged-Semgrep oracle errors, not Jerboa scanner @@ -109,119 +112,99 @@ the command above intentionally ran the full sorted 241-case sweep. ## What Changed In This Checkpoint -This checkpoint clears the upstream `taint_param_default*` family and also -clears `taint_spread_record_small`, reducing the full upstream frontier from: +This checkpoint clears the upstream `taint_param_source1`, +`taint_param_source2`, and `taint_param_source3` cases, reducing the full +upstream frontier from: ```text -218 passed, 21 mismatched, 0 jerboa errors, 2 current errors +223 passed, 16 mismatched, 0 jerboa errors, 2 current errors ``` to: ```text -223 passed, 16 mismatched, 0 jerboa errors, 2 current errors +226 passed, 13 mismatched, 0 jerboa errors, 2 current errors ``` Implementation changes in `src/semgrep/scan.ss`: -- `scan-symbolic-propagation-pattern` now declines broad structural patterns: - bare metavariables such as `$URL` and string wildcard literals such as - `"..."` or `'...'`. -- This matters because `symbolic_propagation: true` was hijacking - `pattern-inside: "..."` in `taint_param_default.yaml`, returning one - propagated variable instead of all string literals. The source formula then - lost the four `http://` literals Semgrep reports. -- Focused `fix-regex` findings now drop the `fix` extra when - `focus-metavariable` changes the reported range. Current Semgrep preserves - template `fix:` under focus, but omits `fix-regex`; `taint_param_default` - exposed this after the ranges were fixed. -- Built-in Python implicit assignment propagation now includes: - - ```text - def $F(..., $L = $R, ...): - ... - ``` - - This lets taint flow from default parameter values into the function body. -- Built-in JavaScript/TypeScript implicit assignment propagation now includes: +- Built-in JavaScript/TypeScript implicit assignment propagation now includes + `"$L += $R"`. This lets `buf += chunk` propagate taint in + `taint_param_source3`. +- Pattern seed selection now avoids a `pattern-either` made only of + `pattern-inside` clauses when another positive clause exists. This is needed + for source formulas shaped like: - ```text - function $F(..., $L = $R, ...) { - ... - } + ```yaml + patterns: + - pattern-either: + - pattern-inside: function ... ($REQ, $RES) {...} + - pattern-inside: function ... ($REQ, $RES, $NEXT) {...} + - patterns: + - pattern: $REQ.on('data', function ($CHUNK) { ... }) + - focus-metavariable: $CHUNK ``` - This clears JS default-parameter propagation in `taint_param_default3`. -- Token taint can now reach a containing sink when the source is an unfocused - bare source with no metavariables and the sink syntactically contains it. - This clears `def test(x=sink(taint))` in `taint_param_default1` and also - the spread-record-small shape `sink({...source})`. -- That token-in-sink allowance is intentionally narrow. It requires - `(null? (finding-metavars source))`, so focused by-side-effect typestate - sources such as `lock($L)` do not report their own source call as a sink. + Seeding on the inside-only `pattern-either` produced broad containing + candidates and prevented the later focused callback source from surviving. + Seeding on the concrete nested `patterns` clause lets the `pattern-inside` + alternatives act as filters. +- Assignment kills now include an exact textual LHS/source match. This helps + focused parameter sources such as `function bar({user_input}) { ... }` + clear when `user_input = 1` appears before `sink(user_input)`. +- Function scope detection now recognizes top-level JavaScript/TypeScript + `function ...` declarations as simple scopes. This prevents a focused source + in sibling function `foo` from leaking into sibling function `bar`, while + leaving nested callbacks inside the same top-level function under the same + broad scope. New smoke coverage in `tests/smoke.ss`: ```text -scan Python taint bare source inside default sink -scan Python taint default parameter reaches session request -scan JavaScript taint default parameter propagation -focus-metavariable omits fix-regex +scan JavaScript taint augmented assignment propagation +scan JavaScript taint source under either pattern-inside +scan JavaScript taint focused param reassignment clears source ``` -These add to the existing taint and symbolic propagation guardrails from prior -checkpoints. - ## Cleared Cases The following cases are clean in this checkpoint: ```text -taint_param_default -taint_param_default1 -taint_param_default2 -taint_param_default3 -taint_spread_record_small +taint_param_source1 +taint_param_source2 +taint_param_source3 ``` Relevant upstream files: ```text -/Users/user/mine/semgrep/tests/rules/taint_param_default.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_default.py -/Users/user/mine/semgrep/tests/rules/taint_param_default1.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_default1.py -/Users/user/mine/semgrep/tests/rules/taint_param_default2.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_default2.py -/Users/user/mine/semgrep/tests/rules/taint_param_default3.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_default3.js -/Users/user/mine/semgrep/tests/rules/taint_spread_record_small.yaml -/Users/user/mine/semgrep/tests/rules/taint_spread_record_small.js +/Users/user/mine/semgrep/tests/rules/taint_param_source1.yaml +/Users/user/mine/semgrep/tests/rules/taint_param_source1.js +/Users/user/mine/semgrep/tests/rules/taint_param_source2.yaml +/Users/user/mine/semgrep/tests/rules/taint_param_source2.ts +/Users/user/mine/semgrep/tests/rules/taint_param_source3.yaml +/Users/user/mine/semgrep/tests/rules/taint_param_source3.js ``` -Important isolation finding from this checkpoint: +Important isolation findings: -- On `taint_param_default.py`, the sink side already matched. The source side - failed only with `symbolic_propagation: true`. -- With `symbolic_propagation` enabled, `pattern-inside: "..."` returned a - single propagated `url` candidate instead of the twelve string literal - candidates in the target file. -- After broad-pattern symbolic fallback was disabled, the exact source formula - found the four non-localhost HTTP literals and the taint rule reported the - same four ranges as Semgrep. -- The last mismatch was `fix-regex` metadata. Semgrep emitted no fix for - focused `fix-regex`; Jerboa initially kept replacements such as - `session.get("https://example.com")`. +- `taint_param_source1` and `taint_param_source2` were false positives. The + focused parameter source in `foo({user_input})` correctly tainted `x`, but a + sibling source in `bar({user_input})` should be killed by `user_input = 1`. + The fix required both exact reassignment kill and sibling top-level function + scope separation. +- `taint_param_source3` was a false negative. `req.on('data', function + (chunk) { ... })` matched and `buf += chunk` matched after adding `+=`, but + the full source formula returned zero until inside-only `pattern-either` + stopped being chosen as the seed. ## Current Full-Sweep Frontier The current full sorted upstream sweep compared 241 rule/target pairs and has -these 16 mismatches: +these 13 mismatches: ```text -taint_param_source1 -taint_param_source2 -taint_param_source3 taint_propagator2 taint_react taint_safe_comparisons @@ -243,152 +226,79 @@ Use this command to refresh the full list: SEMGREP_CURRENT=/Users/user/.local/bin/semgrep LIST_MISMATCHES=1 MAX_DIFFS=0 tests/oracle/upstream-sweep.sh ``` -## Recommended Next Target: taint_param_source* +## Recommended Next Target: taint_propagator2 -The next sorted frontier is the `taint_param_source1/2/3` cluster. +The next sorted frontier is `taint_propagator2`. Focused command: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_param_source[123]$' LIST_MISMATCHES=1 MAX_DIFFS=3 tests/oracle/upstream-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^taint_propagator2$' LIST_MISMATCHES=1 MAX_DIFFS=1 tests/oracle/upstream-sweep.sh ``` Current result: ```text -MISMATCH taint_param_source1 -@@ -1 +1,2 @@ -+(finding "tainting" ".../taint_param_source1.js" 10 3 146 10 19 162 "ERROR" "This confirms taint mode works.\n" "") - (finding "tainting" ".../taint_param_source1.js" 4 3 69 4 10 76 "ERROR" "This confirms taint mode works.\n" "") - -MISMATCH taint_param_source2 -@@ -1 +1,2 @@ -+(finding "tainting" ".../taint_param_source2.ts" 10 3 158 10 19 174 "ERROR" "This confirms taint mode works.\n" "") - (finding "tainting" ".../taint_param_source2.ts" 4 3 75 4 10 82 "ERROR" "This confirms taint mode works.\n" "") - -MISMATCH taint_param_source3 +MISMATCH taint_propagator2 @@ -1 +0,0 @@ --(finding "test" ".../taint_param_source3.js" 8 9 195 8 18 204 "ERROR" "Test" "") - -upstream-sweep: 0 passed, 3 mismatched, 0 jerboa errors, 0 current errors, 3 compared +-(finding "test" ".../taint_propagator2.py" 7 10 95 7 11 96 "INFO" "Test" "") +upstream-sweep: 0 passed, 1 mismatched, 0 jerboa errors, 0 current errors, 1 compared ``` -`taint_param_source1` and `taint_param_source2` are false positives. - -Files: - -```text -/Users/user/mine/semgrep/tests/rules/taint_param_source1.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_source1.js -/Users/user/mine/semgrep/tests/rules/taint_param_source2.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_source2.ts -``` - -Rule shape: - -```yaml -pattern-sources: - - patterns: - - pattern: function $FUNC({$PARAM}) { ... } - - focus-metavariable: $PARAM -pattern-sinks: - - pattern: sink(...) -``` - -TypeScript variant: - -```yaml -pattern-sources: - - patterns: - - pattern: function $FUNC({$PARAM} : $TYPE) { ... } - - focus-metavariable: $PARAM -``` - -Targets: - -```javascript -function foo({user_input}) { - x = user_input; - // ruleid - sink(x); -} - -function bar({user_input}) { - user_input = 1; - // ok - sink(user_input); -} -``` - -Expected Semgrep behavior: - -- Report `sink(x)` in `foo`. -- Do not report `sink(user_input)` in `bar` after `user_input = 1`. - -Likely fix area: - -- Assignment kills for focused parameter sources. The scanner currently keeps - the focused `$PARAM` taint alive after a direct assignment to the same - parameter name. -- Start around `taint-assignment-kills-source?`, - `taint-source-killed-before-sink?`, and access-path extraction helpers near - the `source-state-reaches-sink-spec-shape?` path. -- Be careful with by-side-effect sources and field sensitivity; run - `make test` after any kill-change because this area is regression-prone. - -`taint_param_source3` is a false negative. - -Files: - -```text -/Users/user/mine/semgrep/tests/rules/taint_param_source3.yaml -/Users/user/mine/semgrep/tests/rules/taint_param_source3.js -``` - -Rule shape: +Rule: ```yaml -pattern-sources: - - patterns: - - pattern-either: - - pattern-inside: function ... ($REQ, $RES) {...} - - pattern-inside: function ... ($REQ, $RES, $NEXT) {...} - - patterns: - - pattern: $REQ.on('data', function ($CHUNK) { ... }) - - focus-metavariable: $CHUNK -pattern-sinks: - - pattern: sink(...) +rules: + - id: test + message: Test + severity: INFO + languages: [py] + mode: taint + pattern-sources: + - pattern: user_input + pattern-propagators: + - pattern: | + $Y.f($X) + ... + $Y.g($Z) + from: $X + to: $Z + pattern-sinks: + - patterns: + - pattern: sink($SINK) + - focus-metavariable: $SINK ``` Target: -```javascript -function test() { - var foobar = function(req, res, next) { - var buf = ''; - req.on('data', function (chunk) { - buf += chunk - }); - // ruleid - sink(buf); - }; -} +```python +def test(): + x = user_input + y.f(x) + x = "safe" + y.g(z) + # ruleid: test + sink(z) ``` Expected Semgrep behavior: -- `chunk` from the `data` callback taints `buf`. -- `buf += chunk` propagates taint to `buf`. -- `sink(buf)` is reported. +- `user_input` taints `x`. +- The custom propagator spans `y.f(x) ... y.g(z)` and should propagate taint + from `$X` to `$Z`. +- The later `x = "safe"` must not block this propagator because the propagator + itself already matched a sequence from the tainted `$X` occurrence to `$Z`. Likely fix area: -- JavaScript augmented assignment propagation. Current implicit assignment - patterns include `"$L = $R"` and declaration forms, but not `"$L += $R"`. -- Adding broad compound assignment propagation may affect clearing behavior and - arithmetic safe-value options, so start with a narrow smoke around - `buf += chunk` and run the existing assignment, typestate, and JS taint smoke - tests. +- Start around `source-taints-binding?`, `propagator-applies*`, and + `taint-source-killed-before-finding?`. +- The current assignment-kill logic likely treats `x = "safe"` between the + source and `$Z` as killing the source before propagation. Semgrep appears to + evaluate the propagator sequence match as a dataflow edge from the `$X` + occurrence in `y.f(x)`, not from the original `user_input` occurrence. +- A focused smoke should mirror this exact shape before changing shared + propagator logic. ## Useful Commands @@ -435,19 +345,21 @@ SCM - Prefer narrow smoke tests that reproduce an upstream shape before changing shared taint logic. -- `scan-symbolic-propagation-pattern` is ordered before structural fallback in - several paths. If a symbolic case starts returning surprising matches, check - whether it is claiming a broad pattern that should be structural. +- `scan-symbolic-propagation-pattern` declines broad structural patterns such + as bare metavariables and string literal ellipses. Recheck this if symbolic + rules unexpectedly stop matching. - `focus-metavariable` preserves finding extras by default. Current Semgrep omits `fix-regex` under focus but keeps template `fix:`. - `finding-focused-on-binding` is also used by taint propagator helpers. Avoid global behavior changes there unless the propagation call sites are checked. -- `same-simple-function-scope?` is Python-oriented. JS taint function scope is - still approximate and can leak if broad source/sink rules are loosened. -- After touching assignment-kill or source reachability logic, run at minimum: +- JavaScript function scope is deliberately conservative: only top-level + `function ...` lines are recognized as simple scopes. This avoids sibling + leaks without breaking nested callback taint cases already covered by smoke + tests. +- After touching propagator or assignment-kill logic, run at minimum: ```sh make test - SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(taint_param_source1|taint_param_source2|taint_param_source3|taint_typestate|taint_spread_record_small|taint_spread_record_big)$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh + SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(taint_propagator2|taint_param_source1|taint_param_source2|taint_param_source3|taint_typestate|taint_spread_record_big)$' LIST_MISMATCHES=1 MAX_DIFFS=260 tests/oracle/upstream-sweep.sh SEMGREP_CURRENT=/Users/user/.local/bin/semgrep make oracle ``` --- a/lib/semgrep/scan.sls +++ b/lib/semgrep/scan.sls @@ -4569,6 +4569,22 @@ [else #f])) (def (pattern-inside-clause? entry) (eq? (car entry) 'pattern-inside)) + (def (pattern-either-inside-only-entry? entry) + (and (eq? (car entry) 'pattern-either) + (not (null? (cdr entry))) + (let loop ([xs (cdr entry)]) + (or (null? xs) + (and (pattern-inside-clause? (car xs)) + (loop (cdr xs))))))) + (def (seed-positive-clauses positive-clauses) + (let ([non-inside-either (sg-filter + (lambda (entry) + (not (pattern-either-inside-only-entry? + entry))) + positive-clauses)]) + (if (null? non-inside-either) + positive-clauses + non-inside-either))) (def (preferred-seed-positive-entry? entry) (case (car entry) [(pattern-regex pattern pattern-as) #t] @@ -8471,6 +8487,8 @@ (let* ([positive-clauses (sg-filter positive-pattern-entry? clauses)] + [seedable-positive-clauses (seed-positive-clauses + positive-clauses)] [preferred-positive-clauses (sg-filter preferred-seed-positive-entry? clauses)]) @@ -8480,8 +8498,8 @@ [seed-entry (cond [(not (null? preferred-positive-clauses)) (car preferred-positive-clauses)] - [(not (null? positive-clauses)) - (car positive-clauses)] + [(not (null? seedable-positive-clauses)) + (car seedable-positive-clauses)] [(not (null? inside-seeds)) (car inside-seeds)] [else #f])]) @@ -8574,6 +8592,8 @@ (let* ([positive-clauses (sg-filter positive-pattern-entry? clauses)] + [seedable-positive-clauses (seed-positive-clauses + positive-clauses)] [preferred-positive-clauses (sg-filter preferred-seed-positive-entry? clauses)]) @@ -8583,8 +8603,8 @@ [seed-entry (cond [(not (null? preferred-positive-clauses)) (car preferred-positive-clauses)] - [(not (null? positive-clauses)) - (car positive-clauses)] + [(not (null? seedable-positive-clauses)) + (car seedable-positive-clauses)] [(not (null? inside-seeds)) (car inside-seeds)] [else #f])]) @@ -9097,10 +9117,11 @@ (string=? language "ts") (string=? language "tsx")) '("$L = $R" "$L[$I] = $R" "const $L = $R" "let $L = $R" "var $L = $R" - "const $L: $T = $R" "let $L: $T = $R" "var $L: $T = $R" - "const { $L } = $R" "let { $L } = $R" "var { $L } = $R" - "const { $FIELD: $L } = $R" "let { $FIELD: $L } = $R" - "var { $FIELD: $L } = $R" "const { $FIELD: { $L } } = $R" + "$L += $R" "const $L: $T = $R" "let $L: $T = $R" + "var $L: $T = $R" "const { $L } = $R" "let { $L } = $R" + "var { $L } = $R" "const { $FIELD: $L } = $R" + "let { $FIELD: $L } = $R" "var { $FIELD: $L } = $R" + "const { $FIELD: { $L } } = $R" "let { $FIELD: { $L } } = $R" "var { $FIELD: { $L } } = $R" "function $F(..., $L = $R, ...) {\n ...\n}")] [(or (string=? language "python") @@ -9307,12 +9328,18 @@ (let ([trimmed (string-trim line)]) (or (sg-string-prefix? "def " trimmed) (sg-string-prefix? "async def " trimmed)))) + (def (simple-top-level-js-function-line? line) + (and (= (line-indent line) 0) + (sg-string-prefix? "function " (string-trim line)))) + (def (simple-function-scope-line? line) + (or (simple-python-def-line? line) + (simple-top-level-js-function-line? line))) (def (finding-simple-function-scope source finding) (let* ([line (finding-start-line finding)] [target-line (source-line source line)] [target-indent (line-indent target-line)]) (if (= target-indent 0) - (and (simple-python-def-line? target-line) line) + (and (simple-function-scope-line? target-line) line) (let loop ([current-line (- line 1)]) (cond [(< current-line 1) #f] @@ -9320,7 +9347,7 @@ (let* ([text (source-line source current-line)] [indent (line-indent text)]) (if (and (< indent target-indent) - (simple-python-def-line? text)) + (simple-function-scope-line? text)) current-line (loop (- current-line 1))))]))))) (def (same-simple-function-scope? source a b) @@ -10066,6 +10093,13 @@ assignment-finding sink source-text) + (let ([lhs (assignment-lhs-text + assignment-finding + source-text)]) + (and lhs + (string=? + (string-trim lhs) + (string-trim (finding-text source source-text))))) (and to-binding (string=? (finding-text source source-text) --- a/src/.jerbuild-hashes +++ b/src/.jerbuild-hashes @@ -3,7 +3,7 @@ ("src/semgrep/output/json.ss" . "293881CFA2ADB7BC") ("src/semgrep/lang.ss" . "7E5441BD00A7F1D4") ("src/semgrep/parse/parse-target.ss" . "E74854DDDACF6BA") - ("src/semgrep/scan.ss" . "93304BC0FC3B86D0") + ("src/semgrep/scan.ss" . "5CDBAEC3368097F9") ("src/semgrep/schema/lang.ss" . "CAE2CA859C9A9FD0") ("src/semgrep/rule.ss" . "E12C108153C181FA") ("src/semgrep/fix.ss" . "2E5B65B1FEF3B2B1") --- a/src/semgrep/scan.ss +++ b/src/semgrep/scan.ss @@ -4870,6 +4870,24 @@ (def (pattern-inside-clause? entry) (eq? (car entry) 'pattern-inside)) +(def (pattern-either-inside-only-entry? entry) + (and (eq? (car entry) 'pattern-either) + (not (null? (cdr entry))) + (let loop ([xs (cdr entry)]) + (or (null? xs) + (and (pattern-inside-clause? (car xs)) + (loop (cdr xs))))))) + +(def (seed-positive-clauses positive-clauses) + (let ([non-inside-either + (sg-filter + (lambda (entry) + (not (pattern-either-inside-only-entry? entry))) + positive-clauses)]) + (if (null? non-inside-either) + positive-clauses + non-inside-either))) + (def (preferred-seed-positive-entry? entry) (case (car entry) [(pattern-regex pattern pattern-as) #t] @@ -8984,6 +9002,7 @@ target-root regex-captures?) (let* ([positive-clauses (sg-filter positive-pattern-entry? clauses)] + [seedable-positive-clauses (seed-positive-clauses positive-clauses)] [preferred-positive-clauses (sg-filter preferred-seed-positive-entry? clauses)]) (let* ([inside-seeds (sg-filter pattern-inside-clause? clauses)] @@ -8991,8 +9010,8 @@ (cond [(not (null? preferred-positive-clauses)) (car preferred-positive-clauses)] - [(not (null? positive-clauses)) - (car positive-clauses)] + [(not (null? seedable-positive-clauses)) + (car seedable-positive-clauses)] [(not (null? inside-seeds)) (car inside-seeds)] [else #f])]) @@ -9138,6 +9157,7 @@ regex-captures? initial-bindings) (let* ([positive-clauses (sg-filter positive-pattern-entry? clauses)] + [seedable-positive-clauses (seed-positive-clauses positive-clauses)] [preferred-positive-clauses (sg-filter preferred-seed-positive-entry? clauses)]) (let* ([inside-seeds (sg-filter pattern-inside-clause? clauses)] @@ -9145,8 +9165,8 @@ (cond [(not (null? preferred-positive-clauses)) (car preferred-positive-clauses)] - [(not (null? positive-clauses)) - (car positive-clauses)] + [(not (null? seedable-positive-clauses)) + (car seedable-positive-clauses)] [(not (null? inside-seeds)) (car inside-seeds)] [else #f])]) @@ -9683,6 +9703,7 @@ "const $L = $R" "let $L = $R" "var $L = $R" + "$L += $R" "const $L: $T = $R" "let $L: $T = $R" "var $L: $T = $R" @@ -9912,12 +9933,20 @@ (or (sg-string-prefix? "def " trimmed) (sg-string-prefix? "async def " trimmed)))) +(def (simple-top-level-js-function-line? line) + (and (= (line-indent line) 0) + (sg-string-prefix? "function " (string-trim line)))) + +(def (simple-function-scope-line? line) + (or (simple-python-def-line? line) + (simple-top-level-js-function-line? line))) + (def (finding-simple-function-scope source finding) (let* ([line (finding-start-line finding)] [target-line (source-line source line)] [target-indent (line-indent target-line)]) (if (= target-indent 0) - (and (simple-python-def-line? target-line) line) + (and (simple-function-scope-line? target-line) line) (let loop ([current-line (- line 1)]) (cond [(< current-line 1) #f] @@ -9925,7 +9954,7 @@ (let* ([text (source-line source current-line)] [indent (line-indent text)]) (if (and (< indent target-indent) - (simple-python-def-line? text)) + (simple-function-scope-line? text)) current-line (loop (- current-line 1))))]))))) @@ -10611,6 +10640,11 @@ assignment-finding sink source-text) + (let ([lhs (assignment-lhs-text assignment-finding source-text)]) + (and lhs + (string=? + (string-trim lhs) + (string-trim (finding-text source source-text))))) (and to-binding (string=? (finding-text source source-text) --- a/tests/smoke.ss +++ b/tests/smoke.ss @@ -1547,6 +1547,30 @@ (check (length findings) => 1) (check (finding-start-line (car findings)) => 3))) +(test-case "scan JavaScript taint augmented assignment propagation" + (let* ([taint-config + "rules:\n - id: demo.taint.js.augassign\n mode: taint\n languages: [javascript]\n message: augmented assignment\n severity: WARNING\n pattern-sources:\n - patterns:\n - pattern: req.on('data', function ($CHUNK) { ... })\n - focus-metavariable: $CHUNK\n pattern-sinks:\n - pattern: sink(...)\n"] + [findings + (scan-config-string + taint-config + "javascript" + "demo.js" + "function test(req) {\n var buf = '';\n req.on('data', function (chunk) {\n buf += chunk;\n });\n sink(buf);\n}\n")]) + (check (length findings) => 1) + (check (finding-start-line (car findings)) => 6))) + +(test-case "scan JavaScript taint source under either pattern-inside" + (let* ([taint-config + "rules:\n - id: demo.taint.js.inside-either-source\n mode: taint\n languages: [javascript]\n message: inside either source\n severity: WARNING\n pattern-sources:\n - patterns:\n - pattern-either:\n - pattern-inside: function ... ($REQ, $RES) {...}\n - pattern-inside: function ... ($REQ, $RES, $NEXT) {...}\n - patterns:\n - pattern: $REQ.on('data', function ($CHUNK) { ... })\n - focus-metavariable: $CHUNK\n pattern-sinks:\n - pattern: sink(...)\n"] + [findings + (scan-config-string + taint-config + "javascript" + "demo.js" + "function test() {\n var foobar = function(req, res, next) {\n var buf = '';\n req.on('data', function (chunk) {\n buf += chunk;\n });\n sink(buf);\n };\n}\n")]) + (check (length findings) => 1) + (check (finding-start-line (car findings)) => 7))) + (test-case "scan JavaScript taint default parameter propagation" (let* ([taint-config "rules:\n - id: demo.taint.js.default-param\n mode: taint\n languages: [javascript]\n message: default param\n severity: WARNING\n pattern-sources:\n - pattern: source()\n pattern-sinks:\n - pattern: sink(...)\n"] @@ -1559,6 +1583,18 @@ (check (length findings) => 2) (check (map finding-start-line findings) => '(4 8)))) +(test-case "scan JavaScript taint focused param reassignment clears source" + (let* ([taint-config + "rules:\n - id: demo.taint.js.param-source\n mode: taint\n languages: [javascript]\n message: param source\n severity: WARNING\n pattern-sources:\n - patterns:\n - pattern: |\n function $FUNC({$PARAM}) { ... }\n - focus-metavariable: $PARAM\n pattern-sinks:\n - pattern: sink(...)\n"] + [findings + (scan-config-string + taint-config + "javascript" + "demo.js" + "function foo({user_input}) {\n x = user_input;\n sink(x);\n}\n\nfunction bar({user_input}) {\n user_input = 1;\n sink(user_input);\n}\n")]) + (check (length findings) => 1) + (check (finding-start-line (car findings)) => 3))) + (test-case "scan JavaScript taint reassignment clears token" (let* ([taint-config "rules:\n - id: demo.taint.js.reassignment.clean\n mode: taint\n languages: [javascript]\n message: cleaned assignment\n severity: WARNING\n pattern-sources:\n - pattern: taint\n pattern-sinks:\n - pattern: sink(...)\n"]