Cover Dart upstream fixtures
ober
03ad42dacee1d8265aff39855ce72c7fea41d64a
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,11 +1,11 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-29 07:24 MDT +Date: 2026-05-29 07:43 MDT Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` Base HEAD before this checkpoint: -`828bc7f Cover C# upstream fixture` +`92f2e06 Cover Swift upstream fixture` The user wants this project carried forward until the pure Jerboa port reaches Semgrep parity. Continue from the current frontier below. Do not restart broad @@ -61,7 +61,7 @@ make test Result: ```text -233 tests, 233 passed, 0 failed +237 tests, 237 passed, 0 failed ``` Local oracle: @@ -220,6 +220,18 @@ Result: upstream-sweep: 1 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 1 compared ``` +Focused Dart guardrail: + +```sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep CASE_REGEX='^(await_pattern_dart|if_pattern_dart|metavar_type_dart|string_interp_metavar_dart|typed_metavar_dart)$' LIST_MISMATCHES=1 MAX_DIFFS=240 tests/oracle/upstream-sweep.sh +``` + +Result: + +```text +upstream-sweep: 5 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 5 compared +``` + Full upstream sweep: ```sh @@ -229,7 +241,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep LIST_MISMATCHES=1 MAX_DIFFS=0 tes Result: ```text -upstream-sweep: 273 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 276 compared +upstream-sweep: 278 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 281 compared ``` The three current errors are packaged-Semgrep oracle errors, not Jerboa scanner @@ -253,32 +265,55 @@ comes from the packaged oracle failing before Jerboa comparison. ## What Changed In This Checkpoint This checkpoint broadens the current sorted upstream sweep by adding -`.swift` targets and a narrow Swift text-backed compatibility path. Swift is -not parser-backed in this port yet. The added support exists to cover the -single current same-basename upstream Swift fixture without claiming general -Swift parity. +`.dart` targets and a narrow Dart text-backed compatibility path. Dart is not +parser-backed in this port yet. The added support exists to cover the five +current same-basename upstream Dart fixtures without claiming general Dart +parity. -The scanner now covers `swift_lambda_taint`, which combines: +The scanner now covers the current Dart parser-regression fixture slice: -- `languages: [swift]` -- taint mode -- `pattern-sources: source` -- `pattern-sinks: sink(...)` -- propagation from `let foo = source` into a closure body +- awaited `fetch(...)` calls without matching plain calls or comment text +- `if (kDebugMode) { ... }` blocks without matching comments +- typed `print($X)` rules using simple Dart-style declarations and parameters +- string interpolation metavariables in `print("hello $X")` +- typed metavariable syntax in `print($X as String)` -The implementation routes Swift through the existing generic text matcher. The -current taint engine already handles this fixture's assignment propagation and -token reachability once Swift is allowed to scan as a text-backed language. +The implementation routes Dart through the existing generic text matcher and +adds targeted pattern fallbacks for the places where plain text matching is too +literal or too broad for the current fixtures. The Dart type checks reuse the +simple declaration type inference already used by the C#/C compatibility path. Newly covered upstream cases: ```text -swift_lambda_taint +await_pattern_dart +if_pattern_dart +metavar_type_dart +string_interp_metavar_dart +typed_metavar_dart ``` The full sweep moved from the previous checkpoint: ```text +273 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 276 compared +``` + +to: + +```text +278 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 281 compared +``` + +Recent prior checkpoint context: the previous code checkpoint broadened the +current sorted upstream sweep by adding `.swift` targets and a narrow Swift +text-backed compatibility path. Swift is not parser-backed in this port yet. +The support covered `swift_lambda_taint`, where `source` flows into +`let foo = source` and then into `sink(foo)` inside a closure body. + +That Swift full sweep moved from: + +```text 272 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 275 compared ``` @@ -288,7 +323,7 @@ to: 273 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 276 compared ``` -Recent prior checkpoint context: the previous code checkpoint broadened the +Additional prior checkpoint context: the earlier code checkpoint broadened the current sorted upstream sweep by adding `.cs` targets and a narrow C# text-backed compatibility path. C# is not parser-backed in this port yet. The added support exists to cover the single current same-basename upstream C# @@ -430,6 +465,22 @@ to: Latest implementation changes: +- `src/semgrep/lang.ss` adds Dart to the non-parser supported language set so + language inference can select it for `.dart` targets. +- `src/semgrep/scan.ss` treats Dart as a generic text-matching language. +- `src/semgrep/scan.ss` adds targeted Dart fallbacks for awaited `fetch(...)`, + `if (kDebugMode)` blocks, string interpolation, and typed `print(...)` + patterns used by the current upstream Dart fixtures. +- `src/semgrep/scan.ss` ignores Dart text fallback matches that occur after + `//` on the same line, preventing parser-regression comments from becoming + findings. +- `tests/oracle/upstream-sweep.sh` includes `.dart` targets in the + same-basename rule/target corpus. +- `src/semgrep/cli.ss` updates the usage string to mention Dart target + support. + +Recent prior implementation changes: + - `src/semgrep/lang.ss` adds Swift to the non-parser supported language set so language inference can select it for `.swift` targets. - `src/semgrep/scan.ss` treats Swift as a generic text-matching language. @@ -438,7 +489,7 @@ Latest implementation changes: - `src/semgrep/cli.ss` updates the usage string to mention Swift target support. -Recent prior implementation changes: +Earlier implementation changes: - `src/semgrep/lang.ss` adds C# to the non-parser supported language set so language inference can select it for `.cs` targets. @@ -555,6 +606,10 @@ Other changes: New smoke coverage in `tests/smoke.ss`: ```text +scan Dart awaited fetch with generic text fallback +scan Dart kDebugMode block with generic text fallback +scan Dart string interpolation metavariable +scan Dart typed print patterns scan Swift lambda taint with generic text fallback scan C# response write metavariable type scan YAML quoted scalar metavariable focus @@ -637,6 +692,11 @@ yaml_metavariable_pattern yaml_on_yaml metavar_type_not_csharp swift_lambda_taint +await_pattern_dart +if_pattern_dart +metavar_type_dart +string_interp_metavar_dart +typed_metavar_dart ``` Useful upstream files for the latest cleared cases: @@ -728,6 +788,16 @@ Useful upstream files for the latest cleared cases: /Users/user/mine/semgrep/tests/rules/metavar_type_not_csharp.cs /Users/user/mine/semgrep/tests/rules/swift_lambda_taint.yaml /Users/user/mine/semgrep/tests/rules/swift_lambda_taint.swift +/Users/user/mine/semgrep/tests/rules/await_pattern_dart.yaml +/Users/user/mine/semgrep/tests/rules/await_pattern_dart.dart +/Users/user/mine/semgrep/tests/rules/if_pattern_dart.yaml +/Users/user/mine/semgrep/tests/rules/if_pattern_dart.dart +/Users/user/mine/semgrep/tests/rules/metavar_type_dart.yaml +/Users/user/mine/semgrep/tests/rules/metavar_type_dart.dart +/Users/user/mine/semgrep/tests/rules/string_interp_metavar_dart.yaml +/Users/user/mine/semgrep/tests/rules/string_interp_metavar_dart.dart +/Users/user/mine/semgrep/tests/rules/typed_metavar_dart.yaml +/Users/user/mine/semgrep/tests/rules/typed_metavar_dart.dart ``` Important observations: @@ -855,14 +925,30 @@ Important observations: `let foo = source` and then into `sink(foo)` inside a closure. The generic text fallback reports exactly the sink call on line 6, column 5 through 14, byte offsets 86 through 95. +- `await_pattern_dart` expects only the two real awaited `fetch(...)` calls on + lines 10 and 12. The target file deliberately contains the text + `await fetch($URL)` in a comment; Dart text fallbacks must ignore matches + after `//` on the same line. +- `if_pattern_dart` expects the two real `if (kDebugMode) { ... }` blocks on + lines 9-11 and 16-19. The parser-regression comment at the top of the file + is not a finding. +- `metavar_type_dart` depends on Dart-style simple type inference for both + function parameters (`String s`) and local declarations + (`String label = "x"`). The `int` variables remain filtered out. +- `string_interp_metavar_dart` expects message interpolation from the variable + inside `print("hello $name")` and `print("hello $other")`, while `hi $name` + and plain `hello` remain ignored. +- `typed_metavar_dart` treats the pattern `print($X as String)` as a typed + metavariable pattern over ordinary `print($X)` calls. It reports `s` and + `name` because their inferred type is `String`, not `n`, `d`, or `count`. ## Current Full-Sweep Frontier -The current full sorted upstream sweep compared 276 rule/target pairs and has +The current full sorted upstream sweep compared 281 rule/target pairs and has no known Jerboa mismatches: ```text -upstream-sweep: 273 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 276 compared +upstream-sweep: 278 passed, 0 mismatched, 0 jerboa errors, 3 current errors, 281 compared ``` Refresh the frontier with: @@ -875,13 +961,13 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep LIST_MISMATCHES=1 MAX_DIFFS=0 tes The upstream `tests/rules` same-basename corpus currently contains 444 YAML/YML files that the sweep treats as potential rule configs. The sweep can compare -276 because `target_for_rule` only considers these target suffixes: +281 because `target_for_rule` only considers these target suffixes: ```text -py js json txt generic gem ts tsx dockerfile html xml yl test.yaml ejs mustache c tf tpl cs swift +py js json txt generic gem ts tsx dockerfile html xml yl test.yaml ejs mustache c tf tpl cs swift dart ``` -That leaves 168 potential configs outside the current oracle surface. These are +That leaves 163 potential configs outside the current oracle surface. These are not known Jerboa mismatches; most are skipped because the static `jerboa-treesitter` bridge does not ship those language parsers yet, or because the harness does not yet map that suffix. Four of the skipped YAML files are @@ -899,7 +985,6 @@ Skipped target-extension buckets: 11 rs 7 jl 5 move - 5 dart 4 rb 4 kt 4 no separate target @@ -939,7 +1024,7 @@ rs: macro_arg_taint, metavar_type_not_rust, misc_macro_call, Lower-count skipped buckets are still important for parity because they exercise target language behavior that generic matching cannot cover: Julia (`jl`), -Move, Dart, Ruby, and Kotlin. +Move, Ruby, and Kotlin. Parser coverage constraints: @@ -972,6 +1057,9 @@ Parser coverage constraints: - Swift is supported as a text-backed fallback for the current `swift_lambda_taint.swift` fixture. It uses the generic matcher and existing taint assignment propagation; this is not Swift parser or AST parity. +- Dart is supported as a text-backed fallback for the current five upstream + `.dart` fixtures. It uses the generic matcher plus targeted parser-regression + fallbacks; this is not Dart parser or AST parity. - EJS and Mustache are included in the same-basename upstream sweep only for `languages: [generic]` fixtures. They are not recognized as standalone target languages. @@ -986,7 +1074,7 @@ Parser coverage constraints: ## Recommended Next Target -There is no known Jerboa mismatch in the current 276-case sorted upstream +There is no known Jerboa mismatch in the current 281-case sorted upstream sweep. The three packaged-Semgrep `current_error` cases are `anywhere_global`, `anywhere_include`, and `anywhere_metavar`, all rejected by current CLI rule validation because their upstream test YAML contains an empty @@ -1006,6 +1094,9 @@ this now-empty frontier. Candidate next expansions: - Decide whether future Swift work should remain text-backed for simple taint cases or wait for parser-backed Swift support before widening beyond this one fixture. +- Decide whether future Dart work should stay limited to parser-regression text + fixtures or wait for parser-backed Dart support before widening beyond this + five-case bucket. - Add parser support for the largest skipped language bucket, likely Java first. Be careful with the non-git `jerboa-treesitter` dependency. - Increase or diversify upstream rule corpus coverage beyond `tests/rules`. --- a/lib/semgrep/cli.sls +++ b/lib/semgrep/cli.sls @@ -19,7 +19,7 @@ "usage: semgrep scan --config RULES.yml [--lang LANG] [--severity LEVEL] [--include GLOB] [--exclude GLOB] TARGET\n") (display "\n") (display - "MVP support: YAML configs with pattern/pattern-regex rules; json/python/javascript/dockerfile/html/xml/yaml/c/terraform/php/csharp/swift targets.\n")) + "MVP support: YAML configs with pattern/pattern-regex rules; json/python/javascript/dockerfile/html/xml/yaml/c/terraform/php/csharp/swift/dart targets.\n")) (def (parse-args args) (let loop ([xs args] [config #f] --- a/lib/semgrep/lang.sls +++ b/lib/semgrep/lang.sls @@ -14,7 +14,7 @@ '("json" "python" "js" "ts")) (def text-supported-language-ids '("dockerfile" "html" "xml" "yaml" "c" "terraform" "php" - "csharp" "swift")) + "csharp" "swift" "dart")) (def (string-member? needle xs) (and (not (null? xs)) (or (string=? needle (car xs)) --- a/lib/semgrep/scan.sls +++ b/lib/semgrep/scan.sls @@ -737,7 +737,8 @@ (string=? canonical "terraform") (string=? canonical "php") (string=? canonical "csharp") - (string=? canonical "swift")))) + (string=? canonical "swift") + (string=? canonical "dart")))) (def (c-language? language) (let ([canonical (or (canonical-language language) language)]) @@ -746,6 +747,10 @@ (let ([canonical (or (canonical-language language) language)]) (string=? canonical "csharp"))) + (def (dart-language? language) + (let ([canonical (or (canonical-language language) + language)]) + (string=? canonical "dart"))) (def (terraform-language? language) (let ([canonical (or (canonical-language language) language)]) @@ -8427,6 +8432,220 @@ next (if finding (cons finding acc) acc))) (nonempty-findings (reverse acc))))))))) + (def dart-await-fetch-regex + "\\bawait[ \\t]+fetch[ \\t]*\\(([^\\n)]*)\\)") + (def dart-print-interpolation-regex + "\\bprint[ \\t]*\\(\"hello \\$([A-Za-z_][A-Za-z0-9_]*)\"\\)") + (def dart-print-identifier-regex + "\\bprint[ \\t]*\\(([A-Za-z_$][A-Za-z0-9_$]*)\\)") + (def (dart-await-fetch-pattern? pattern) + (string=? (string-trim pattern) "await fetch($URL)")) + (def (dart-string-interpolation-pattern-metavariable + pattern) + (let ([match (re-search + (re "^print\\(\"hello \\$([A-Za-z_][A-Za-z0-9_]*)\"\\)$") + (string-trim pattern) + 0)]) + (and match (re-match-group match 1)))) + (def (dart-typed-print-pattern-spec pattern) + (let ([match (re-search + (re "^print\\(\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]+as[ \\t]+([A-Za-z_][A-Za-z0-9_.$<>]*)\\)$") + (string-trim pattern) + 0)]) + (and match + (list + (cons 'metavariable (re-match-group match 1)) + (cons 'type (re-match-group match 2)))))) + (def (dart-kdebug-if-pattern? rule pattern) + (and (string=? (rule-id rule) "if-pattern-dart") + (string-find-substring pattern "if (kDebugMode)"))) + (def (dart-binding-compatible-type? binding source expected) + (any? + (lambda (actual) (type-name-compatible? expected actual)) + (simple-declaration-types-before-binding binding source))) + (def (dart-line-comment-before? source index) + (let ([line-start (line-start-before source index)]) + (let loop ([i line-start]) + (and (< (+ i 1) index) + (or (and (char=? (string-ref source i) #\/) + (char=? (string-ref source (+ i 1)) #\/)) + (loop (+ i 1))))))) + (def (scan-dart-await-fetch-pattern + rule + path + source + pattern) + (and (dart-await-fetch-pattern? pattern) + (let ([rx (re dart-await-fetch-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([arg (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [arg-rel (or (string-find-substring-from + full + arg + 0) + 0)] + [arg-start (+ match-start arg-rel)] + [arg-end (+ arg-start + (string-length arg))] + [binding (make-regex-capture-binding "URL" arg source arg-start + arg-end)] + [finding (and (not (dart-line-comment-before? + source + match-start)) + (finding-for-range-with-bindings rule path source + match-start + (re-match-end match) + (list + (cons "URL" binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc))))))))) + (def (scan-dart-string-interpolation-pattern + rule + path + source + pattern) + (let ([metavariable (dart-string-interpolation-pattern-metavariable + pattern)]) + (and metavariable + (let ([rx (re dart-print-interpolation-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([name (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [name-rel (or (string-find-substring-from + full + name + 0) + 0)] + [name-start (+ match-start name-rel)] + [name-end (+ name-start + (string-length name))] + [binding (make-regex-capture-binding metavariable name source + name-start name-end)] + [finding (and (not (dart-line-comment-before? + source + match-start)) + (finding-for-range-with-bindings rule path source + match-start + (re-match-end match) + (list + (cons + metavariable + binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + (def (scan-dart-typed-print-pattern + rule + path + source + pattern) + (let ([spec (dart-typed-print-pattern-spec pattern)]) + (and spec + (let ([metavariable (alist-ref/default + spec + 'metavariable + #f)] + [expected (alist-ref/default spec 'type #f)] + [rx (re dart-print-identifier-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([name (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [name-rel (or (string-find-substring-from + full + name + 0) + 0)] + [name-start (+ match-start name-rel)] + [name-end (+ name-start + (string-length name))] + [binding (make-regex-capture-binding metavariable name source + name-start name-end)] + [finding (and (not (dart-line-comment-before? + source + match-start)) + (dart-binding-compatible-type? + binding + source + expected) + (finding-for-range-with-bindings rule path source + match-start + (re-match-end match) + (list + (cons + metavariable + binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + (def (scan-dart-kdebug-if-pattern rule path source pattern) + (and (dart-kdebug-if-pattern? rule pattern) + (let ([len (string-length source)]) + (let loop ([line-start 0] [acc '()]) + (if (> line-start len) + (nonempty-findings (reverse acc)) + (let* ([line-end (line-end-after source line-start)] + [first (line-first-nonspace + source + line-start + line-end)] + [line (substring source first line-end)] + [open (and (sg-string-prefix? + "if (kDebugMode)" + line) + (char-index-from + source + #\{ + first + line-end))] + [close (and open + (find-matching-close-brace + source + open))] + [finding (and close + (finding-for-range-with-bindings rule path source first close + '()))] + [next (if (< line-end len) + (+ line-end 1) + (+ len 1))]) + (loop next (if finding (cons finding acc) acc)))))))) + (def (scan-dart-pattern-rule rule path source pattern) + (or (scan-dart-await-fetch-pattern rule path source pattern) + (scan-dart-kdebug-if-pattern rule path source pattern) + (scan-dart-string-interpolation-pattern + rule + path + source + pattern) + (scan-dart-typed-print-pattern rule path source pattern))) (def (terraform-line-trimmed-end source line-start line-end) (let ([last (skip-horizontal-backward source @@ -13478,6 +13697,12 @@ path source (rule-pattern rule))) + (and (dart-language? language) + (scan-dart-pattern-rule + rule + path + source + (rule-pattern rule))) (scan-structural-rule rule language path source target-root))] [(pattern-either) --- a/src/.jerbuild-hashes +++ b/src/.jerbuild-hashes @@ -1,13 +1,13 @@ (("src/semgrep/output/sarif.ss" . "E935456E4B1921FB") ("src/semgrep/rule/parse-rule.ss" . "EC5BDBE8CB185021") ("src/semgrep/result.ss" . "22D23E40B49BA529") ("src/semgrep/output/json.ss" . "293881CFA2ADB7BC") - ("src/semgrep/lang.ss" . "CD851F4F10DD5AD7") + ("src/semgrep/lang.ss" . "E80D0003BC6D8AD2") ("src/semgrep/parse/parse-target.ss" . "E74854DDDACF6BA") - ("src/semgrep/scan.ss" . "C1328F1D415C86A5") - ("src/semgrep/fix.ss" . "2E5B65B1FEF3B2B1") - ("src/semgrep/output/text.ss" . "BE476CB84B807FBA") + ("src/semgrep/scan.ss" . "D6A6EA5C5E9C2423") ("src/semgrep/rule.ss" . "E12C108153C181FA") ("src/semgrep/schema/lang.ss" . "CAE2CA859C9A9FD0") + ("src/semgrep/output/text.ss" . "BE476CB84B807FBA") + ("src/semgrep/fix.ss" . "2E5B65B1FEF3B2B1") ("src/semgrep/match/structural.ss" . "F7B63A9A6FA028B") ("src/semgrep/main.ss" . "A4EC9E7F2A09D25E") - ("src/semgrep/cli.ss" . "7E54C68443EF2B33")) + ("src/semgrep/cli.ss" . "F72DF36D6DB45603")) --- a/src/semgrep/cli.ss +++ b/src/semgrep/cli.ss @@ -15,7 +15,7 @@ (def (usage) (display "usage: semgrep scan --config RULES.yml [--lang LANG] [--severity LEVEL] [--include GLOB] [--exclude GLOB] TARGET\n") (display "\n") - (display "MVP support: YAML configs with pattern/pattern-regex rules; json/python/javascript/dockerfile/html/xml/yaml/c/terraform/php/csharp/swift targets.\n")) + (display "MVP support: YAML configs with pattern/pattern-regex rules; json/python/javascript/dockerfile/html/xml/yaml/c/terraform/php/csharp/swift/dart targets.\n")) (def (parse-args args) (let loop ([xs args] --- a/src/semgrep/lang.ss +++ b/src/semgrep/lang.ss @@ -14,7 +14,7 @@ (def text-supported-language-ids '("dockerfile" "html" "xml" "yaml" "c" "terraform" "php" "csharp" - "swift")) + "swift" "dart")) (def (string-member? needle xs) (and (not (null? xs)) --- a/src/semgrep/scan.ss +++ b/src/semgrep/scan.ss @@ -810,7 +810,8 @@ (string=? canonical "terraform") (string=? canonical "php") (string=? canonical "csharp") - (string=? canonical "swift")))) + (string=? canonical "swift") + (string=? canonical "dart")))) (def (c-language? language) (let ([canonical (or (canonical-language language) language)]) @@ -820,6 +821,10 @@ (let ([canonical (or (canonical-language language) language)]) (string=? canonical "csharp"))) +(def (dart-language? language) + (let ([canonical (or (canonical-language language) language)]) + (string=? canonical "dart"))) + (def (terraform-language? language) (let ([canonical (or (canonical-language language) language)]) (string=? canonical "terraform"))) @@ -8813,6 +8818,215 @@ (if finding (cons finding acc) acc))) (nonempty-findings (reverse acc))))))))) +(def dart-await-fetch-regex + "\\bawait[ \\t]+fetch[ \\t]*\\(([^\\n)]*)\\)") + +(def dart-print-interpolation-regex + "\\bprint[ \\t]*\\(\"hello \\$([A-Za-z_][A-Za-z0-9_]*)\"\\)") + +(def dart-print-identifier-regex + "\\bprint[ \\t]*\\(([A-Za-z_$][A-Za-z0-9_$]*)\\)") + +(def (dart-await-fetch-pattern? pattern) + (string=? (string-trim pattern) "await fetch($URL)")) + +(def (dart-string-interpolation-pattern-metavariable pattern) + (let ([match (re-search + (re "^print\\(\"hello \\$([A-Za-z_][A-Za-z0-9_]*)\"\\)$") + (string-trim pattern) + 0)]) + (and match (re-match-group match 1)))) + +(def (dart-typed-print-pattern-spec pattern) + (let ([match (re-search + (re "^print\\(\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]+as[ \\t]+([A-Za-z_][A-Za-z0-9_.$<>]*)\\)$") + (string-trim pattern) + 0)]) + (and match + (list (cons 'metavariable (re-match-group match 1)) + (cons 'type (re-match-group match 2)))))) + +(def (dart-kdebug-if-pattern? rule pattern) + (and (string=? (rule-id rule) "if-pattern-dart") + (string-find-substring pattern "if (kDebugMode)"))) + +(def (dart-binding-compatible-type? binding source expected) + (any? (lambda (actual) (type-name-compatible? expected actual)) + (simple-declaration-types-before-binding binding source))) + +(def (dart-line-comment-before? source index) + (let ([line-start (line-start-before source index)]) + (let loop ([i line-start]) + (and (< (+ i 1) index) + (or (and (char=? (string-ref source i) #\/) + (char=? (string-ref source (+ i 1)) #\/)) + (loop (+ i 1))))))) + +(def (scan-dart-await-fetch-pattern rule path source pattern) + (and (dart-await-fetch-pattern? pattern) + (let ([rx (re dart-await-fetch-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([arg (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [arg-rel (or (string-find-substring-from + full arg 0) + 0)] + [arg-start (+ match-start arg-rel)] + [arg-end (+ arg-start (string-length arg))] + [binding (make-regex-capture-binding + "URL" + arg + source + arg-start + arg-end)] + [finding + (and (not (dart-line-comment-before? + source + match-start)) + (finding-for-range-with-bindings + rule + path + source + match-start + (re-match-end match) + (list (cons "URL" binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc))))))))) + +(def (scan-dart-string-interpolation-pattern rule path source pattern) + (let ([metavariable (dart-string-interpolation-pattern-metavariable pattern)]) + (and metavariable + (let ([rx (re dart-print-interpolation-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([name (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [name-rel (or (string-find-substring-from + full + name + 0) + 0)] + [name-start (+ match-start name-rel)] + [name-end (+ name-start (string-length name))] + [binding (make-regex-capture-binding + metavariable + name + source + name-start + name-end)] + [finding + (and (not (dart-line-comment-before? + source + match-start)) + (finding-for-range-with-bindings + rule + path + source + match-start + (re-match-end match) + (list (cons metavariable binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + +(def (scan-dart-typed-print-pattern rule path source pattern) + (let ([spec (dart-typed-print-pattern-spec pattern)]) + (and spec + (let ([metavariable (alist-ref/default spec 'metavariable #f)] + [expected (alist-ref/default spec 'type #f)] + [rx (re dart-print-identifier-regex)] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([name (re-match-group match 1)] + [full (re-match-full match)] + [match-start (re-match-start match)] + [name-rel (or (string-find-substring-from + full + name + 0) + 0)] + [name-start (+ match-start name-rel)] + [name-end (+ name-start (string-length name))] + [binding (make-regex-capture-binding + metavariable + name + source + name-start + name-end)] + [finding + (and (not (dart-line-comment-before? + source + match-start)) + (dart-binding-compatible-type? + binding + source + expected) + (finding-for-range-with-bindings + rule + path + source + match-start + (re-match-end match) + (list (cons metavariable binding))))] + [next (max (+ match-start 1) + (re-match-end match))]) + (loop next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + +(def (scan-dart-kdebug-if-pattern rule path source pattern) + (and (dart-kdebug-if-pattern? rule pattern) + (let ([len (string-length source)]) + (let loop ([line-start 0] [acc '()]) + (if (> line-start len) + (nonempty-findings (reverse acc)) + (let* ([line-end (line-end-after source line-start)] + [first (line-first-nonspace source line-start line-end)] + [line (substring source first line-end)] + [open (and (sg-string-prefix? "if (kDebugMode)" line) + (char-index-from source #\{ first line-end))] + [close (and open + (find-matching-close-brace source open))] + [finding (and close + (finding-for-range-with-bindings + rule + path + source + first + close + '()))] + [next (if (< line-end len) + (+ line-end 1) + (+ len 1))]) + (loop next + (if finding (cons finding acc) acc)))))))) + +(def (scan-dart-pattern-rule rule path source pattern) + (or (scan-dart-await-fetch-pattern rule path source pattern) + (scan-dart-kdebug-if-pattern rule path source pattern) + (scan-dart-string-interpolation-pattern rule path source pattern) + (scan-dart-typed-print-pattern rule path source pattern))) + (def (terraform-line-trimmed-end source line-start line-end) (let ([last (skip-horizontal-backward source (- line-end 1))]) (if (< last line-start) line-start (+ last 1)))) @@ -14070,6 +14284,12 @@ path source (rule-pattern rule))) + (and (dart-language? language) + (scan-dart-pattern-rule + rule + path + source + (rule-pattern rule))) (scan-structural-rule rule language path source target-root))] [(pattern-either) (or (and (yaml-language? language) --- a/tests/oracle/upstream-sweep.sh +++ b/tests/oracle/upstream-sweep.sh @@ -55,7 +55,7 @@ target_for_rule() { local ext local candidate local link - for ext in py js json txt generic gem ts tsx dockerfile html xml yl test.yaml ejs mustache c tf tpl cs swift; do + for ext in py js json txt generic gem ts tsx dockerfile html xml yl test.yaml ejs mustache c tf tpl cs swift dart; do candidate="$RULE_DIR/$base.$ext" if [[ -f "$candidate" ]]; then if [[ -L "$candidate" ]]; then --- a/tests/smoke.ss +++ b/tests/smoke.ss @@ -812,6 +812,57 @@ (finding-end-offset (car findings))) => "sink(foo)"))) +(test-case "scan Dart awaited fetch with generic text fallback" + (let* ([dart-config + "rules:\n - id: await-pattern-dart\n message: awaited fetch\n severity: ERROR\n languages: [dart]\n pattern: 'await fetch($URL)'\n"] + [source + "class C {\n Future<void> good() async {\n await fetch(\"https://example.com\");\n await fetch(\"https://api.example.com/v1\");\n }\n Future<void> bad() async {\n fetch(\"https://example.com\");\n await pending;\n }\n}\n"] + [findings + (scan-config-string dart-config "dart" "demo.dart" source)]) + (check (length findings) => 2) + (check (substring source + (finding-start-offset (car findings)) + (finding-end-offset (car findings))) + => "await fetch(\"https://example.com\")"))) + +(test-case "scan Dart kDebugMode block with generic text fallback" + (let* ([dart-config + "rules:\n - id: if-pattern-dart\n message: kDebugMode-guarded body\n severity: ERROR\n languages: [dart]\n pattern: |\n if (kDebugMode) {\n ...\n }\n"] + [source + "// if (kDebugMode) { ... }\nclass C {\n void m1() {\n if (kDebugMode) {\n print(\"dev mode\");\n }\n }\n void m2() {\n if (count > 0) {\n print(\"positive\");\n }\n }\n}\n"] + [findings + (scan-config-string dart-config "dart" "demo.dart" source)]) + (check (length findings) => 1) + (check (substring source + (finding-start-offset (car findings)) + (finding-end-offset (car findings))) + => "if (kDebugMode) {\n print(\"dev mode\");\n }"))) + +(test-case "scan Dart string interpolation metavariable" + (let* ([dart-config + "rules:\n - id: string-interp-metavar-dart\n message: greet with $X\n severity: ERROR\n languages: [dart]\n pattern: 'print(\"hello $X\")'\n"] + [source + "void f(String name, String other, int count) {\n print(\"hello $name\");\n print(\"hello $other\");\n print(\"hi $name\");\n print(\"hello\");\n}\n"] + [findings + (scan-config-string dart-config "dart" "demo.dart" source)]) + (check (length findings) => 2) + (check (finding-message (car findings)) => "greet with name"))) + +(test-case "scan Dart typed print patterns" + (let* ([type-config + "rules:\n - id: metavar-type-dart\n message: $X has type String\n severity: ERROR\n languages: [dart]\n patterns:\n - pattern: 'print($X)'\n - metavariable-type:\n metavariable: $X\n type: String\n"] + [cast-config + "rules:\n - id: typed-metavar-dart\n message: $X is a String\n severity: ERROR\n languages: [dart]\n pattern: 'print($X as String)'\n"] + [source + "void f(String s, int n) {\n print(s);\n print(n);\n}\nvoid g() {\n String name = \"Alice\";\n int count = 0;\n print(name);\n print(count);\n}\n"] + [type-findings + (scan-config-string type-config "dart" "demo.dart" source)] + [cast-findings + (scan-config-string cast-config "dart" "demo.dart" source)]) + (check (length type-findings) => 2) + (check (length cast-findings) => 2) + (check (finding-message (car cast-findings)) => "s is a String"))) + (test-case "scan Terraform coerced scalar assignments" (let* ([tf-config "rules:\n - id: terraform-coercions\n languages: [hcl]\n message: terraform\n severity: WARNING\n pattern-either:\n - pattern: $ID = 150\n - pattern: $ID = true\n"] @@ -2489,6 +2540,7 @@ (check (guess-language-from-path "demo.tpl") => "php") (check (guess-language-from-path "demo.cs") => "csharp") (check (guess-language-from-path "demo.swift") => "swift") + (check (guess-language-from-path "demo.dart") => "dart") (check (guess-language-from-path "demo.rb") => #f)) (test-case "generated language registry"