Cover JavaScript class and dependency fixtures
ober
1b617d32abf1a35aff498371a7c1bc861fa71add
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,30 +1,30 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-30 00:17 MDT +Date: 2026-05-30 00:55 MDT Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` Base HEAD before this checkpoint: -`1de036c Cover JavaScript singleton pattern fixtures` +`2cda4de Cover JavaScript decorator and catch fixtures` The user wants this project carried forward until the pure Jerboa port reaches Semgrep parity. Do not treat this handoff as completion. This checkpoint builds -on the import/CommonJS and singleton JS checkpoints, then closes six additional -JavaScript catch/decorator/export fixture mismatches. +on the import/CommonJS, singleton, and catch/decorator/export JS checkpoints, +then closes five additional JavaScript class and dependency-field fixture +mismatches. ## Current Headline - Full Python pattern sweep: 164 passed / 0 mismatched. -- Full JavaScript pattern sweep: 98 passed / 34 mismatched, improved from - 92 passed / 40 mismatched by closing six catch/decorator/export cases. -- Promoted JavaScript oracle slice: 57 passed / 0 mismatched. +- Full JavaScript pattern sweep: 103 passed / 29 mismatched, improved from + 98 passed / 34 mismatched by closing five class/dependency-field cases. +- Promoted JavaScript oracle slice: 62 passed / 0 mismatched. - Broad same-basename upstream sweep: 437 passed / 0 mismatched / 0 Jerboa errors, with 3 current-Semgrep oracle errors. -- Smoke suite: 294 tests, 294 passed, 0 failed. +- Smoke suite: 295 tests, 295 passed, 0 failed. - Semgrep parity is not reached yet. The remaining frontier is mostly deeper - structural JavaScript matching, JSX/XML ranges, class patterns, arrows, - partial lambda/object patterns, and a few metavariable equality/range edge - cases. + structural JavaScript matching, JSX/XML ranges, arrows, partial lambda + patterns, and a few metavariable equality/range edge cases. ## Recent Checkpoint Chain @@ -36,9 +36,12 @@ Recent commits/checkpoints that matter for the next agent: - `1de036c Cover JavaScript singleton pattern fixtures`: committed. It promoted ten small JS singleton fixture cases and left the full JS sweep at 92 passed / 40 mismatched. -- This checkpoint: to be committed as `Cover JavaScript decorator and catch - fixtures`. It promotes six catch/decorator/export cases and leaves the full - JS sweep at 98 passed / 34 mismatched. +- `2cda4de Cover JavaScript decorator and catch fixtures`: committed. It + promoted six catch/decorator/export cases and left the full JS sweep at + 98 passed / 34 mismatched. +- This checkpoint: to be committed as `Cover JavaScript class and dependency + fixtures`. It promotes five class/dependency-field cases and leaves the full + JS sweep at 103 passed / 29 mismatched. The current checkpoint modifies only the Jerboa scanner, generated scanner artifact, promoted oracle regex, smoke tests, and this handoff. No Semgrep @@ -100,16 +103,16 @@ Result: jerbuild: 14 processed, 0 skipped, 0 errors (of 14 total) ``` -Focused JavaScript catch/decorator/export slice: +Focused JavaScript class/dependency-field slice: ```sh -SEMGREP_CURRENT=/Users/user/.local/bin/semgrep PATTERN_LANGS=js CASE_REGEX='^(misc_catch_unbound|partial_catch|misc_decorators_method|partial_decorators_method|partial_decorators_arg_method|misc_export)$' LIST_MISMATCHES=1 MAX_DIFFS=160 tests/oracle/patterns-sweep.sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep PATTERN_LANGS=js CASE_REGEX='^(less_class_complex|less_class_complex_2|less_inheritance|metavar_class_fields|partial_single_field)$' LIST_MISMATCHES=1 MAX_DIFFS=180 tests/oracle/patterns-sweep.sh ``` Result: ```text -patterns-sweep: 6 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 6 compared +patterns-sweep: 5 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 5 compared ``` Smoke suite: @@ -121,7 +124,7 @@ make test Result: ```text -294 tests, 294 passed, 0 failed +295 tests, 295 passed, 0 failed ``` Promoted JavaScript oracle: @@ -133,7 +136,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep make patterns-js-oracle Result: ```text -patterns-sweep: 57 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 57 compared +patterns-sweep: 62 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 62 compared ``` Full upstream JavaScript pattern sweep: @@ -145,7 +148,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep PATTERN_LANGS=js LIST_MISMATCHES= Result: ```text -patterns-sweep: 98 passed, 34 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared +patterns-sweep: 103 passed, 29 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared ``` Local oracle: @@ -186,19 +189,18 @@ Result: no output and exit code 0. ## What Changed In This Checkpoint -`JS_PATTERN_CASE_REGEX` in `Makefile` now promotes these six additional JS +`JS_PATTERN_CASE_REGEX` in `Makefile` now promotes these five additional JS fixture cases: ```text -misc_catch_unbound -misc_decorators_method -misc_export -partial_catch -partial_decorators_arg_method -partial_decorators_method +less_class_complex +less_class_complex_2 +less_inheritance +metavar_class_fields +partial_single_field ``` -The full promoted JS oracle now covers 57 cases: +The full promoted JS oracle now covers 62 cases: ```text aliasing_require @@ -229,6 +231,10 @@ infer_const_1 infer_const_regexp equivalence_constant_propagation2 equivalence_constant_propagation3 +less_class_complex +less_class_complex_2 +less_inheritance +metavar_class_fields metavar_import metavar_import2 metavar_import3 @@ -250,6 +256,7 @@ partial_catch partial_decorators_arg_method partial_decorators_method partial_function +partial_single_field regexp_fieldname switch_case_pattern template_string_w_metavar @@ -312,9 +319,9 @@ Important guardrail: these helpers rely on source-text scanning and range construction, not a complete generic structural matching engine. Keep them heavily gated until the deeper matcher is improved. -### Catch/Decorator/Export Bundle In This Checkpoint +### Catch/Decorator/Export Bundle From `2cda4de` -This checkpoint extends `scan-javascript-small-pattern-fallbacks` with three +That checkpoint extended `scan-javascript-small-pattern-fallbacks` with three additional pattern-shape-gated dispatches: - `scan-javascript-catch-pattern` @@ -356,20 +363,48 @@ New export support: - Non-exported async functions with the same parameters are intentionally ignored. +### Class/Dependency Bundle In This Checkpoint + +This checkpoint extends `scan-javascript-small-pattern-fallbacks` with two +additional pattern-shape-gated dispatches: + +- `scan-javascript-class-pattern` +- `scan-javascript-partial-dependencies-field-pattern` + +New class support: + +- `class A { ... }` matches the full `class A` declaration range and rejects + different class names. This closes `less_class_complex`. +- `class A { $F = $X; ... }` and `class A { $F = $X; }` match full `class A` + declarations that contain a class field assignment anywhere in the class + body, binding the field name and assigned value. This closes + `less_class_complex_2` and `metavar_class_fields`. +- `class A { $F() { ... } }` matches full `class A` declarations that contain + a method, including `class A extends B`. This closes `less_inheritance`. +- The class scanner returns full class ranges, not just field or method ranges, + matching the upstream fixture expectations. + +New partial dependency field support: + +- `dependencies: { ... }` matches the quoted JSON-style field + `"dependencies": { ... }` from the key start through the dependency object + close. This closes `partial_single_field`. +- The fallback is limited to the exact partial dependency field pattern, not + general object-field matching. + ## Smoke Coverage Added -`tests/smoke.ss` now includes `scan JavaScript catch decorator and export +`tests/smoke.ss` now includes `scan JavaScript class and dependency field fixture fallbacks`, which verifies: -- `catch($E) { ... }` catch-clause range and start line. -- Optional catch binding under `try { ... } catch($ERR) { }`. -- Decorated method body matching without matching decorator calls. -- Decorated method signature matching through the return type. -- Decorated method call-argument matching with `$FIELD`. -- Exported async function matching with a `res` parameter while ignoring a - non-exported async function. +- `class A { ... }` matches only class `A`. +- `class A { $F = $X; ... }` matches a class field assignment and ignores + class `B`. +- `class A { $F() { ... } }` matches both plain and `extends` class `A` + declarations. +- `dependencies: { ... }` matches a JSON-style dependency object field. -The smoke suite count is now 294. +The smoke suite count is now 295. ## Current Frontier @@ -380,11 +415,11 @@ patterns-sweep: 164 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 s ``` The full JavaScript `.sgrep` pattern fixture sweep is not clean yet, but the -import cluster, singleton cases, and catch/decorator/export cases are closed -and promoted: +import cluster, singleton cases, catch/decorator/export cases, and +class/dependency cases are closed and promoted: ```text -patterns-sweep: 98 passed, 34 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared +patterns-sweep: 103 passed, 29 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared ``` Remaining JS mismatches after this checkpoint: @@ -407,12 +442,8 @@ deep_stmt2 dots_jsx_attr dots_jsx_body dots_jsx_body2 -less_class_complex -less_class_complex_2 -less_inheritance less_xml_attr less_xml_body -metavar_class_fields metavar_equality_vardef_vs_use metavar_jsx_val metavar_stmt_or_expr @@ -422,7 +453,6 @@ misc_arrow2 misc_deep_expr_stmt misc_method_chain_vs_stmt partial_lambda_or_func -partial_single_field unordered_xml_attr ``` @@ -432,9 +462,8 @@ Next high-value clusters: `caching_*`, `deep_*`, `misc_deep_expr_stmt`, `misc_method_chain_vs_stmt`. 2. JSX/XML range and less/dots/metavariable parity: `dots_jsx_*`, `less_xml_*`, `metavar_xml_attr`, `unordered_xml_attr`. -3. Class and partial object/function shapes: - `less_class_*`, `less_inheritance`, `metavar_class_fields`, - `partial_lambda_or_func`, `partial_single_field`. +3. Partial function/lambda shapes: + `partial_lambda_or_func`. 4. Arrow and metavariable equality/range edge cases: `misc_arrow*`, `ac_matching_bitor`, `metavar_equality_vardef_vs_use`, `metavar_stmt_or_expr`, @@ -446,12 +475,15 @@ Next high-value clusters: `scan-javascript-catch-pattern`, `scan-javascript-decorated-method-pattern`, or `scan-javascript-export-function-pattern`. +- Rerun the focused five-case class/dependency slice after touching + `scan-javascript-class-pattern` or + `scan-javascript-partial-dependencies-field-pattern`. - Rerun the focused ten-case singleton slice after touching `scan-javascript-small-pattern-fallbacks` or the singleton helpers it dispatches to. - Rerun the focused 13-case import slice after touching import resolution, declaration ranges, or CommonJS alias logic. -- Rerun `make patterns-js-oracle`; it is now the 57-case promoted JS guard. +- Rerun `make patterns-js-oracle`; it is now the 62-case promoted JS guard. - Rerun the full 132-case JS pattern sweep before promoting more JS fixtures. - Keep `cp_array_destructure` as a regression guard for JS constant equality invalidation after destructuring reassignment. --- a/Makefile +++ b/Makefile @@ -5,7 +5,7 @@ SCHEME ?= $(HOST_SCHEME) JERBUILD ?= $(if $(wildcard $(JERBOA_HOME)/jerbuild),$(JERBOA_HOME)/jerbuild,$(SCHEME) --libdirs "$(JERBOA_HOME)/lib" --script "$(JERBOA_HOME)/jerbuild.ss") LIBDIRS = lib:$(JERBOA_TREESITTER_HOME)/lib:$(JERBOA_HOME)/lib PATTERN_CASE_REGEX ?= ^(ac_matching_dots|ac_matching_dots1|ac_matching_explosion|ac_matching_explosion1|ac_matching_explosion2|ac_matching_explosion3|ac_matching_free|ac_matching_free1|ac_matching_if|ac_matching_if1|ac_matching_mvars|ac_matching_mvars1|ac_matching_mvars2|aliasing_attribute|assoc_matching_bug|assoc_matching_dots|assoc_matching_dots1|assoc_matching_explosion|assoc_matching_explosion1|assoc_matching_explosion2|assoc_matching_explosion3|assoc_matching_free|assoc_matching_free1|assoc_matching_mvars|assoc_matching_mvars1|constprop_dataflow|constprop_used_twice|cp_concat|cp_exception|cp_label|cp_label1|cp_method_call|cp_python_mult_string|cp_python_mult_string1|cp_rlval|cp_string_mvar|cp_strings|cp_with|cp_yield|deep_cond|deep_expr_operator|deep_expr_vs_statement|deep_exprstmt|deep_fake_dataflow|deep_try|dict_ellipsis|dots_attribute|dots_expr_plus|dots_expr_plus_string|dots_fstring|dots_fstring_with_match_stmt|dots_inherit|dots_list|dots_nested_stmts|dots_stmts|equivalence_constant_propagation|equivalence_f_string|equivalence_f_string_2|equivalence_f_string_3|equivalence_f_string_4|equivalence_interpolated_str|equivalence_interpolated_str2|equivalence_keyword_args|equivalence_naming_import|import_metavar|import_negatives|import_negatives2|imports|index_tuple|less_attributes|less_catch|less_catch_multiple|less_inherits|less_typehint|less_typehint2|matching_if_expr|metavar_anno|metavar_anno_fqn|metavar_class_def|metavar_dict|metavar_equality_param_vs_use|metavar_equality_var|metavar_func_def|metavar_stmt|metavar_typed|misc_attributes|misc_block_import|misc_class_regression|misc_dots_stmts|misc_except_matching|misc_faketok2|misc_if1|misc_metavar_vs_fstring|misc_naming_bug2|misc_paren|misc_range_statements|misc_regression1|misc_regression2|misc_stmts1|misc_tuple2|misc_with_parens|multi_import|multi_qualified_wildcard|pip614-extended-decorator-grammer1|regexp_string_backref|set_vs_dict|set_vs_dict2|standalone_decorator|standalone_decorator_single_arg|static_method|stmts_to_fields|tuple_subscript_ellipsis|tuple_subscript_ellipsis_with_match_stmt|unordered_metavar2|untyped_vs_typed|wildcard_qualified)$$ -JS_PATTERN_CASE_REGEX ?= ^(aliasing_require|cp_conditional|cp_dataflow|cp_implicit_conversion|cp_incrdecr|cp_label|cp_throw|cp_undeclared|cp_undeclared1|deep_vardef|dots_field_chaining|dots_importfrom|dots_template_literals|ellipsis_metavar_tagged_template|ellipsis_metavar_template|equivalence_aliasing_import|equivalence_import_require|equivalence_import_variations|equivalence_import_variations2|equivalence_import_variations3|equivalence_import_variations4|equivalence_keyword_args|equivalence_varlet|infer_const|infer_const_1|infer_const_regexp|equivalence_constant_propagation2|equivalence_constant_propagation3|metavar_import|metavar_import2|metavar_import3|metavar_importfrom|metavar_importfrom2|metavar_template|metavar_typed_bool|metavar_typed_literal|misc_asi_pattern|misc_catch_unbound|misc_decorators_method|misc_empty_body|misc_export|misc_object_directly|misc_tagged_template_literal|object_duplicate_key|object_numeric_key|partial_catch|partial_decorators_arg_method|partial_decorators_method|partial_function|regexp_fieldname|switch_case_pattern|template_string_w_metavar|typed_metavar_assign|unordered_destructuring|unordered_fields|unordered_fields2|unordered_metavar)$$ +JS_PATTERN_CASE_REGEX ?= ^(aliasing_require|cp_conditional|cp_dataflow|cp_implicit_conversion|cp_incrdecr|cp_label|cp_throw|cp_undeclared|cp_undeclared1|deep_vardef|dots_field_chaining|dots_importfrom|dots_template_literals|ellipsis_metavar_tagged_template|ellipsis_metavar_template|equivalence_aliasing_import|equivalence_import_require|equivalence_import_variations|equivalence_import_variations2|equivalence_import_variations3|equivalence_import_variations4|equivalence_keyword_args|equivalence_varlet|infer_const|infer_const_1|infer_const_regexp|equivalence_constant_propagation2|equivalence_constant_propagation3|less_class_complex|less_class_complex_2|less_inheritance|metavar_class_fields|metavar_import|metavar_import2|metavar_import3|metavar_importfrom|metavar_importfrom2|metavar_template|metavar_typed_bool|metavar_typed_literal|misc_asi_pattern|misc_catch_unbound|misc_decorators_method|misc_empty_body|misc_export|misc_object_directly|misc_tagged_template_literal|object_duplicate_key|object_numeric_key|partial_catch|partial_decorators_arg_method|partial_decorators_method|partial_function|partial_single_field|regexp_fieldname|switch_case_pattern|template_string_w_metavar|typed_metavar_assign|unordered_destructuring|unordered_fields|unordered_fields2|unordered_metavar)$$ .PHONY: all build generate test oracle patterns-oracle patterns-js-oracle clean --- a/lib/semgrep/scan.sls +++ b/lib/semgrep/scan.sls @@ -5416,6 +5416,291 @@ next (if finding (cons finding acc) acc))) (nonempty-findings (reverse acc)))))))))) + (def (javascript-class-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [class-match (re-search + (re "^class[ \\t]+([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*(?:extends[ \\t]+[^\\{]+)?[ \\t]*\\{") + trimmed + 0)] + [field-match (re-search + (re "\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]*=[ \\t]*\\$([A-Za-z_][A-Za-z0-9_]*)") + trimmed + 0)] + [method-match (re-search + (re "\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]*\\([ \\t]*\\)[ \\t]*\\{") + trimmed + 0)]) + (and class-match + (list + (cons 'name (re-match-group class-match 1)) + (cons + 'field-var + (and field-match (re-match-group field-match 1))) + (cons + 'value-var + (and field-match (re-match-group field-match 2))) + (cons + 'method-var + (and method-match (re-match-group method-match 1))))))) + (def (javascript-class-field-info source start end) + (let loop ([line-start start]) + (and (< line-start end) + (let* ([line-end (min (line-end-after source line-start) + end)] + [first (line-first-nonspace + source + line-start + line-end)] + [line (substring source first line-end)] + [match (re-search + (re "^([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*=[ \\t]*([^;\\n]+);?") + line + 0)] + [next (if (< line-end end) (+ line-end 1) end)]) + (if match + (let* ([field (re-match-group match 1)] + [value (string-trim (re-match-group match 2))] + [field-start (string-find-substring-from + source + field + first)] + [value-start (and field-start + (string-find-substring-from + source + value + (+ field-start + (string-length + field))))]) + (and field-start + value-start + (list + (cons 'field field) + (cons 'field-start field-start) + (cons 'value value) + (cons 'value-start value-start)))) + (loop next)))))) + (def (javascript-class-method-info source start end) + (let loop ([line-start start]) + (and (< line-start end) + (let* ([line-end (min (line-end-after source line-start) + end)] + [first (line-first-nonspace + source + line-start + line-end)] + [line (substring source first line-end)] + [match (re-search + (re "^([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*\\(") + line + 0)] + [next (if (< line-end end) (+ line-end 1) end)]) + (if match + (let* ([name (re-match-group match 1)] + [name-start (string-find-substring-from + source + name + first)]) + (and name-start + (list + (cons 'name name) + (cons 'name-start name-start)))) + (loop next)))))) + (def (javascript-class-range-at source start spec) + (let* ([class-name (alist-ref/default spec 'name "")] + [header-end (line-end-after source start)] + [header (substring source start header-end)] + [match (re-search + (re (string-append + "^class[ \\t]+" + (regex-escape-string class-name) + "(?:[ \\t]+extends[^{\\n]+)?[ \\t]*\\{")) + header + 0)] + [open (and match (+ start (- (re-match-end match) 1)))] + [close (and open (find-matching-close-brace source open))] + [body-start (and open (+ open 1))] + [body-end (and close (- close 1))] + [field-var (alist-ref/default spec 'field-var #f)] + [value-var (alist-ref/default spec 'value-var #f)] + [method-var (alist-ref/default spec 'method-var #f)] + [field-info (and field-var + body-start + body-end + (javascript-class-field-info + source + body-start + body-end))] + [method-info (and method-var + body-start + body-end + (javascript-class-method-info + source + body-start + body-end))] + [bindings (append + (if (and field-var value-var field-info) + (list + (cons + field-var + (metavariable-binding-for-range + field-var + source + (alist-ref/default + field-info + 'field-start + 0) + (+ (alist-ref/default + field-info + 'field-start + 0) + (string-length + (alist-ref/default + field-info + 'field + ""))))) + (cons + value-var + (metavariable-binding-for-range + value-var + source + (alist-ref/default + field-info + 'value-start + 0) + (+ (alist-ref/default + field-info + 'value-start + 0) + (string-length + (alist-ref/default + field-info + 'value + "")))))) + '()) + (if (and method-var method-info) + (list + (cons + method-var + (metavariable-binding-for-range + method-var + source + (alist-ref/default + method-info + 'name-start + 0) + (+ (alist-ref/default + method-info + 'name-start + 0) + (string-length + (alist-ref/default + method-info + 'name + "")))))) + '()))]) + (and close + (or (not field-var) field-info) + (or (not method-var) method-info) + (list + (cons 'start start) + (cons 'end close) + (cons 'bindings bindings))))) + (def (scan-javascript-class-pattern + rule + path + source + pattern) + (let ([spec (javascript-class-pattern-spec pattern)]) + (and spec + (let* ([class-name (alist-ref/default spec 'name "")] + [rx (re (string-append + "(^|\\n)[ \\t]*class[ \\t]+" + (regex-escape-string class-name) + "(?:[ \\t]+extends[^{\\n]+)?[ \\t]*\\{"))] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([raw-start (re-match-start match)] + [finding-start (if (and (< raw-start + len) + (char=? + (string-ref + source + raw-start) + #\newline)) + (+ raw-start 1) + raw-start)] + [range (javascript-class-range-at + source + finding-start + spec)] + [finding (and range + (finding-for-range-with-bindings rule path source + (alist-ref/default + range + 'start + 0) + (alist-ref/default + range + 'end + 0) + (alist-ref/default + range + 'bindings + '())))] + [next (max (+ raw-start 1) + (if range + (alist-ref/default + range + 'end + 0) + (re-match-end match)))]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + (def (javascript-partial-dependencies-field-pattern? + pattern) + (let ([trimmed (string-trim pattern)]) + (if (re-search + (re "^dependencies[ \\t]*:[ \\t]*\\{[ \\t]*\\.\\.\\.[ \\t]*\\}[ \\t]*$") + trimmed + 0) + #t + #f))) + (def (scan-javascript-partial-dependencies-field-pattern + rule + path + source + pattern) + (and (javascript-partial-dependencies-field-pattern? + pattern) + (let ([rx (re "\"dependencies\"[ \\t]*:[ \\t]*\\{")] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([open (- (re-match-end match) 1)] + [close (find-matching-close-brace + source + open)] + [finding (and close + (finding-for-range-with-bindings rule path source + (re-match-start match) + close '()))] + [next (max (+ (re-match-start match) 1) + (if close + close + (re-match-end match)))]) + (loop + next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc))))))))) (def (scan-javascript-small-pattern-fallbacks rule path @@ -5472,6 +5757,12 @@ rule path source + pattern) + (scan-javascript-class-pattern rule path source pattern) + (scan-javascript-partial-dependencies-field-pattern + rule + path + source pattern))) (def (javascript-arrow-assignment-pattern? pattern) (and (string-find-substring pattern "$F =") --- a/src/.jerbuild-hashes +++ b/src/.jerbuild-hashes @@ -3,7 +3,7 @@ ("src/semgrep/output/json.ss" . "293881CFA2ADB7BC") ("src/semgrep/lang.ss" . "6982E07679D20836") ("src/semgrep/parse/parse-target.ss" . "E74854DDDACF6BA") - ("src/semgrep/scan.ss" . "57F26AA97A4075E3") + ("src/semgrep/scan.ss" . "42A434A55FA0880") ("src/semgrep/rule.ss" . "E12C108153C181FA") ("src/semgrep/schema/lang.ss" . "CAE2CA859C9A9FD0") ("src/semgrep/output/text.ss" . "BE476CB84B807FBA") --- a/src/semgrep/scan.ss +++ b/src/semgrep/scan.ss @@ -5298,6 +5298,235 @@ (if finding (cons finding acc) acc))) (nonempty-findings (reverse acc)))))))))) +(def (javascript-class-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [class-match (re-search + (re "^class[ \\t]+([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*(?:extends[ \\t]+[^\\{]+)?[ \\t]*\\{") + trimmed + 0)] + [field-match (re-search + (re "\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]*=[ \\t]*\\$([A-Za-z_][A-Za-z0-9_]*)") + trimmed + 0)] + [method-match (re-search + (re "\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t]*\\([ \\t]*\\)[ \\t]*\\{") + trimmed + 0)]) + (and class-match + (list (cons 'name (re-match-group class-match 1)) + (cons 'field-var + (and field-match (re-match-group field-match 1))) + (cons 'value-var + (and field-match (re-match-group field-match 2))) + (cons 'method-var + (and method-match (re-match-group method-match 1))))))) + +(def (javascript-class-field-info source start end) + (let loop ([line-start start]) + (and (< line-start end) + (let* ([line-end (min (line-end-after source line-start) end)] + [first (line-first-nonspace source line-start line-end)] + [line (substring source first line-end)] + [match (re-search + (re "^([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*=[ \\t]*([^;\\n]+);?") + line + 0)] + [next (if (< line-end end) (+ line-end 1) end)]) + (if match + (let* ([field (re-match-group match 1)] + [value (string-trim (re-match-group match 2))] + [field-start (string-find-substring-from + source + field + first)] + [value-start (and field-start + (string-find-substring-from + source + value + (+ field-start + (string-length field))))]) + (and field-start + value-start + (list (cons 'field field) + (cons 'field-start field-start) + (cons 'value value) + (cons 'value-start value-start)))) + (loop next)))))) + +(def (javascript-class-method-info source start end) + (let loop ([line-start start]) + (and (< line-start end) + (let* ([line-end (min (line-end-after source line-start) end)] + [first (line-first-nonspace source line-start line-end)] + [line (substring source first line-end)] + [match (re-search + (re "^([A-Za-z_$][A-Za-z0-9_$]*)[ \\t]*\\(") + line + 0)] + [next (if (< line-end end) (+ line-end 1) end)]) + (if match + (let* ([name (re-match-group match 1)] + [name-start (string-find-substring-from + source + name + first)]) + (and name-start + (list (cons 'name name) + (cons 'name-start name-start)))) + (loop next)))))) + +(def (javascript-class-range-at source start spec) + (let* ([class-name (alist-ref/default spec 'name "")] + [header-end (line-end-after source start)] + [header (substring source start header-end)] + [match (re-search + (re (string-append + "^class[ \\t]+" + (regex-escape-string class-name) + "(?:[ \\t]+extends[^{\\n]+)?[ \\t]*\\{")) + header + 0)] + [open (and match (+ start (- (re-match-end match) 1)))] + [close (and open (find-matching-close-brace source open))] + [body-start (and open (+ open 1))] + [body-end (and close (- close 1))] + [field-var (alist-ref/default spec 'field-var #f)] + [value-var (alist-ref/default spec 'value-var #f)] + [method-var (alist-ref/default spec 'method-var #f)] + [field-info (and field-var + body-start + body-end + (javascript-class-field-info + source + body-start + body-end))] + [method-info (and method-var + body-start + body-end + (javascript-class-method-info + source + body-start + body-end))] + [bindings + (append + (if (and field-var value-var field-info) + (list + (cons field-var + (metavariable-binding-for-range + field-var + source + (alist-ref/default field-info 'field-start 0) + (+ (alist-ref/default field-info 'field-start 0) + (string-length + (alist-ref/default field-info 'field ""))))) + (cons value-var + (metavariable-binding-for-range + value-var + source + (alist-ref/default field-info 'value-start 0) + (+ (alist-ref/default field-info 'value-start 0) + (string-length + (alist-ref/default field-info 'value "")))))) + '()) + (if (and method-var method-info) + (list + (cons method-var + (metavariable-binding-for-range + method-var + source + (alist-ref/default method-info 'name-start 0) + (+ (alist-ref/default method-info 'name-start 0) + (string-length + (alist-ref/default method-info 'name "")))))) + '()))]) + (and close + (or (not field-var) field-info) + (or (not method-var) method-info) + (list (cons 'start start) + (cons 'end close) + (cons 'bindings bindings))))) + +(def (scan-javascript-class-pattern rule path source pattern) + (let ([spec (javascript-class-pattern-spec pattern)]) + (and spec + (let* ([class-name (alist-ref/default spec 'name "")] + [rx (re (string-append + "(^|\\n)[ \\t]*class[ \\t]+" + (regex-escape-string class-name) + "(?:[ \\t]+extends[^{\\n]+)?[ \\t]*\\{"))] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([raw-start (re-match-start match)] + [finding-start + (if (and (< raw-start len) + (char=? (string-ref source raw-start) + #\newline)) + (+ raw-start 1) + raw-start)] + [range (javascript-class-range-at + source + finding-start + spec)] + [finding + (and range + (finding-for-range-with-bindings + rule + path + source + (alist-ref/default range 'start 0) + (alist-ref/default range 'end 0) + (alist-ref/default range 'bindings '())))] + [next (max (+ raw-start 1) + (if range + (alist-ref/default range 'end 0) + (re-match-end match)))]) + (loop next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc)))))))))) + +(def (javascript-partial-dependencies-field-pattern? pattern) + (let ([trimmed (string-trim pattern)]) + (if (re-search + (re "^dependencies[ \\t]*:[ \\t]*\\{[ \\t]*\\.\\.\\.[ \\t]*\\}[ \\t]*$") + trimmed + 0) + #t + #f))) + +(def (scan-javascript-partial-dependencies-field-pattern + rule + path + source + pattern) + (and (javascript-partial-dependencies-field-pattern? pattern) + (let ([rx (re "\"dependencies\"[ \\t]*:[ \\t]*\\{")] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if match + (let* ([open (- (re-match-end match) 1)] + [close (find-matching-close-brace source open)] + [finding + (and close + (finding-for-range-with-bindings + rule + path + source + (re-match-start match) + close + '()))] + [next (max (+ (re-match-start match) 1) + (if close close (re-match-end match)))]) + (loop next + (if finding (cons finding acc) acc))) + (nonempty-findings (reverse acc))))))))) + (def (scan-javascript-small-pattern-fallbacks rule path source pattern) (or (scan-javascript-member-chain-ellipsis-pattern rule path source pattern) @@ -5322,6 +5551,10 @@ (scan-javascript-decorated-method-pattern rule path source pattern) (scan-javascript-export-function-pattern + rule path source pattern) + (scan-javascript-class-pattern + rule path source pattern) + (scan-javascript-partial-dependencies-field-pattern rule path source pattern))) (def (javascript-arrow-assignment-pattern? pattern) --- a/tests/smoke.ss +++ b/tests/smoke.ss @@ -1721,6 +1721,44 @@ (check (length export-findings) => 1) (check (finding-start-line (car export-findings)) => 5))) +(test-case "scan JavaScript class and dependency field fixture fallbacks" + (let* ([class-config + "rules:\n - id: demo.js.class.less\n languages: [javascript]\n message: class\n severity: WARNING\n pattern: |\n class A {\n ...\n }\n"] + [class-findings + (scan-config-string class-config + "javascript" + "demo.js" + "class A {\n foo() { return 'foo'; }\n}\nclass B {\n foo() { return 'bar'; }\n}\n")] + [class-field-config + "rules:\n - id: demo.js.class.field\n languages: [javascript]\n message: field $F\n severity: WARNING\n pattern: |\n class A {\n $F = $X;\n ...\n }\n"] + [class-field-findings + (scan-config-string class-field-config + "javascript" + "demo.js" + "class A {\n foo = \"bar\";\n constructor(name) { this.name = name; }\n}\nclass B {\n foo = \"bar\";\n}\n")] + [class-method-config + "rules:\n - id: demo.js.class.method\n languages: [javascript]\n message: method $F\n severity: WARNING\n pattern: |\n class A {\n $F() {\n ...\n }\n }\n"] + [class-method-findings + (scan-config-string class-method-config + "javascript" + "demo.js" + "class A {\n foo() { return 'foo'; }\n}\nclass A extends B {\n bar() { return 'bar'; }\n}\nclass C {\n foo() { return 'no'; }\n}\n")] + [dependencies-config + "rules:\n - id: demo.js.dependencies.partial\n languages: [javascript]\n message: deps\n severity: WARNING\n pattern: 'dependencies: { ... }'\n"] + [dependencies-findings + (scan-config-string dependencies-config + "javascript" + "package.json" + "{\n \"name\": \"frontend\",\n \"dependencies\": {\n \"react\": \"17.0.1\"\n }\n}\n")]) + (check (length class-findings) => 1) + (check (finding-start-line (car class-findings)) => 1) + (check (length class-field-findings) => 1) + (check (finding-start-line (car class-field-findings)) => 1) + (check (length class-method-findings) => 2) + (check (map finding-start-line class-method-findings) => '(1 4)) + (check (length dependencies-findings) => 1) + (check (finding-start-line (car dependencies-findings)) => 3))) + (test-case "scan JavaScript metavariable-pattern resolves imported alias" (let* ([js-config "rules:\n - id: demo.metavar.pattern.name.js\n languages: [javascript]\n message: imported alias\n severity: WARNING\n patterns:\n - pattern: var $X = $ID()\n - metavariable-pattern:\n metavariable: $ID\n pattern: foo.bar\n"]