Cover JavaScript object field fixtures
ober
0109749b78f47f4d7c6924c7a10c81328bf811c4
--- a/HANDOFF_OPUS_4_8.md +++ b/HANDOFF_OPUS_4_8.md @@ -1,22 +1,23 @@ # Opus 4.8 Handoff: jerboa-semgrep Semgrep Parity -Date: 2026-05-29 20:28 MDT +Date: 2026-05-29 21:33 MDT Workspace: `/Users/user/mine/jerboa-semgrep` Sibling upstream Semgrep checkout: `/Users/user/mine/semgrep` Packaged Semgrep oracle: `/Users/user/.local/bin/semgrep` Base HEAD before this checkpoint: -`40880ca Close Python pattern constant propagation fixtures` +`9490f84 Cover JavaScript equivalence fixtures` The user wants this project carried forward until the pure Jerboa port reaches Semgrep parity. Do not treat this handoff as completion. This handoff records -the clean Python `tests/patterns/python/*.sgrep` frontier and the first promoted -JavaScript `tests/patterns/js/*.sgrep` constant-propagation/equivalence slice. +the clean Python `tests/patterns/python/*.sgrep` frontier and the promoted +JavaScript `tests/patterns/js/*.sgrep` +constant-propagation/equivalence/object-field slice. Current headline: - Full Python pattern sweep: 164 passed / 0 mismatched. -- Full JavaScript pattern sweep: 56 passed / 76 mismatched, improved from - 41 passed / 91 mismatched by closing the first JS CP/equivalence slices. +- Full JavaScript pattern sweep: 63 passed / 69 mismatched, improved from + 56 passed / 76 mismatched by closing the JS object/destructuring slice. - Broad same-basename upstream sweep: 437 passed / 0 mismatched / 0 Jerboa errors, with 3 current-Semgrep oracle errors. @@ -73,7 +74,7 @@ make test Result: ```text -287 tests, 287 passed, 0 failed +288 tests, 288 passed, 0 failed ``` Focused constant/dataflow slice: @@ -132,7 +133,31 @@ git diff --check Result: no output and exit code 0. -Promoted JavaScript constant-propagation/equivalence oracle: +Focused JavaScript object/destructuring slice: + +```sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep PATTERN_LANGS=js CASE_REGEX='^(misc_object_directly|object_duplicate_key|object_numeric_key|unordered_fields|unordered_fields2|unordered_metavar|unordered_destructuring)$' LIST_MISMATCHES=1 MAX_DIFFS=100 tests/oracle/patterns-sweep.sh +``` + +Result: + +```text +patterns-sweep: 7 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 7 compared +``` + +Focused broad-regression fix: + +```sh +SEMGREP_CURRENT=/Users/user/.local/bin/semgrep LIST_MISMATCHES=1 MAX_DIFFS=20 CASE_REGEX='^misc_var_is_var$' tests/oracle/upstream-sweep.sh +``` + +Result: + +```text +upstream-sweep: 1 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 1 compared +``` + +Promoted JavaScript constant-propagation/equivalence/object-field oracle: ```sh SEMGREP_CURRENT=/Users/user/.local/bin/semgrep make patterns-js-oracle @@ -141,7 +166,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep make patterns-js-oracle Result: ```text -patterns-sweep: 15 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 15 compared +patterns-sweep: 22 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 22 compared ``` Full upstream JavaScript pattern sweep: @@ -153,7 +178,7 @@ SEMGREP_CURRENT=/Users/user/.local/bin/semgrep PATTERN_LANGS=js LIST_MISMATCHES= Result: ```text -patterns-sweep: 56 passed, 76 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared +patterns-sweep: 63 passed, 69 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared ``` Broad same-basename upstream sweep: @@ -188,8 +213,8 @@ The full Python pattern sweep also includes `cp_eval2`; it remains unpromoted in `PATTERN_CASE_REGEX`, but was explicitly regression-checked because the new wildcard string propagation initially overmatched it after `s += argv[1]`. -The latest checkpoint also adds `JS_PATTERN_CASE_REGEX` and a -`patterns-js-oracle` Make target. The promoted JS pattern slice is: +`JS_PATTERN_CASE_REGEX` and the `patterns-js-oracle` Make target now cover the +promoted JS pattern slice below: ```text cp_conditional @@ -207,8 +232,20 @@ equivalence_constant_propagation2 equivalence_constant_propagation3 equivalence_keyword_args equivalence_varlet +misc_object_directly +object_duplicate_key +object_numeric_key +unordered_destructuring +unordered_fields +unordered_fields2 +unordered_metavar ``` +`scan-javascript-varlet-equivalence-pattern` and the new declaration object +fallbacks now respect `options: let_is_var: false`; the broad guardrail +initially exposed an overmatch in `tests/rules/misc_var_is_var.js`, where +`var $X = 1;` must not match `let z = 1`. + ## Implementation Map All new scanner logic is in `src/semgrep/scan.ss` and generated into @@ -254,7 +291,7 @@ New scanner fallbacks: Wiring is in the Python fallback chain near the end of `scan.ss`, before the older exact constant-propagation fallbacks. -JavaScript constant-propagation support added in the latest checkpoint: +JavaScript constant-propagation support already in the current frontier: - `scan-javascript-cp-pattern-fallbacks` is wired into the JavaScript `pattern` path before structural matching. @@ -277,15 +314,39 @@ JavaScript constant-propagation support added in the latest checkpoint: - JS CP call/literal scanning ignores line comments so examples in comments, such as `new RegExp(/a/)`, do not become false positives. -JavaScript equivalence support added in the latest checkpoint: +JavaScript equivalence support already in the current frontier: - `scan-javascript-varlet-equivalence-pattern` matches `var $X = ...` patterns against `var`, `let`, and `const` declarations with Semgrep-compatible - declaration ranges. + declaration ranges, unless the rule explicitly sets `let_is_var: false`. - `scan-javascript-object-call-equivalence-pattern` matches object argument fields independent of order and allows extra fields, covering `foo({kwd2:$X, kwd1:$Y})` against `foo({kwd1:1, kwd2:2, kwd3:3})`. +JavaScript object/destructuring support added in this checkpoint: + +- `scan-javascript-object-pattern-fallbacks` is wired into the JavaScript + `pattern` path after the object-call equivalence fallback and before JS + constant propagation. +- `javascript-field-records` parses comma-separated object/destructuring + members while respecting nested brackets/braces/parens and quoted strings. +- Object key normalization treats quoted and unquoted keys such as `"bar"` and + `bar` as equal, and treats numeric keys such as `0x1`, `1`, and `1.0` as + equal when their numeric value matches. +- Direct object literal matching covers required fields with extra fields + allowed, including `{ fld1: $X }` against `{ fld2: 4, fld1: 5 }`. +- Duplicate-key matching covers the upstream `$X: $A ... $X: $B` pattern for + string and numeric key equivalence. +- Declaration object matching covers `var $F = { ... }` with unordered + required fields, extra fields allowed, fixed values, and metavariable field + names such as `$FLD: 2`. +- Destructuring declaration matching covers `var { one, two } = foo;` against + declarations where the required names appear in any order with additional + names allowed. +- Declaration-style object/destructuring fallbacks use the same + `let_is_var: false` option check as var/let equivalence so rule-level + semantics are preserved. + ## Smoke Coverage Added `tests/smoke.ss` extends `scan Python constant propagation pattern fixture @@ -298,16 +359,21 @@ fallbacks` with checks for: - R/L-value propagation from `a = "foo"` into `b = a`, `c[a]`, dict values, and `return a`. -The latest checkpoint adds a new top-level smoke case, +The current smoke suite includes a top-level smoke case, `scan JavaScript constant propagation pattern fixture fallbacks`, covering template/concat string calls, conditional string calls, subscript propagation, throw-unreachable return propagation, and increment invalidation. -The latest checkpoint adds a new top-level smoke case, +The current smoke suite includes a top-level smoke case, `scan JavaScript equivalence pattern fixture fallbacks`, covering var/let/const declaration equivalence and object-call field-order equivalence. -The smoke suite count is now 287. +This checkpoint adds a new top-level smoke case, +`scan JavaScript object field pattern fixture fallbacks`, covering direct object +field matching, numeric key normalization, duplicate keys, unordered object +declarations, and unordered destructuring declarations. + +The smoke suite count is now 288. ## Current Frontier @@ -318,25 +384,29 @@ patterns-sweep: 164 passed, 0 mismatched, 0 jerboa errors, 0 current errors, 0 s ``` The full JavaScript `.sgrep` pattern fixture sweep is not clean yet, but the -first JS constant-propagation/equivalence slice is promoted and green: +promoted JS constant-propagation/equivalence/object-field slice is green: ```text -patterns-sweep: 56 passed, 76 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared +patterns-sweep: 63 passed, 69 mismatched, 0 jerboa errors, 0 current errors, 0 skipped, 132 compared ``` Do not mark the overall goal complete from this alone. Next high-value work: 1. Continue the JavaScript pattern frontier. The next concentrated clusters from the full JS sweep are deep/caching expression matching, import - equivalence variations, JSX/XML range parity, and unordered object/XML - fields. + equivalence/import metavariable variations, JSX/XML dots/less/metavariable + range parity, tagged/template literal metavariables, and arrow/class/partial + pattern shapes. 2. Keep `cp_array_destructure` as a regression guard: the JS equality fallback must invalidate constants after destructuring reassignment and avoid matching `imageOrNull == null`. -3. Decide whether to promote all 164 Python pattern fixtures into +3. Keep `misc_var_is_var` as a regression guard for `let_is_var: false` when + touching declaration equivalence or declaration object/destructuring + fallbacks. +4. Decide whether to promote all 164 Python pattern fixtures into `PATTERN_CASE_REGEX` or keep `make patterns-oracle` at the curated 116-case runtime. -4. Rerun the broad upstream same-basename sweep after any fallback that can +5. Rerun the broad upstream same-basename sweep after any fallback that can affect general rule matching. ## Operational Notes --- a/Makefile +++ b/Makefile @@ -5,7 +5,7 @@ SCHEME ?= $(HOST_SCHEME) JERBUILD ?= $(if $(wildcard $(JERBOA_HOME)/jerbuild),$(JERBOA_HOME)/jerbuild,$(SCHEME) --libdirs "$(JERBOA_HOME)/lib" --script "$(JERBOA_HOME)/jerbuild.ss") LIBDIRS = lib:$(JERBOA_TREESITTER_HOME)/lib:$(JERBOA_HOME)/lib PATTERN_CASE_REGEX ?= ^(ac_matching_dots|ac_matching_dots1|ac_matching_explosion|ac_matching_explosion1|ac_matching_explosion2|ac_matching_explosion3|ac_matching_free|ac_matching_free1|ac_matching_if|ac_matching_if1|ac_matching_mvars|ac_matching_mvars1|ac_matching_mvars2|aliasing_attribute|assoc_matching_bug|assoc_matching_dots|assoc_matching_dots1|assoc_matching_explosion|assoc_matching_explosion1|assoc_matching_explosion2|assoc_matching_explosion3|assoc_matching_free|assoc_matching_free1|assoc_matching_mvars|assoc_matching_mvars1|constprop_dataflow|constprop_used_twice|cp_concat|cp_exception|cp_label|cp_label1|cp_method_call|cp_python_mult_string|cp_python_mult_string1|cp_rlval|cp_string_mvar|cp_strings|cp_with|cp_yield|deep_cond|deep_expr_operator|deep_expr_vs_statement|deep_exprstmt|deep_fake_dataflow|deep_try|dict_ellipsis|dots_attribute|dots_expr_plus|dots_expr_plus_string|dots_fstring|dots_fstring_with_match_stmt|dots_inherit|dots_list|dots_nested_stmts|dots_stmts|equivalence_constant_propagation|equivalence_f_string|equivalence_f_string_2|equivalence_f_string_3|equivalence_f_string_4|equivalence_interpolated_str|equivalence_interpolated_str2|equivalence_keyword_args|equivalence_naming_import|import_metavar|import_negatives|import_negatives2|imports|index_tuple|less_attributes|less_catch|less_catch_multiple|less_inherits|less_typehint|less_typehint2|matching_if_expr|metavar_anno|metavar_anno_fqn|metavar_class_def|metavar_dict|metavar_equality_param_vs_use|metavar_equality_var|metavar_func_def|metavar_stmt|metavar_typed|misc_attributes|misc_block_import|misc_class_regression|misc_dots_stmts|misc_except_matching|misc_faketok2|misc_if1|misc_metavar_vs_fstring|misc_naming_bug2|misc_paren|misc_range_statements|misc_regression1|misc_regression2|misc_stmts1|misc_tuple2|misc_with_parens|multi_import|multi_qualified_wildcard|pip614-extended-decorator-grammer1|regexp_string_backref|set_vs_dict|set_vs_dict2|standalone_decorator|standalone_decorator_single_arg|static_method|stmts_to_fields|tuple_subscript_ellipsis|tuple_subscript_ellipsis_with_match_stmt|unordered_metavar2|untyped_vs_typed|wildcard_qualified)$$ -JS_PATTERN_CASE_REGEX ?= ^(cp_conditional|cp_dataflow|cp_implicit_conversion|cp_incrdecr|cp_label|cp_throw|cp_undeclared|cp_undeclared1|equivalence_keyword_args|equivalence_varlet|infer_const|infer_const_1|infer_const_regexp|equivalence_constant_propagation2|equivalence_constant_propagation3)$$ +JS_PATTERN_CASE_REGEX ?= ^(cp_conditional|cp_dataflow|cp_implicit_conversion|cp_incrdecr|cp_label|cp_throw|cp_undeclared|cp_undeclared1|equivalence_keyword_args|equivalence_varlet|infer_const|infer_const_1|infer_const_regexp|equivalence_constant_propagation2|equivalence_constant_propagation3|misc_object_directly|object_duplicate_key|object_numeric_key|unordered_destructuring|unordered_fields|unordered_fields2|unordered_metavar)$$ .PHONY: all build generate test oracle patterns-oracle patterns-js-oracle clean --- a/lib/semgrep/scan.sls +++ b/lib/semgrep/scan.sls @@ -4442,8 +4442,10 @@ (and spec (let* ([rhs (alist-ref/default spec 'rhs "")] [mvar (alist-ref/default spec 'metavariable "")] - [rx (re (string-append "(^|\\n)[ \\t]*(?:var|let|const)[ \\t]+" - "([A-Za-z_$][A-Za-z0-9_$]*)" + [declaration-keyword (javascript-declaration-regex-keyword + rule)] + [rx (re (string-append "(^|\\n)[ \\t]*" declaration-keyword + "[ \\t]+" "([A-Za-z_$][A-Za-z0-9_$]*)" "[ \\t]*=[ \\t]*" (regex-escape-string rhs) "[ \\t]*;?"))]) (let loop ([start 0] [acc '()]) @@ -4496,6 +4498,10 @@ (loop next (if finding (cons finding acc) acc)))))))))) + (def (javascript-declaration-regex-keyword rule) + (if (rule-option-explicit-false? rule 'let_is_var) + "var" + "(?:var|let|const)")) (def (javascript-object-call-equivalence-pattern-spec pattern) (let* ([trimmed (string-trim pattern)] @@ -4679,6 +4685,862 @@ (loop next (if finding (cons finding acc) acc)))))))))) + (def (javascript-skip-whitespace-forward source start end) + (let loop ([i start]) + (if (and (< i end) (char-whitespace? (string-ref source i))) + (loop (+ i 1)) + i))) + (def (javascript-skip-whitespace-backward-exclusive + source + start + end) + (let loop ([i (- end 1)]) + (cond + [(< i start) start] + [(char-whitespace? (string-ref source i)) (loop (- i 1))] + [else (+ i 1)]))) + (def (javascript-top-level-char-index source ch start end) + (let loop ([i start] + [paren-depth 0] + [bracket-depth 0] + [brace-depth 0] + [state 'normal] + [escaped? #f]) + (cond + [(>= i end) #f] + [(eq? state 'normal) + (let ([current (string-ref source i)]) + (cond + [(and (char=? current ch) + (= paren-depth 0) + (= bracket-depth 0) + (= brace-depth 0)) + i] + [(char=? current #\") + (loop (+ i 1) paren-depth bracket-depth brace-depth + 'double #f)] + [(char=? current #\') + (loop (+ i 1) paren-depth bracket-depth brace-depth + 'single #f)] + [(char=? current #\`) + (loop (+ i 1) paren-depth bracket-depth brace-depth + 'backtick #f)] + [(char=? current #\() + (loop (+ i 1) (+ paren-depth 1) bracket-depth brace-depth + state #f)] + [(char=? current #\)) + (loop (+ i 1) (max 0 (- paren-depth 1)) bracket-depth + brace-depth state #f)] + [(char=? current #\[) + (loop (+ i 1) paren-depth (+ bracket-depth 1) brace-depth + state #f)] + [(char=? current #\]) + (loop (+ i 1) paren-depth (max 0 (- bracket-depth 1)) + brace-depth state #f)] + [(char=? current #\{) + (loop (+ i 1) paren-depth bracket-depth (+ brace-depth 1) + state #f)] + [(char=? current #\}) + (loop (+ i 1) paren-depth bracket-depth + (max 0 (- brace-depth 1)) state #f)] + [else + (loop (+ i 1) paren-depth bracket-depth brace-depth state + #f)]))] + [escaped? + (loop (+ i 1) paren-depth bracket-depth brace-depth state + #f)] + [(char=? (string-ref source i) #\\) + (loop (+ i 1) paren-depth bracket-depth brace-depth state + #t)] + [(and (eq? state 'double) + (char=? (string-ref source i) #\")) + (loop (+ i 1) paren-depth bracket-depth brace-depth 'normal + #f)] + [(and (eq? state 'single) + (char=? (string-ref source i) #\')) + (loop (+ i 1) paren-depth bracket-depth brace-depth 'normal + #f)] + [(and (eq? state 'backtick) + (char=? (string-ref source i) #\`)) + (loop (+ i 1) paren-depth bracket-depth brace-depth 'normal + #f)] + [else + (loop (+ i 1) paren-depth bracket-depth brace-depth state + #f)]))) + (def (javascript-field-record source raw-start raw-end) + (let* ([start (javascript-skip-whitespace-forward + source + raw-start + raw-end)] + [end (javascript-skip-whitespace-backward-exclusive + source + start + raw-end)]) + (and (< start end) + (let* ([colon (javascript-top-level-char-index + source + #\: + start + end)] + [key-start0 start] + [key-end0 (or colon end)] + [key-start (javascript-skip-whitespace-forward + source + key-start0 + key-end0)] + [key-end (javascript-skip-whitespace-backward-exclusive + source + key-start + key-end0)] + [value-start0 (and colon (+ colon 1))] + [value-start (and value-start0 + (javascript-skip-whitespace-forward + source + value-start0 + end))] + [value-end (and value-start + (javascript-skip-whitespace-backward-exclusive + source + value-start + end))]) + (list (cons 'start start) (cons 'end end) + (cons 'text (substring source start end)) + (cons 'key-start key-start) (cons 'key-end key-end) + (cons 'key (substring source key-start key-end)) + (cons 'value-start value-start) + (cons 'value-end value-end) + (cons + 'value + (and value-start + value-end + (substring source value-start value-end)))))))) + (def (javascript-field-records source start end) + (let loop ([i start] + [item-start start] + [paren-depth 0] + [bracket-depth 0] + [brace-depth 0] + [state 'normal] + [escaped? #f] + [acc '()]) + (cond + [(>= i end) + (let ([record (javascript-field-record + source + item-start + end)]) + (reverse (if record (cons record acc) acc)))] + [(eq? state 'normal) + (let ([ch (string-ref source i)]) + (cond + [(and (char=? ch #\,) + (= paren-depth 0) + (= bracket-depth 0) + (= brace-depth 0)) + (let ([record (javascript-field-record + source + item-start + i)]) + (loop (+ i 1) (+ i 1) paren-depth bracket-depth + brace-depth state #f + (if record (cons record acc) acc)))] + [(char=? ch #\") + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'double #f acc)] + [(char=? ch #\') + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'single #f acc)] + [(char=? ch #\`) + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'backtick #f acc)] + [(char=? ch #\() + (loop (+ i 1) item-start (+ paren-depth 1) bracket-depth + brace-depth state #f acc)] + [(char=? ch #\)) + (loop (+ i 1) item-start (max 0 (- paren-depth 1)) + bracket-depth brace-depth state #f acc)] + [(char=? ch #\[) + (loop (+ i 1) item-start paren-depth (+ bracket-depth 1) + brace-depth state #f acc)] + [(char=? ch #\]) + (loop (+ i 1) item-start paren-depth + (max 0 (- bracket-depth 1)) brace-depth state #f acc)] + [(char=? ch #\{) + (loop (+ i 1) item-start paren-depth bracket-depth + (+ brace-depth 1) state #f acc)] + [(char=? ch #\}) + (loop (+ i 1) item-start paren-depth bracket-depth + (max 0 (- brace-depth 1)) state #f acc)] + [else + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth state #f acc)]))] + [escaped? + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth state #f acc)] + [(char=? (string-ref source i) #\\) + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth state #t acc)] + [(and (eq? state 'double) + (char=? (string-ref source i) #\")) + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'normal #f acc)] + [(and (eq? state 'single) + (char=? (string-ref source i) #\')) + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'normal #f acc)] + [(and (eq? state 'backtick) + (char=? (string-ref source i) #\`)) + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth 'normal #f acc)] + [else + (loop (+ i 1) item-start paren-depth bracket-depth + brace-depth state #f acc)]))) + (def (javascript-object-key-text key) + (let* ([trimmed (string-trim key)] + [len (string-length trimmed)]) + (cond + [(quoted-string? trimmed) (unquote-string trimmed)] + [(and (>= len 2) + (char=? (string-ref trimmed 0) #\`) + (char=? (string-ref trimmed (- len 1)) #\`)) + (substring trimmed 1 (- len 1))] + [else trimmed]))) + (def (javascript-normalized-object-key key) + (let* ([text (javascript-object-key-text key)] + [number (parse-number-literal text #f)]) + (if number (cons 'number number) (cons 'text text)))) + (def (javascript-normalized-object-key=? left right) + (cond + [(and (eq? (car left) 'number) (eq? (car right) 'number)) + (= (cdr left) (cdr right))] + [(and (eq? (car left) 'text) (eq? (car right) 'text)) + (string=? (cdr left) (cdr right))] + [else #f])) + (def (javascript-object-key-spec text) + (let ([trimmed (string-trim text)]) + (if (and (> (string-length trimmed) 0) + (char=? (string-ref trimmed 0) #\$)) + (list + (cons 'kind 'metavariable) + (cons 'name (normalize-metavariable-name trimmed))) + (list + (cons 'kind 'fixed) + (cons 'key (javascript-normalized-object-key trimmed)))))) + (def (javascript-object-value-spec text) + (let ([trimmed (string-trim text)]) + (cond + [(string=? trimmed "...") (list (cons 'kind 'ellipsis))] + [(and (> (string-length trimmed) 0) + (char=? (string-ref trimmed 0) #\$)) + (list + (cons 'kind 'metavariable) + (cons 'name (normalize-metavariable-name trimmed)))] + [else (list (cons 'kind 'fixed) (cons 'text trimmed))]))) + (def (javascript-object-requirement-from-record record) + (let ([value (alist-ref/default record 'value #f)]) + (and value + (let ([key (alist-ref/default record 'key "")]) + (list + (cons 'key (javascript-object-key-spec key)) + (cons 'value (javascript-object-value-spec value))))))) + (def (javascript-object-requirements records) + (let loop ([xs records] [acc '()]) + (cond + [(null? xs) (reverse acc)] + [(javascript-object-requirement-from-record (car xs)) => + (lambda (requirement) + (loop (cdr xs) (cons requirement acc)))] + [else (loop (cdr xs) acc)]))) + (def (javascript-literal-equivalent? expected actual) + (let* ([expected-trimmed (string-trim expected)] + [actual-trimmed (string-trim actual)] + [expected-number (parse-number-literal expected-trimmed #f)] + [actual-number (parse-number-literal actual-trimmed #f)]) + (cond + [(and expected-number actual-number) + (= expected-number actual-number)] + [(and (quoted-string? expected-trimmed) + (quoted-string? actual-trimmed)) + (string=? + (unquote-string expected-trimmed) + (unquote-string actual-trimmed))] + [else (string=? expected-trimmed actual-trimmed)]))) + (def (javascript-binding-text=? left right) + (string=? + (metavariable-binding-text left) + (metavariable-binding-text right))) + (def (javascript-merge-bindings new-bindings bindings) + (let loop ([xs new-bindings] [acc bindings]) + (cond + [(null? xs) acc] + [else + (let* ([entry (car xs)] [existing (assoc (car entry) acc)]) + (cond + [(not existing) (loop (cdr xs) (cons entry acc))] + [(javascript-binding-text=? (cdr existing) (cdr entry)) + (loop (cdr xs) acc)] + [else #f]))]))) + (def (javascript-field-key-bindings spec record source) + (let ([kind (alist-ref/default spec 'kind #f)]) + (case kind + [(fixed) + (let* ([expected (alist-ref/default spec 'key #f)] + [actual (javascript-normalized-object-key + (alist-ref/default record 'key ""))]) + (and expected + (javascript-normalized-object-key=? expected actual) + '()))] + [(metavariable) + (let* ([name (alist-ref/default spec 'name "")] + [start (alist-ref/default record 'key-start #f)] + [end (alist-ref/default record 'key-end #f)] + [text (and start end (substring source start end))]) + (and text + (list + (cons + name + (make-regex-capture-binding name text source start + end)))))] + [else #f]))) + (def (javascript-field-value-bindings spec record source) + (let ([kind (alist-ref/default spec 'kind #f)]) + (case kind + [(ellipsis) '()] + [(fixed) + (let ([expected (alist-ref/default spec 'text "")] + [actual (alist-ref/default record 'value #f)]) + (and actual + (javascript-literal-equivalent? expected actual) + '()))] + [(metavariable) + (let* ([name (alist-ref/default spec 'name "")] + [start (alist-ref/default record 'value-start #f)] + [end (alist-ref/default record 'value-end #f)] + [text (and start end (substring source start end))]) + (and text + (list + (cons + name + (make-regex-capture-binding name text source start + end)))))] + [else #f]))) + (def (javascript-object-requirement-bindings + requirement + record + source) + (let* ([key-spec (alist-ref/default requirement 'key #f)] + [value-spec (alist-ref/default requirement 'value #f)] + [key-bindings (and key-spec + (javascript-field-key-bindings + key-spec + record + source))] + [value-bindings (and key-bindings + value-spec + (javascript-field-value-bindings + value-spec + record + source))]) + (and key-bindings + value-bindings + (javascript-merge-bindings value-bindings key-bindings)))) + (def (javascript-object-requirements-bindings + requirements + fields + source) + (let loop ([remaining requirements] + [available fields] + [bindings '()]) + (cond + [(null? remaining) bindings] + [else + (let ([requirement (car remaining)]) + (let try ([candidates available]) + (cond + [(null? candidates) #f] + [else + (let* ([record (car candidates)] + [record-bindings (javascript-object-requirement-bindings + requirement + record + source)] + [merged (and record-bindings + (javascript-merge-bindings + record-bindings + bindings))] + [next (and merged + (loop + (cdr remaining) + (remove-first-eq record available) + merged))]) + (or next (try (cdr candidates))))])))]))) + (def (javascript-repeated-metavariable-key-pattern? records) + (let loop ([xs records] [seen '()]) + (cond + [(null? xs) #f] + [else + (let* ([record (car xs)] + [value (alist-ref/default record 'value #f)] + [key (alist-ref/default record 'key "")] + [trimmed (string-trim key)]) + (if (and value + (> (string-length trimmed) 0) + (char=? (string-ref trimmed 0) #\$)) + (let ([name (normalize-metavariable-name trimmed)]) + (if (assoc name seen) + #t + (loop (cdr xs) (cons (cons name #t) seen)))) + (loop (cdr xs) seen)))]))) + (def (javascript-object-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [len (string-length trimmed)]) + (and (outer-pair? trimmed #\{ #\}) + (let* ([records (javascript-field-records + trimmed + 1 + (- len 1))] + [requirements (javascript-object-requirements + records)]) + (cond + [(javascript-repeated-metavariable-key-pattern? records) + (list (cons 'kind 'duplicate-key))] + [(not (null? requirements)) + (list + (cons 'kind 'fields) + (cons 'requirements requirements))] + [else #f]))))) + (def (javascript-object-field-key record) + (let ([value (alist-ref/default record 'value #f)]) + (and value + (javascript-normalized-object-key + (alist-ref/default record 'key ""))))) + (def (javascript-key-in-list? key keys) + (and (not (null? keys)) + (or (javascript-normalized-object-key=? key (car keys)) + (javascript-key-in-list? key (cdr keys))))) + (def (javascript-object-has-duplicate-key? fields) + (let loop ([xs fields] [seen '()]) + (cond + [(null? xs) #f] + [else + (let ([key (javascript-object-field-key (car xs))]) + (cond + [(not key) (loop (cdr xs) seen)] + [(javascript-key-in-list? key seen) #t] + [else (loop (cdr xs) (cons key seen))]))]))) + (def (javascript-object-literal-open? source open) + (let ([before (previous-nonspace-index source open)]) + (or (not before) + (let ([ch (string-ref source before)]) + (or (char=? ch #\=) + (char=? ch #\() + (char=? ch #\,) + (char=? ch #\[) + (char=? ch #\:))) + (let ([word (previous-word-before source open)]) + (and word (string=? word "return")))))) + (def (scan-javascript-object-literal-field-pattern + rule + path + source + pattern) + (let ([spec (javascript-object-pattern-spec pattern)]) + (and spec + (let ([kind (alist-ref/default spec 'kind #f)] + [requirements (alist-ref/default + spec + 'requirements + '())] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([open (string-find-substring-from + source + "{" + start)]) + (if (not open) + (nonempty-findings (reverse acc)) + (let* ([close (find-matching-close-brace + source + open)] + [literal? (and close + (javascript-object-literal-open? + source + open))] + [fields (and literal? + (javascript-field-records + source + (+ open 1) + (- close 1)))] + [bindings (and fields + (case kind + [(duplicate-key) + (and (javascript-object-has-duplicate-key? + fields) + '())] + [(fields) + (javascript-object-requirements-bindings + requirements + fields + source)] + [else #f]))] + [finding (and bindings + (finding-for-range-with-bindings rule path source open + close bindings))] + [next (if close + (max (+ open 1) close) + (+ open 1))]) + (loop + next + (if finding + (cons finding acc) + acc))))))))))) + (def (javascript-var-object-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [match (re-search + (re "^var[ \\t\\r\\n]+\\$([A-Za-z_][A-Za-z0-9_]*)[ \\t\\r\\n]*=[ \\t\\r\\n]*\\{") + trimmed + 0)]) + (and match + (let* ([name (re-match-group match 1)] + [open (- (re-match-end match) 1)] + [close (find-matching-close-brace trimmed open)] + [tail-start (and close + (javascript-skip-whitespace-forward + trimmed + close + (string-length trimmed)))] + [tail-ok? (and close + (or (= tail-start + (string-length trimmed)) + (and (< tail-start + (string-length trimmed)) + (char=? + (string-ref + trimmed + tail-start) + #\;) + (= (javascript-skip-whitespace-forward + trimmed + (+ tail-start 1) + (string-length trimmed)) + (string-length + trimmed)))))] + [records (and tail-ok? + (javascript-field-records + trimmed + (+ open 1) + (- close 1)))] + [requirements (and records + (javascript-object-requirements + records))]) + (and tail-ok? + requirements + (not (null? requirements)) + (list + (cons 'metavariable name) + (cons 'requirements requirements))))))) + (def (scan-javascript-var-object-field-pattern + rule + path + source + pattern) + (let ([spec (javascript-var-object-pattern-spec pattern)]) + (and spec + (let* ([name (alist-ref/default spec 'metavariable "")] + [requirements (alist-ref/default + spec + 'requirements + '())] + [rx (re (string-append + "(^|\\n)[ \\t]*" + (javascript-declaration-regex-keyword rule) + "[ \\t]+([A-Za-z_$][A-Za-z0-9_$]*)" + "[ \\t]*=[ \\t]*\\{"))] + [len (string-length source)]) + (let loop ([start 0] [acc '()]) + (if (> start len) + (nonempty-findings (reverse acc)) + (let ([match (re-search rx source start)]) + (if (not match) + (nonempty-findings (reverse acc)) + (let* ([decl-start (if (and (< (re-match-start + match) + (string-length + source)) + (char=? + (string-ref + source + (re-match-start + match)) + #\newline)) + (+ (re-match-start + match) + 1) + (re-match-start match))] + [first (line-first-nonspace + source + decl-start + (line-end-after + source + decl-start))] + [var-name (re-match-group match 2)] + [open (- (re-match-end match) 1)] + [close (find-matching-close-brace + source + open)] + [fields (and close + (javascript-field-records + source + (+ open 1) + (- close 1)))] + [field-bindings (and fields + (javascript-object-requirements-bindings + requirements + fields + source))] + [name-start (and field-bindings + (string-find-substring-from + source + var-name + first))] + [name-binding (and name-start + (make-regex-capture-binding name var-name + source name-start + (+ name-start + (string-length + var-name))))] + [bindings (and field-bindings + name-binding + (javascript-merge-bindings + (list + (cons + name + name-binding)) + field-bindings))] + [finding (and bindings + (finding-for-range-with-bindings rule path source first + close bindings))] + [next (if close + (max (+ (re-match-start match) + 1) + close) + (re-match-end match))]) + (loop + next + (if finding + (cons finding acc) + acc))))))))))) + (def (javascript-destructuring-name-items source start end) + (let loop ([records (javascript-field-records + source + start + end)] + [acc '()]) + (cond + [(null? records) (reverse acc)] + [else + (let* ([record (car records)] + [text (string-trim + (alist-ref/default record 'text ""))]) + (if (python-simple-identifier? text) + (loop (cdr records) (cons text acc)) + (loop (cdr records) acc)))]))) + (def (javascript-name-in-list? name names) + (and (not (null? names)) + (or (string=? name (car names)) + (javascript-name-in-list? name (cdr names))))) + (def (javascript-names-subset? required actual) + (all? + (lambda (name) (javascript-name-in-list? name actual)) + required)) + (def (javascript-var-destructuring-pattern-spec pattern) + (let* ([trimmed (string-trim pattern)] + [match (re-search (re "^var[ \\t\\r\\n]+\\{") trimmed 0)]) + (and match + (let* ([open (- (re-match-end match) 1)] + [close (find-matching-close-brace trimmed open)] + [names (and close + (javascript-destructuring-name-items + trimmed + (+ open 1) + (- close 1)))] + [after-close (and close + (javascript-skip-whitespace-forward + trimmed + close + (string-length trimmed)))] + [equals? (and after-close + (< after-close (string-length trimmed)) + (char=? + (string-ref trimmed after-close) + #\=))] + [rhs-start (and equals? + (javascript-skip-whitespace-forward + trimmed + (+ after-close 1) + (string-length trimmed)))] + [semicolon (and rhs-start + (char-index-from + trimmed + #\; + rhs-start + (string-length trimmed)))] + [rhs-end0 (and rhs-start + (or semicolon + (string-length trimmed)))] + [rhs-end (and rhs-start + rhs-end0 + (javascript-skip-whitespace-backward-exclusive + trimmed + rhs-start + rhs-end0))] + [tail-ok? (and rhs-start + rhs-end + (or (not semicolon) + (= (javascript-skip-whitespace-forward + trimmed + (+ semicolon 1) + (string-length trimmed)) + (string-length trimmed))))]) + (and tail-ok? + (not (null? names)) + (list + (cons 'names names) + (cons + 'rhs