Port Rule 12 (DGA domain detection) to pure Jerboa
ober
31db6071cc5792373387aaf3b3201933c8bfe471
--- a/Makefile +++ b/Makefile @@ -8,7 +8,7 @@ SCHEME ?= $(JERBOA)/.chez/bin/scheme BUILD ?= build/rust TYPED := $(wildcard typed/*.ss) -.PHONY: rust test ffi-demo kernels-check triage-check triage-store-check analytics-check detect-check storage-check threats-check geoip-check sigma-check yaml-rules-check buffer-check dns-sniffer-check suspicious-check netconn-check kernmod-check selinux-check container-check dns-servers-check sensitive-path-check dtrace-parse-check proc-linux-check freebsd-parse-check event-meta-check config-check event-danger-check persistence-check file-change-check webshell-check platform-mounts-check analyze-cli-check collector-cli-check event-summary-check ioc-check frame-check correlate-check revshell-check cron-check logtamper-check detection-rules-check ipaddr-check auth-check lolbin-check checks clean +.PHONY: rust test ffi-demo kernels-check triage-check triage-store-check analytics-check detect-check storage-check threats-check geoip-check sigma-check yaml-rules-check buffer-check dns-sniffer-check suspicious-check netconn-check kernmod-check selinux-check container-check dns-servers-check sensitive-path-check dtrace-parse-check proc-linux-check freebsd-parse-check event-meta-check config-check event-danger-check persistence-check file-change-check webshell-check platform-mounts-check analyze-cli-check collector-cli-check event-summary-check ioc-check frame-check correlate-check revshell-check cron-check logtamper-check detection-rules-check ipaddr-check auth-check lolbin-check dga-check checks clean # Combined libdir path so sibling libraries `(jsecmon ...)` resolve to ./jsecmon # (a second --libdirs would replace, not append, the jerboa one). LIBDIRS := "$(JERBOA)/lib:$(CURDIR)" @@ -171,6 +171,13 @@ lolbin-check: rust cd $(BUILD) && cargo build --release $(SCHEME) --libdirs $(LIBDIRS) --script examples/lolbin_check.ss +# DGA detection (secmon src/dga.rs + storage Rule 12 detect_dga_domain). The +# typed kernel owns scoring; the untyped (jsecmon dga) decodes the reason +# bitmask + builds verdicts and anomaly seeds. Needs the dylib, like lolbin. +dga-check: rust + cd $(BUILD) && cargo build --release + $(SCHEME) --libdirs $(LIBDIRS) --script examples/dga_check.ss + # FreeBSD line parsers (secmon src/platform/freebsd.rs): parse_kldstat_line # (kld module rows) and parse_address (sockstat/netstat host:port, with the # "*" wildcard and ipv6 bracket forms). Ports are DECIMAL here, unlike Linux's @@ -353,6 +360,7 @@ checks: kernels-check $(SCHEME) --libdirs $(LIBDIRS) --script examples/ipaddr_check.ss $(SCHEME) --libdirs $(LIBDIRS) --script examples/auth_check.ss $(SCHEME) --libdirs $(LIBDIRS) --script examples/lolbin_check.ss + $(SCHEME) --libdirs $(LIBDIRS) --script examples/dga_check.ss clean: rm -rf $(BUILD) --- a/README.md +++ b/README.md @@ -98,10 +98,11 @@ then crypto orchestration, then I/O / async / FFI (monitors, server, storage). |--------------------------|--------------------|---------------------------------| | `dga::max_consonant_run` | `typed/dga.ss` | ✅ ported, vectors pass | | `dga::shannon_entropy` | `typed/dga.ss` | ✅ ported, vectors pass | -| `dga::score_domain` | `typed/dga.ss` | ✅ full: lowercase + dot-trim + benign-suffix + label split + score; vectors pass (diagnostic `reasons` list pending) | +| `dga::score_domain` (+ `score_label_bits`, `dga_label`) | `typed/dga.ss` | ✅ full: lowercase + dot-trim + benign-suffix + label split + score; vectors pass. Sibling kernels `score-label-bits` (ORs one disjoint power-of-2 bit per reason, in the same source order as the weight table — `+` is bitwise-or since the bits don't overlap) and `dga-label` (the exact leftmost label scored, `""` for a benign CDN suffix / empty label) let the untyped layer recover `DgaVerdict.reasons`/`label`/`entropy`/`run`/`length` with no second copy of any scoring decision. | | `&str` ops (lowercase/ends_with/starts_with/contains/split/whole-word) | `typed/strbytes.ss` | ✅ Bytes toolkit, vectors pass — shared by dga/lolbin/sigma | | `lolbin::score` + `severity` | `typed/lolbin.ss` | ✅ full 25-pattern table + severity buckets, plus `match-bits` (a u64 bitmask of which patterns fired, bit 0…24 in PATTERNS order) so the diagnostic breakdown needs no second copy of the matchers; vectors pass (JSON-cmdline parse stays in untyped wrapper) | | `lolbin::LolScore` (match breakdown + JSON) | `jsecmon/lolbin.ss` | ✅ **untyped layer** — the diagnostic companion to the typed kernel: `score-cmdline` returns a `lol-score` (total from `lolbin-score-cmdline`, per-pattern `matches` decoded from `lolbin-match-bits` against a static label/score/explanation table — data, not logic), `score-json-cmdline` adds the JSON-array parse (falls back to the raw string like `from_str::<Vec<String>>`), `lol-severity`/`lol-label-summary` mirror the Rust methods. No matcher is re-implemented, so total and matches can't drift. Rule 11 `detect-lolbin-cmdline` (`storage::detect_lolbin_cmdline`) also lives here rather than in `correlate.ss` — it's the one correlation rule that needs the native scorer, so hosting it beside `score-json-cmdline` keeps `correlate.ss` kernel-free: it scores each `process_start` row's cmdline (row `(host ts pid proc-name cmdline exe)`), skips rows whose cmdline **and** exe are both empty (Rust `continue`), and emits a `suspicious_cmdline` anomaly seed for totals ≥50 (`pname` falls back to `"?"`). `make lolbin-check` runs secmon's 10 `#[test]` vectors plus exact-total, label-order, bit↔label pins, and the Rule-11 threshold/skip/`"?"`-fallback edges (41 cases). | +| `dga::DgaVerdict` (diagnostic verdict + match breakdown) | `jsecmon/dga.ss` | ✅ **untyped layer** — the diagnostic companion to the typed DGA kernel, exactly mirroring `jsecmon/lolbin.ss`: `score-domain-verdict` returns a `dga-verdict` whose `score` is the headline kernel total, whose `reasons` are decoded from `dga-score-label-bits` against a static bit→reason table (data, not logic), and whose `entropy`/`max-consonant-run`/`length` are read off the SAME kernels applied to `dga-label` — so nothing here recomputes a scoring decision and the breakdown can't drift from the total. Rule 12 `detect-dga-domain` (`storage::detect_dga_domain`) also lives here rather than in `correlate.ss` — like Rule 11 it needs the native scorer, so hosting it beside the verdict keeps `correlate.ss` kernel-free: it scores each `dns_query` row (`(host ts pid proc-name query-name)`), skips a `#f`/empty query-name (Rust `Some(q) if !q.is_empty()`), emits a `dga_domain` seed for label scores ≥60, and dedups per `(host, process, scored-label)` keeping the first qualifying query (Rust `HashMap::or_insert`) so a host querying many subdomains of one DGA label alerts once (`pname` falls back to `"?"`). `make dga-check` runs secmon's 5 `dga.rs` `#[test]` vectors plus the bit↔reason pins and the Rule-12 threshold/skip/dedup/cross-host/`"?"`-fallback edges (39 cases). | | `analytics::compute_host_risks` | `typed/analytics.ss` | ✅ risk-score kernel (clamped weighted sum); vectors pass | | `analytics` grouping + `group_incidents` | `jsecmon/analytics.ss` | ✅ **untyped layer** — per-host accumulation/sort/top-N driving the risk-score kernel, plus incident dedup/collapse; secmon analytics vectors pass (`make analytics-check`) | | `storage::detect_sequence_pair` (kill-chain core) | `jsecmon/analytics.ss` | ✅ **untyped layer** — the pure pairing primitive behind `detect_priv_escalation_chain`/`lateral_after_shell`/`persistence_after_access`/`log_cover`: given two event streams as `(host . ts-ms)` lists (the SQL `ORDER BY host,timestamp_ms` fetch is deferred I/O), pair each A with the **first** same-host B strictly later and within `window-ms` — at most one per A (Rust's inner `break`) — returning `((host …) (a-ts …) (b-ts …) (gap-seconds …))` for the caller to wrap as an Anomaly (`format_ts` is calendar-deferred). `gap-seconds` is integer ms/1000 (Rust i64 `/`). `make analytics-check` adds window-edge (≤ inclusive), strictly-later, cross-host, first-B-only, multi-A, and empty-stream cases. | @@ -110,7 +111,7 @@ then crypto orchestration, then I/O / async / FFI (monitors, server, storage). | `storage::detect_kill_chain` (chain core) | `jsecmon/analytics.ss` | ✅ **untyped layer** — the pure multi-phase kill-chain detector: given `(host ts-ms event-type)` rows pre-sorted by host then ts (SQL fetch deferred), slide from each i over the same-host run with `ts ≤ ts_i + window-ms`, map each type to an ATT&CK-ish phase via `event-type->attack-phase` (also exported; unmapped types skipped), and when the **distinct** phases reach `min-phases` (3) emit a chain then skip past it (Rust `i = j`), else advance one. Emits `((host …) (window-start …) (window-end …) (phases …) (event-types …))`; Rust collects phases from an unordered `HashSet`, so `phases` is canonicalized to first-seen order (treat as a set) while `event-types` keeps phase-mapped types in order. `make analytics-check` adds the classifier table, three-phases, two-distinct-only, unmapped-skip, host-boundary, window-edge, past-edge, two-chains-after-skip, and empty cases. | | `storage::detect_off_hours` (predicate) | `jsecmon/analytics.ss` | ✅ **untyped layer** — `off-hours?`, the decision rule factored out of the SQL `WHERE`: a critical/high event is off-hours on a weekend or outside 08:00–18:00 UTC (`weekday` = strftime `%w` 0=Sun…6=Sat, `hour` = `%H` 0–23). The timestamp→(weekday,hour) decomposition is calendar-deferred. `make analytics-check` adds weekend, midday, and the 08:00/17:00/18:00 boundaries. | | `storage::anomaly_rule_attack` (tactic tagger) | `jsecmon/analytics.ss` | ✅ **untyped layer** — the pure rule-name→ATT&CK-tactic table `detect_anomalies` stamps onto each anomaly: `kill_chain`→TA0001/TA0008/TA0010, `off_hours`→TA0005, every other rule→none. `make analytics-check` covers both tagged rules plus untagged/unknown. | -| `storage::detect_lolbin_cmdline` + `detect_dga_domain` | `jsecmon/detect.ss` | ✅ **untyped layer** — the kernel-driven detection rules: score every process_start cmdline (lolbin) / dns_query (dga) into anomalies above threshold. `make detect-check` runs the full events→detect→analytics pipeline; all three scoring kernels fire. Per-pattern lolbin label lists now available via `(jsecmon lolbin)`'s `match-bits`-backed breakdown; label-level DGA dedup still pending (needs a DGA kernel that returns its match breakdown). | +| `storage::detect_lolbin_cmdline` + `detect_dga_domain` | `jsecmon/detect.ss` | ✅ **untyped layer** — the kernel-driven detection rules: score every process_start cmdline (lolbin) / dns_query (dga) into anomalies above threshold. `make detect-check` runs the full events→detect→analytics pipeline; all three scoring kernels fire. This is the *live* pipeline shape (event hash-tables → anomaly hash-tables for `(jsecmon analytics)`); the faithful storage-Rule ports against secmon's `test_detect_*` vectors — with the per-pattern lolbin label breakdown and the label-level DGA dedup + reason list — live in `jsecmon/lolbin.ss` and `jsecmon/dga.ss`. The `dga-label` kernel that label-level dedup needed now exists, so this pipeline's coarser full-query-name dedup can be upgraded to match. | | `triage` classifiers | `typed/triage.ss` | ✅ pure predicates (transient-unit?, phantom-rootkit-race?); vectors pass | | `triage` engine (rules + dispatch) | `jsecmon/triage.ss` | ✅ **untyped layer** — all 18 false-positive rules + first-match engine, in secmon's exact RULES order, dispatch in ordinary Jerboa delegating byte/string classification to the typed kernels; 40 triage vectors pass (`make triage-check`), incl. the security-relevant negatives (non-sshd reading host keys, systemd impersonated from /tmp, unknown daemon reading passwd). | | `triage::compute_triaged_ids` (triage-aware mode) | `jsecmon/triage-store.ss` | ✅ **untyped layer** — the bridge above storage+triage: query every in-scope event, triage each, return the sorted benign/expected ID set to drop into a filter's `exclude_event_ids`. `make triage-store-check` proves the round-trip — detection then sees only the real attacks. | new file mode 100644 --- /dev/null +++ b/examples/dga_check.ss @@ -0,0 +1,121 @@ +;;; Parity check for (jsecmon dga) against secmon's dga.rs #[test] mod +;;; (known_dga_style_fires, long_consonant_run_fires, hex_blob_fires, +;;; cdn_suffix_suppressed, benign_domains_score_zero) plus the reason-bit +;;; decode and the storage Rule 12 detect_dga_domain dedup behaviour. +;;; +;;; scheme --libdirs "$JERBOA/lib:." --script examples/dga_check.ss + +(import (jerboa prelude) + (jsecmon dga)) + +(def fails 0) +(def (check name got want) + (let ((ok (equal? got want))) + (unless ok (set! fails (+ fails 1))) + (displayln (if ok " ok " " FAIL ") name + (if ok "" (str " got " got " want " want))))) + +;; ── reason-bit decode (pins the bit<->reason mapping) ───────────────────────── +(displayln "dga-reasons decode:") +(check "bits 5 -> high-entropy + consonant-run" + (dga-reasons 5) '("long-high-entropy-label" "long-consonant-run")) +(check "bits 49 -> high-entropy + hex + numeric" + (dga-reasons 49) + '("long-high-entropy-label" "long-hex-label" "numeric-heavy-label")) +(check "bits 0 -> no reasons" (dga-reasons 0) '()) + +;; ── score-domain-verdict vs dga.rs #[test] vectors ──────────────────────────── +(displayln "score-domain-verdict:") +(def v1 (score-domain-verdict "kxq8z23nplkdq.example.com")) ;; known_dga_style_fires +(check "dga label" (dga-verdict-label v1) "kxq8z23nplkdq") +(check "dga score 70" (dga-verdict-score v1) 70) +(check "dga run 6" (dga-verdict-max-consonant-run v1) 6) +(check "dga length 13" (dga-verdict-length v1) 13) +(check "dga reasons" + (dga-verdict-reasons v1) + '("long-high-entropy-label" "long-consonant-run")) +(check "dga entropy > 3.3" (> (dga-verdict-entropy v1) 3.3) #t) + +(def v2 (score-domain-verdict "xkqzmnpw.bad")) ;; long_consonant_run_fires +(check "consonant run 8" (dga-verdict-max-consonant-run v2) 8) +(check "consonant score 35" (dga-verdict-score v2) 35) +(check "consonant reasons" (dga-verdict-reasons v2) '("long-consonant-run")) +(check "consonant entropy = 3.0" (dga-verdict-entropy v2) 3.0) + +(def v3 (score-domain-verdict "a1b2c3d4e5f6789012345.evil.com")) ;; hex_blob_fires +(check "hex blob score 100 (capped)" (dga-verdict-score v3) 100) +(check "hex blob reasons" + (dga-verdict-reasons v3) + '("long-high-entropy-label" "long-hex-label" "numeric-heavy-label")) + +(def v4 (score-domain-verdict "d2hk78xq2k.cloudfront.net")) ;; cdn_suffix_suppressed +(check "cdn suffix label \"\"" (dga-verdict-label v4) "") +(check "cdn suffix score 0" (dga-verdict-score v4) 0) +(check "cdn suffix length 0" (dga-verdict-length v4) 0) +(check "cdn suffix entropy 0.0" (dga-verdict-entropy v4) 0.0) +(check "cdn suffix no reasons" (dga-verdict-reasons v4) '()) + +(def v5 (score-domain-verdict "google.com")) ;; benign_domains_score_zero +(check "benign label google" (dga-verdict-label v5) "google") +(check "benign score 0" (dga-verdict-score v5) 0) +(check "benign no reasons" (dga-verdict-reasons v5) '()) + +;; ── detect-dga-domain: storage Rule 12 over scored dns_query rows ───────────── +;; row = (host ts pid proc-name query-name). +(displayln "detect-dga-domain:") +(def (rule a) (cdr (assq 'rule a))) +(def (desc a) (cdr (assq 'description a))) +(def (sev a) (cdr (assq 'severity a))) +(def (ats a) (cdr (assq 'timestamp-ms a))) +(def (dval a k) (cdr (assq k (cdr (assq 'details a))))) + +(def d1 + (detect-dga-domain + (list (list "h1" 1000 4242 "curl" "kxq8z23nplkdq.example.com")))) +(check "one >=60 query -> 1 anomaly" (length d1) 1) +(check "rule" (rule (car d1)) "dga_domain") +(check "severity" (sev (car d1)) "high") +(check "ts" (ats (car d1)) 1000) +(check "description" + (desc (car d1)) + "'curl' on h1 queried high-entropy domain 'kxq8z23nplkdq.example.com' (score 70)") +(check "score detail" (dval (car d1) 'score) 70) +(check "label detail" (dval (car d1) 'label) "kxq8z23nplkdq") +(check "query-name detail" (dval (car d1) 'query-name) "kxq8z23nplkdq.example.com") +(check "reasons detail" + (dval (car d1) 'reasons) '("long-high-entropy-label" "long-consonant-run")) + +;; below threshold (xkqzmnpw label scores 35) -> none +(check "score 35 query -> none" + (length (detect-dga-domain (list (list "h1" 1 1 "p" "xkqzmnpw.bad")))) 0) +;; #f / empty query-name skipped +(check "#f query-name -> none" + (length (detect-dga-domain (list (list "h1" 1 1 "p" #f)))) 0) +(check "empty query-name -> none" + (length (detect-dga-domain (list (list "h1" 1 1 "p" "")))) 0) + +;; dedup: two suspect subdomains sharing the leftmost label, same host+proc -> 1 +(def dd + (detect-dga-domain + (list (list "h1" 1000 100 "curl" "kxq8z23nplkdq.aaa.com") + (list "h1" 2000 100 "curl" "kxq8z23nplkdq.bbb.com")))) +(check "same label dedup -> 1 anomaly" (length dd) 1) +(check "dedup keeps first ts" (ats (car dd)) 1000) +(check "dedup keeps first query" (dval (car dd) 'query-name) "kxq8z23nplkdq.aaa.com") +;; different host -> not deduped +(check "different host -> 2 anomalies" + (length (detect-dga-domain + (list (list "h1" 1000 100 "curl" "kxq8z23nplkdq.aaa.com") + (list "h2" 2000 100 "curl" "kxq8z23nplkdq.aaa.com")))) 2) +;; missing process_name -> pname "?" in description and detail +(def dq + (detect-dga-domain (list (list "h1" 1000 #f #f "kxq8z23nplkdq.example.com")))) +(check "missing proc -> '?' detail" (dval (car dq) 'process-name) "?") +(check "missing proc -> '?' description" + (desc (car dq)) + "'?' on h1 queried high-entropy domain 'kxq8z23nplkdq.example.com' (score 70)") + +(newline) +(if (= fails 0) + (displayln "OK: dga matches secmon's dga.rs vectors + storage Rule 12 dedup.") + (begin (displayln fails " FAILURES") (exit 1))) new file mode 100644 --- /dev/null +++ b/jsecmon/dga.ss @@ -0,0 +1,118 @@ +#!chezscheme +;;; jsecmon DGA diagnostic + detection layer — the untyped companion to the +;;; typed (jsecmon dga) scoring kernel. +;;; +;;; Every scoring DECISION lives once in typed/dga.ss (compiled to Rust): +;;; `dga-score-domain` is the 0..100 headline (>=60 fires), `dga-score-label-bits` +;;; is a bitmask of which reasons fired (bit 0..6 in dga::score_domain source +;;; order), and `dga-label` is the exact leftmost label that was scored ("" when +;;; a benign CDN suffix or an empty label forces a clean verdict). This module +;;; adds the diagnostic verdict: it decodes the reason bitmask against a static +;;; bit->reason table (data, not logic) and reads entropy / consonant-run / +;;; length off the SAME kernels applied to `dga-label` — so nothing here recomputes +;;; a scoring decision and the breakdown can't drift from the headline. This is +;;; the same bitmask-breakdown technique used for lolbin.ss. +;;; +;;; Then detect-dga-domain (secmon storage Rule 12) turns scored dns_query rows +;;; into anomaly seeds, deduped per (host, process, label) so a host querying +;;; many subdomains of one DGA label alerts once. The SQL fetch and the Anomaly +;;; calendar fields stay the caller's (deferred) boundary, exactly like the other +;;; detect_* rules in correlate.ss / lolbin.ss. +;;; +;;; Verified against secmon dga.rs's #[test] vectors in examples/dga_check.ss +;;; (which also pins the bit<->reason mapping). + +(library (jsecmon dga) + (export make-dga-verdict dga-verdict? dga-verdict-label dga-verdict-score + dga-verdict-entropy dga-verdict-max-consonant-run + dga-verdict-length dga-verdict-reasons + dga-reasons score-domain-verdict detect-dga-domain) + (import (except (chezscheme) + make-hash-table hash-table? + sort sort! + printf fprintf + path-extension path-absolute? + with-input-from-string with-output-to-string + iota 1+ 1- + partition + make-date make-time) + (except (jerboa prelude) meta atom?) + (only (jsecmon kernels) + dga-score-domain dga-score-label-bits dga-label + dga-shannon-entropy dga-max-consonant-run)) + + (defstruct dga-verdict (label score entropy max-consonant-run length reasons)) + + ;; (bit . reason) for each dga::score_domain reason push, in source (= bit) + ;; order. Blocks 1 and 2 are if/else, so bits 1|2 and 4|8 are each exclusive; + ;; ascending bit order therefore equals the Vec<&str> push order. + (def *dga-reason-meta* + (list (cons 1 "long-high-entropy-label") + (cons 2 "medium-high-entropy-label") + (cons 4 "long-consonant-run") + (cons 8 "consonant-run-medium") + (cons 16 "long-hex-label") + (cons 32 "numeric-heavy-label") + (cons 64 "very-long-high-entropy"))) + + ;; decode a score-label-bits mask into the reason strings, in bit order. + (def (dga-reasons bits) + (filter-map (lambda (m) (and (positive? (bitwise-and bits (car m))) (cdr m))) + *dga-reason-meta*)) + + ;; Full dga::DgaVerdict for a domain. label/score/bits come straight from the + ;; kernel; entropy / max-consonant-run / length are read off the SAME kernels + ;; applied to that label. A benign/empty label is "" => entropy 0.0, run 0, + ;; length 0, no reasons — matching DgaVerdict::default in those cases. + (def (score-domain-verdict domain) + (let* ((label (dga-label domain)) + (bits (dga-score-label-bits label))) + (make-dga-verdict label + (dga-score-domain domain) + (dga-shannon-entropy label) + (dga-max-consonant-run label) + (string-length label) + (dga-reasons bits)))) + + ;; ── Rule 12: DGA domain — dns_query whose leftmost label scores >=60 ──────── + ;; row = (host ts pid proc-name query-name); pid integer|#f, proc-name/query-name + ;; string|#f. A #f or empty query-name is skipped (Rust `Some(q) if !q.is_empty()`). + ;; Deduped per (host, proc-or-"", scored-label): the first qualifying query for + ;; a suspect label wins (HashMap or_insert). Seeds are emitted in first-seen key + ;; order (the Rust HashMap iteration order is unspecified — treat it as a set). + (def (detect-dga-domain rows) + (let ((seen (make-hash-table)) (order '())) + (for-each + (lambda (r) + (let ((host (car r)) (ts (cadr r)) (pid (caddr r)) + (proc (or (cadddr r) "")) (qname (list-ref r 4))) + (when (and (string? qname) (not (string-empty? qname))) + (let ((v (score-domain-verdict qname))) + (when (>= (dga-verdict-score v) 60) + (let ((key (str host "\x1f;" proc "\x1f;" (dga-verdict-label v)))) + (unless (hash-key? seen key) + (hash-put! seen key #t) + (set! order (cons (dga-seed host ts pid proc qname v) order))))))))) + rows) + (reverse order))) + + (def (dga-seed host ts pid proc qname v) + (let ((pname (if (string-empty? proc) "?" proc)) + (score (dga-verdict-score v))) + (list (cons 'rule "dga_domain") + (cons 'description + (str "'" pname "' on " host " queried high-entropy domain '" + qname "' (score " score ")")) + (cons 'severity "high") + (cons 'timestamp-ms ts) + (cons 'host host) + (cons 'details + (list (cons 'process-name pname) + (cons 'pid pid) + (cons 'query-name qname) + (cons 'label (dga-verdict-label v)) + (cons 'entropy (dga-verdict-entropy v)) + (cons 'max-consonant-run (dga-verdict-max-consonant-run v)) + (cons 'label-length (dga-verdict-length v)) + (cons 'score score) + (cons 'reasons (dga-verdict-reasons v)))))))) --- a/jsecmon/kernels.ss +++ b/jsecmon/kernels.ss @@ -21,6 +21,7 @@ host-risk-score ;; dga dga-max-consonant-run dga-shannon-entropy dga-score-label dga-score-domain + dga-score-label-bits dga-label ;; strbytes toolkit ascii-lower-bytes bytes-contains? bytes-prefix? bytes-suffix? contains-word? index-of-byte first-label-len @@ -142,10 +143,18 @@ (define dga-max-consonant-run (str->kernel-nat "jt_jsecmon_dga_max_consonant_run")) (define dga-score-label (str->kernel-nat "jt_jsecmon_dga_score_label")) (define dga-score-domain (str->kernel-nat "jt_jsecmon_dga_score_domain")) + (define dga-score-label-bits (str->kernel-nat "jt_jsecmon_dga_score_label_bits")) (define %shannon (fp "jt_jsecmon_dga_shannon_entropy" (u8* size_t) double)) (define (dga-shannon-entropy s) (let ((b (u8->bytes s))) (%shannon b (bytevector-length b)))) + ;; the leftmost label score-domain actually scored ("" for benign/empty), + ;; returned via the (out_ptr, out_len) string convention like lolbin-severity. + (define %dga-label (fp "jt_jsecmon_dga_dga_label" (u8* size_t void* void*) unsigned-8)) + (define (dga-label s) + (let ((b (u8->bytes s))) + (utf8->string (call->bytes (lambda (pp pl) (%dga-label b (bytevector-length b) pp pl)))))) + ;; ── strbytes (operate on raw Bytes; lowercase yourself if needed) ────────── (define %lower (fp "jt_jsecmon_strbytes_ascii_lower_bytes" (u8* size_t void* void*) unsigned-8)) (define (ascii-lower-bytes s) --- a/typed/dga.ss +++ b/typed/dga.ss @@ -4,7 +4,8 @@ ;;; to Rust by the jerboa typed→rust backend. Domains are ASCII, so we score ;;; over UTF-8 bytes (string->utf8 + bytevector-u8-ref) rather than chars. (typed-library (jsecmon dga) - (export max-consonant-run shannon-entropy score-label score-domain) + (export max-consonant-run shannon-entropy score-label score-domain + score-label-bits dga-label) ;; the Bytes-based &str toolkit (to_ascii_lowercase, ends_with, split('.')) (import (jsecmon strbytes)) @@ -153,4 +154,44 @@ (let ((label-len (first-label-len trimmed))) (if (= label-len 0) 0 - (score-label (utf8->string (sub-bytes trimmed label-len)))))))))) + (score-label (utf8->string (sub-bytes trimmed label-len))))))))) + + ;; Reason bitmask for an already-lowercased leftmost label — the sibling of + ;; score-label, ORing one distinct bit per reason instead of summing weights + ;; (the bits are disjoint powers of two, so `+` here IS bitwise-or). The + ;; untyped layer decodes this against a static bit→reason table to recover + ;; dga::DgaVerdict.reasons; the headline score still comes from score-domain, + ;; so the breakdown can never drift from the total. Bit order = source order: + ;; 1 long-high-entropy 2 medium-high-entropy 4 long-consonant-run + ;; 8 consonant-run-medium 16 long-hex 32 numeric-heavy 64 very-long-high-entropy + (def (score-label-bits (label : String)) : Nat + (let ((bs (string->utf8 label))) + (let ((len (bytevector-length bs)) + (ent (shannon-entropy label)) + (run (max-consonant-run label))) + (let ((hexish (and (> len 0) (all-hex? bs len))) + (digits (digit-count bs len))) + (let ((b0 (if (and (>= len 12) (>= ent 3.3)) 1 + (if (and (>= len 8) (>= ent 3.8)) 2 0)))) + (let ((b1 (+ b0 (if (>= run 5) 4 + (if (and (>= run 4) (>= len 10)) 8 0))))) + (let ((b2 (+ b1 (if (and hexish (>= len 20)) 16 0)))) + (let ((b3 (+ b2 (if (and (>= len 10) (>= (* digits 2) len)) 32 0)))) + (let ((b4 (+ b3 (if (and (>= len 20) (>= ent 4.0)) 64 0)))) + b4))))))))) + + ;; The exact leftmost label that score-domain scored: lowercase, strip a + ;; trailing FQDN dot, then split('.').next(). Returns "" when a benign CDN + ;; suffix forces a clean verdict or the label is empty — matching + ;; DgaVerdict.label (default "" in those cases). So the untyped verdict can + ;; take entropy = (shannon-entropy label), run = (max-consonant-run label), + ;; length = (string-length label) from the SAME kernels, no logic duplicated. + (def (dga-label (domain : String)) : String + (let ((lowered (ascii-lower-bytes domain))) + (let ((trimmed (sub-bytes lowered (rtrim-dot-len lowered (bytevector-length lowered))))) + (if (benign-suffix? trimmed) + "" + (let ((label-len (first-label-len trimmed))) + (if (= label-len 0) + "" + (utf8->string (sub-bytes trimmed label-len)))))))))