docs: mark monitors + event-json + collector complete in README
ober
79574aae41961073e18f9cd3adae307e2823df96
--- a/README.md +++ b/README.md @@ -62,6 +62,13 @@ make logtamper-check # logtamper: system-log/history tables + classify-tamper (t make detection-rules-check # DETECTION_RULES ATT&CK catalog: rule_attack + anomaly_rule_attack make ipaddr-check # lateral: faithful IpAddr parse (v4/v6) + is_internal_ip (RFC1918/fc00) make auth-check # auth-log parsers: sshd/sudo/su/pam/useradd/userdel/passwd + extract_field +make monitor-process-check # process monitor scan loop: pid diff/classify/emit over fixture provider +make monitor-network-check # network monitor scan loop: listener/connection dedup+classify over fixture +make monitor-files-check # file integrity monitor: detect-change priority + new-file discovery +make monitor-dns-check # dns monitor connection-polling fallback: port-53 filter + 5 s dedup window +make monitor-manager-check # monitor manager: boot + one poll cycle through all four monitors (fixture) +make event-json-check # event JSON serializer: all 11 variants round-tripped and field-asserted +make collector-check # agent collector loop: boot+tick into a real SQLite store, rows queried back make checks # every Jerboa-side check in one shot ``` @@ -161,4 +168,11 @@ then crypto orchestration, then I/O / async / FFI (monitors, server, storage). | `monitor/lateral` IP parsing + internal-network test (`is_internal_ip`) | `jsecmon/ipaddr.ss` | ✅ **untyped layer** — `is_internal_ip` parses `ip_str.parse::<IpAddr>()` first and only classifies on success, so `parse-ipv4` / `parse-ipv6` faithfully reproduce **Rust std's `IpAddr` FromStr** boundary: IPv4 = exactly 4 octets, 1–3 digits, **no leading zeros**, ≤255; IPv6 = colon-separated 1–4-digit hex groups with at most one `::` (eliding ≥1 zero group) and an optional trailing embedded IPv4 (forbidden before `::`). `is-internal-ip` then mirrors lateral.rs: V4 `10/8` · `172.16/12` · `192.168/16` · `127/8`, V6 `fc00::/7` (`seg0 & 0xfe00 == 0xfc00`) or loopback. lateral.rs has no #[test]; expectations were generated by a **std-only Rust oracle** over ~70 inputs, so `make ipaddr-check` (88 cases) pins the port to real Rust and **is** the spec. | | `monitor/auth` log-line parsers (`parse_auth_line` + 7 sub-parsers) | `jsecmon/auth.ss` | ✅ **untyped layer** — the dispatcher lowercases and routes on substrings (`sshd`→ssh, `sudo`, `su[`/`su:`, `authentication failure`→pam, `useradd`/`adduser`, `userdel`/`deluser`, `password changed`/`passwd`), each sub-parser pulling fields with `extract_field(line, start, end)`. Faithful corners: `extract_field` searches the **raw** line (case-sensitive) while dispatch/success use the lower-cased copy, so `Invalid user admin` → username `unknown`; the SSH "accepted" branch `?`-propagates a missing `for ` to **#f** (no fall-through); and the documented quirks where the username captures `TTY=pts/0` / `authentication failure; TTY=…` / `root)` are preserved. AuthEventType → `'ssh-key-auth 'login 'failed-login 'sudo-attempt 'su-attempt 'user-created 'user-deleted 'password-change`; None → `#f`. auth.rs has no #[test]; `make auth-check` (24 cases) derives every expectation from a **std-only Rust oracle** over the verbatim bodies and **is** the spec. | | `format_ts` / `format_ts_iso` / `parse_hour_to_ms` / `parse_datetime` (chrono UTC calendar) | `jsecmon/calendar.ss` | ✅ **untyped layer** — the pure epoch-ms ↔ civil-date boundary the detect/analytics seeds defer to. secmon formats/parses timestamps with chrono always in UTC (`DateTime::from_timestamp_millis`, `NaiveDateTime`), which is pure proleptic-Gregorian arithmetic, reproduced exactly with Howard Hinnant's `days_from_civil`/`civil_from_days`: `format-ts` (`%Y-%m-%d %H:%M:%S`, storage+analyze), `format-ts-iso` (`%Y-%m-%dT%H:%M:%S%.3fZ`, analyze), `parse-hour-to-ms` (`%Y-%m-%d %H`+`:00:00`, the frequency-spike inverse of `%H` decomposition, bad input → 0), and `parse-datetime-ms` (returns the prelude Result: relative `Nh`/`Nd` from an optional `at`/now, then RFC3339 with `Z`/`±HH:MM` offset and optional `.fff`, then naive ISO assumed-UTC, then date-only; else `err` with the exact Rust message). Faithful corners: Hinnant's `/` truncates toward zero (`quotient`) but the ms→day split must **floor** (a pre-1970 ms keeps a non-negative ms-of-day), so the parts use exact `floor`; the export is `parse-datetime-ms`/`char-index` to avoid shadowing the prelude's own `parse-datetime` (a datetime record) and `string-index`. `make calendar-check` pins the civil round-trip, both formatters (incl. leap-day, negative-ms flooring), `parse_hour_to_ms`, and secmon's `test_parse_datetime` invariants (28 cases). | -| monitors / server / ebpf / dtrace | — | ⏳ I/O+async+FFI, last | +| `monitor::process::ProcessMonitor` scan loop | `jsecmon/monitor-process.ss` | ✅ **untyped layer** — secmon's process-monitor scan in pure Jerboa over an injected `mon-provider` seam (list-pids / get-process / hostname), fixture-tested: diff pid sets → classify each new spawn (check-suspicious delegates to `(jsecmon suspicious)`) → emit process_start/suspicious_exec; exited pids → process_exit; list-pids error → no-op. Faithfully preserves secmon's `continue` before insert (a failed get-process is not added to known and is retried), the parent-cache lookup, and the 1000-entry cache prune. `make-linux-provider` is the thin /proc shell via `(jsecmon proc-linux)` parsers (exe/cwd readlink deferred). `make monitor-process-check` (30 cases). | +| `monitor::network::NetworkMonitor` scan loop | `jsecmon/monitor-network.ss` | ✅ **untyped layer** — the network-monitor scan over an injected `net-provider` seam: listeners → listening_port/info (de-dup key `proto:laddr:lport`); connections → `(jsecmon netconn)` connection-suspicious? → suspicious_connection/high or network_connection/info (de-dup key `proto:laddr:lport:raddr:rport:state`). Provider Err skips that half; never ages out (secmon no-op TODO). `make-linux-network-provider` reads /proc/net/{tcp,udp,tcp6,udp6} via proc-linux parsers (socket-inode→pid walk deferred). `make monitor-network-check` (13 cases). | +| `monitor::files::FileIntegrityMonitor` scan loop | `jsecmon/monitor-files.ss` | ✅ **untyped layer** — file-integrity monitor over an injected `file-provider` seam: `baseline-files` records initial state; `detect-change` produces the ordered diff (deleted > created > permission-changed > owner-changed > modified) — pinned secmon quirk: owner-changed reports `old_mode = new_mode = new.mode`; `scan-files` re-stats tracked paths + discovers new files in watched dirs (Created), classifying via `(jsecmon file-change)` is-suspicious-change. `make-linux-file-provider` uses file-exists?/file-modification-time (stat+sha256 FFI deferred); `linux-monitored-paths` = secmon's Linux PLATFORM_PATHS. `make monitor-files-check` (18 cases). | +| `monitor::dns::DnsMonitor` connection-polling fallback | `jsecmon/monitor-dns.ss` | ✅ **untyped layer** — the polling fallback: reuse net-provider's list-connections; keep only remote-port==53; dedup key `pid:remote_addr:protocol:local_port` (pid defaults to 0); skip if seen <5000 ms ago; emit dns_query/info (query_name `"<unknown>"`, query_type UDP/TCP, server_addr=remote-addr, response_addrs=[]). `cleanup-dns-queries` ages entries ≥30 s. Shares net-provider with the network monitor exactly as secmon. `make monitor-dns-check` (14 cases). | +| `monitor::mod::MonitorManager` (boot + poll cycle) | `jsecmon/monitor-manager.ss` | ✅ **untyped layer** — `monitor-boot` baselines files + emits agent_start; `monitor-tick` runs process/network/files/dns in fixed order and returns the merged event stream; cleanup-dns-queries runs after tick. `make-linux-monitor-set` wires all four live Linux providers. `make monitor-manager-check` drives the full four-monitor cycle end to end with fixture providers. | +| `event_json.rs get_event_json_data` | `jsecmon/event-json.ss` | ✅ **untyped layer** — the flat JSON `data` body for all 11 event variants, matching secmon's json!{...} field SETS exactly: suspicious_exec drops cwd/name; listening_port keeps only local side; suspicious_connection drops state/process_name; suspicious_file_change keeps reason+hashes only. Option::None → JSON null via `(if #f #f)` void sentinel; parent process → nested {pid,name,exe} or null; change_type = Rust Debug form ("Modified", "PermissionChanged", …); dns_query always carries response_addrs=[]. `make event-json-check` drives real monitor events through the serializer, parses back, and asserts all tricky corners. | +| `bin/agent` storage path (collector loop) | `jsecmon/collector.ss` | ✅ **untyped layer** — ties monitor runtime to storage: `collect-event!` derives the full storage row (data = event-data-json, summary = build-summary, pid/pname = extract-pid/extract-process-name), assigns a monotonic seq, INSERT-OR-IGNORE dedup; `collect-events!` drains a batch; `run-boot!`/`run-tick!` tie monitor-boot/monitor-tick to the store. `run-agent` is the thin live shell (real ms clock, Linux monitor-set, poll/sleep). `make collector-check` drives boot+tick of the four-monitor fixture set into a real temp SQLite store, queries rows back to assert event count, derived pid/name/summary, parsed data, and dedup (19 cases). | +| server listener / ebpf / dtrace I/O / live FFI gaps | — | ⏳ I/O+async+FFI, last — server pull protocol (ProtocolMessage codec + request handler); live-shell FFI gaps (process exe/cwd readlink, network socket-inode→pid walk, file stat+sha256); remaining monitor loops (rootkit/persistence/cron/auth/container/lateral/revshell — classifiers done, enumeration shells deferred); ebpf/dtrace AF_PACKET capture; EncryptedPayload bincode framing (only if wire-compat needed). |