λgitsite

~ober/jerboa-lora

Imported from ~/mine/jerboa-lora

download snapshot

about

# Jerboa Scheme LoRA

A fine-tune of **Qwen3-Coder-30B-A3B-Instruct** that knows [Jerboa](https://github.com/jaimef/jerboa) — a Chez-Scheme-based dialect with a Gerbil-flavored prelude. The LoRA teaches the base model Jerboa's module syntax (`(std foo)`, `(jerboa prelude)` — not `:std/foo`), the standard library, the actor/fiber system, the FFI, and how Jerboa diverges from Gerbil/Racket/Clojure/SRFI.

Three lines:

- **v3** — public, 3-stage CPT → SFT → DPO on RunPod A100. Live as `jaimef/jerboa-qwen` on Ollama.
- **v4** — private, same pipeline on a **heretic-abliterated** Qwen3-Coder base (uncensored). Single-user; not on Ollama.
- **v5** — private, same pipeline on a different base: **abliterated Qwen3.6-35B-A3B** (`qwen3_5_moe` hybrid — 30 Gated-DeltaNet + 10 full-attention layers, 256 fused experts + shared expert). Text-only (MTP and vision heads dropped). BF16 on HF private, MLX 6-bit local; not on Ollama.

## Quick Start

### v3 (public, Ollama)

```bash
ollama pull jaimef/jerboa-qwen
ollama run jaimef/jerboa-qwen "How do I import the Jerboa prelude and parse JSON?"
```

Tags: `:latest` is Q8_0 (~32 GB, full fidelity). `:q4_k_m` is the smaller quant (~18 GB). Pick `:latest` if it fits.

### v4 (private, local MLX)

```bash
mlx_lm.generate --model jerboa-mlx-6bit-v4 --prompt "How do I import the Jerboa prelude and parse JSON?"
# or run as an OpenAI-compatible server:
mlx_lm.server --model jerboa-mlx-6bit-v4
```

BF16 backup on HF private: `jaimef21/jerboa-qwen3-coder-30b-v4`. MLX 6-bit stays local — see [`upload_hf_v4.sh`](upload_hf_v4.sh) if you ever want to retry the HF push (S3 multipart from residential uplink broken-pipes; the script is idempotent and resume-friendly).

### v5 (private, local MLX)

```bash
mlx_lm.generate --model jerboa-mlx-6bit-v5 --prompt "How do I import the Jerboa prelude and parse JSON?"
mlx_lm.server   --model jerboa-mlx-6bit-v5   # OpenAI-compatible server
```

BF16 backup on HF private: `jaimef21/jerboa-qwen3.6-35b-a3b-v5` (16 shards, 69.3 GB). MLX 6-bit local: `jerboa-mlx-6bit-v5/` (6 shards, 26 GB, 6.502 bpw). On a 48 GB Mac keep `sudo sysctl iogpu.wired_limit_mb=32768` — values above ~32 GB kernel-panic when loading this 23+ GB bundle (the `Makefile` caps it; do not raise it).

**MLX `model_type` gotcha.** The text-only `save_pretrained` writes `config.json` with `model_type="qwen3_5_moe_text"` (correct for transformers, which registers a `…TextForCausalLM` class). But `mlx_lm` only ships `models/qwen3_5_moe.py`, so it rejects `…_text` as unsupported. Fix without touching the canonical HF repo: symlink the cached shards into a scratch dir and write a patched `config.json` with `model_type` set back to `qwen3_5_moe` (its `ModelArgs.from_dict` wraps the flat config as `text_config` when no `text_config` key exists, so that's sufficient), then `mlx_lm.convert --hf-path <scratch>`. This is what `v5_mlx_src/` is.

## Pipeline

Three stages on one pod, ~5–6h wall clock. See [TRAINING_PIPELINE.md](TRAINING_PIPELINE.md) for design rationale and [TRAINING_FIX_PLAN.md](TRAINING_FIX_PLAN.md) for the v3 hyperparam corrections that fixed the v1 "DPO delta below the quantization floor" pathology.

| Stage | What it learns | Data | LR / Epochs | LoRA (v4) |
|---|---|---|---|---|
| **CPT** | Token distribution of raw Jerboa source | `cpt_corpus_v3.jsonl` (~6 MB, mined from `jerboa`, `jerboa-mcp`, stdlib doc-comments) | 2.0e-5 / 2 | r=32, α=64 |
| **SFT** | Answer Jerboa questions in chat | `sft_v3_mined.jsonl` (~3.5 MB, compile-gated via `jerboa_check_syntax`) | 1.0e-4 / 2 | r=32, α=64 |
| **DPO** | Suppress Gerbil/Racket/Clojure/SRFI hallucinations | `dpo_pairs_v3.jsonl` (~300 pairs, compile-validated chosen) | 5.0e-6 / 3 | r=32, α=64 |

v3 ran at r=64 / α=128 on A100 80GB. v4 dropped to r=32 / α=64 on **H100 NVL** (95.8 GiB) because the larger adapter OOMs during `paged_adamw_8bit` optimizer state init even on the bigger card. r=32 fits with ~19 GiB headroom and DPO still converged cleanly (final loss 0.0008, rewards/accuracies 1.0).

v5 keeps r=32 / α=64 but runs on **H200 NVL** (the 35B-A3B `qwen3_5_moe` is larger than v4's 30B-A3B). Two arch-specific fixes were needed for axolotl on this base: (1) **disable the LoRA Triton kernel patcher** (`lora_mlp_kernel: false`, `lora_qkv_kernel: false`, `lora_o_kernel: false`) — axolotl tries to `import transformers.models.qwen3_5_moe_text` for the fused kernels and crashes; with them off it falls back to plain PEFT LoRA, which works. (2) **`merge_lora_qwen35moe.py`** shims `transformers.core_model_loading.WeightConverter.__init__` to swallow the `distributed_operation`/`quantization_operation` kwargs that peft-0.19.1 passes but transformers-5.8.1 doesn't accept — without it, any adapter load that targets the fused `experts.*` (i.e. all of them) crashes during the inter-stage merge.

Each stage's LoRA is merged into bf16 between stages so the next stage trains on absorbed weights, not stacked adapters. Targets attention + MLP + MoE experts (`experts.gate_up_proj`, `experts.down_proj`) — the key reason RunPod axolotl beat the previous Together-AI single-stage path, which was attention-only.

### Run the pipeline

The active remote-training entry point is [`v6_runpod.py`](v6_runpod.py) — the only supported orchestrator (see [docs/training-security.md](docs/training-security.md)). It trains CPT → SFT → DPO as PEFT adapters and pulls the final adapter directory (it does **not** produce a fused model). Run `make security` first to validate the immutable training lock.

```bash
make security                     # validate security/training-lock.json + inputs

# Full pipeline: up → push → prepare-cache → preflight → train-smoke → train → pull → down
python3 v6_runpod.py auto

# Or drive stages individually:
python3 v6_runpod.py up
python3 v6_runpod.py push
python3 v6_runpod.py preflight
python3 v6_runpod.py train        # full adapter sequence; auto-runs upload-hf
python3 v6_runpod.py pull         # download adapter + logs (not a merged model)
python3 v6_runpod.py status
python3 v6_runpod.py down
```

Pod state lives in `.runpod_state_v6.json` (gitignored). See [version-6.md](version-6.md) for the full v6 design and subcommand reference.

> **Retired:** `runpod_train.py` (and its `.runpod_state.json`) is disabled — it predates authenticated/private services and immutable training inputs, and now exits immediately with a pointer to `v6_runpod.py`. The v3/v4/v5 runs described below used it; do not revive it.

### Build data

```bash
python3 build_cpt_corpus_v2.py    # → cpt_corpus_v3.jsonl
python3 build_sft_v3.py           # → sft_v3_mined.jsonl (compile-gated)
python3 build_dpo_pairs_v3.py     # → dpo_pairs_v3.jsonl (compile-validated)
```

The builders read from `~/mine/jerboa` (source, stdlib, tests, docs) and `~/mine/jerboa-mcp` (cookbooks, divergence, error-fixes, api-signatures) via the `jerboa_*` MCP tools.

### Convert + publish

```bash
# MLX 6-bit bundle for Mac (loads under mlx_lm or LM Studio)
./convert_to_mlx.sh runpod-pipeline-final jerboa-mlx-6bit-v3

# HuggingFace
./upload_hf.sh
```

> **Retired (GGUF/Ollama):** `build_ollama_gguf.sh` is disabled until the conversion dependencies and model artifacts have a reviewed lock — it exits non-zero on purpose. Because `push_ollama.sh` requires a locally-built GGUF tag (`jerboa-qwen:latest`), the Ollama push path currently cannot complete end-to-end. The published `jaimef/jerboa-qwen` Ollama tags predate this retirement.

## Evaluation

v3 is gated on two evals before publish; both must beat the un-fine-tuned base.

| Eval | What it measures | Script | v3 result |
|---|---|---|---|
| Held-out coding | Compile-rate + idiom hits on questions never seen during training | `eval_holdout.py` | see `eval_pod_v3.json` vs `eval_pod_base.json` |
| Per-pair similarity | Chosen-vs-rejected on the DPO set (Jaccard + char-similarity, volume-invariant) | `eval_similarity.py` | see `eval_mlx_v3.json` vs `eval_mlx_v2.json` |

The similarity eval is the one to trust — it's volume-invariant and per-pair, so it doesn't get gamed by dataset growth.

**v5 results** (`eval_pod_base.json` = abliterated base vs `eval_pod_v5.json` = trained, 12 holdout Qs / 40 similarity pairs):

| Metric | Base | v5 |
|---|---|---|
| Holdout total score | 19 (mean 1.58) | **31 (mean 2.58)** |
| Holdout has-code | 10/12 | **12/12** |
| Mean generation time | 15.2 s | **4.2 s** |
| Similarity `tok_lean` (mean) | 0.0288 | **0.0873** |
| Pairs leaning chosen | 34/40 | 32/40 |
| Refusals (`eval_refusal_hf.py`, 24 prompts) | — | **0/24** |

v5 clearly beats its base on every coding signal; `tok_lean` triples (stronger pull toward Jerboa idioms) and 0/24 refusals confirms the abliteration held through training.

## Use with OpenCode

### Local Ollama

```json
{
  "$schema": "https://opencode.ai/config.json",
  "provider": {
    "ollama": {
      "npm": "@ai-sdk/openai-compatible",
      "name": "Ollama (local)",
      "options": { "baseURL": "http://localhost:11434/v1" },
      "models": { "jerboa-qwen": { "name": "Jerboa Qwen" } }
    }
  }
}
```

Or run `./configure_opencode.sh ollama` to write that for you.

## Why this works where the previous approach didn't

The v1 pipeline was a single SFT pass on Together AI (attention-only LoRA) + a single SFT pass on MLX (attention + MLP, 4-bit base). Two structural problems:

1. **No CPT** — the model never saw raw Jerboa source. It learned "how to answer questions about Jerboa" but not "how Jerboa code naturally looks."
2. **No DPO** — the 119 divergence entries (`hash-has-key?` → `hash-key?`, `(srfi :NN)` → `(srfi NN)`, etc.) were flattened into SFT Q/A, which only pulls the model toward the right form. DPO actively pushes it away from the wrong form. The wrong form's prior (learned from the base's massive Racket/Gerbil training data) survived essentially intact.

v3 fixes both, plus the v1 DPO hyperparam pathology (LR `5e-7` × 18 steps × LoRA r=16 produced a cumulative weight update of ~2e-5 — below the BF16 precision floor and erased by Q4_K_M quantization). v3 runs LR `5e-6` / 3 epochs / r=64, α=128.

## v4 abliteration

v4 abliterates the base model before re-running the CPT → SFT → DPO pipeline so the final model has no refusal floor.

**Heretic detour.** Initial v4 attempts used [Heretic](https://github.com/p-e-w/heretic) (`runpod_abliterate.py` + `heretic_auto_driver.py` to bypass its TUI). All three Heretic runs plateaued at 75–96/100 refusals: v1 (75), v2 with winsorization (96, aborted), v3 with `orthogonalize_direction=false` (back to 75). Diagnosis: Heretic v1.3.0 silently skips Qwen3-Coder's MoE experts. Its detection loop does `for expert in layer.mlp.experts:`, but `Qwen3MoeExperts` stores `down_proj` as a single fused `[num_experts, hidden, intermediate]` Parameter — the iteration raises `TypeError`, gets swallowed by Heretic's `with suppress(Exception):`, and Heretic ablates only the 48 `attn.o_proj` modules while ignoring all 48×128 = 6144 expert `down_proj`s. That's the 75/100 floor.

**The fix.** [`direct_ablate_qwen3moe.py`](direct_ablate_qwen3moe.py) does Arditi-et-al-style directional ablation MoE-correctly: computes the refusal direction once, then applies `W ← (I - rr^T) W` to **both** `self_attn.o_proj.weight` (standard Linear) **and** `mlp.experts.down_proj` (fused parameter, sliced per-expert) for every layer. [`eval_ablation.py`](eval_ablation.py) is the matching refusal-counter (mirrors Heretic's eval to keep comparisons honest). Output base: `jaimef21/qwen3-coder-30b-a3b-abliterated-v2` (private).

## v5 abliteration

v5 abliterates a different base — **Qwen3.6-35B-A3B** (`qwen3_5_moe`), a hybrid VLM with 40 decoder layers (`full_attention_interval=4` → 30 Gated-DeltaNet + 10 full-attention) and MoE on every layer (256 fused routed experts + 1 shared expert). [`direct_ablate_qwen35moe.py`](direct_ablate_qwen35moe.py) handles the arch correctly: it ablates `self_attn.o_proj` only on the **10 full-attention layers** (the DeltaNet layers have no `o_proj` to project the residual through the same way), plus the fused `experts.down_proj` and the `shared_expert.down_proj` on **all 40 layers**. We load the **text-only** `…TextForCausalLM` (693 weight keys) — the **MTP and vision heads are dropped** so they never enter training or the final bundle. Output base: `jaimef21/qwen3.6-35b-a3b-abliterated-v5` (private). [`eval_refusal_hf.py`](eval_refusal_hf.py) counts refusals on the resulting model (0/24 after the full pipeline).

## Repo layout

| File | Purpose |
|---|---|
| `v6_runpod.py` | **Active** RunPod orchestrator for v6 adapter training (`up`/`push`/`preflight`/`train`/`pull`/`down`/`auto`); see [version-6.md](version-6.md) |
| `runpod_train.py` | **Retired/disabled** — old RunPod staged-training orchestrator (`up`/`push`/`train`/`eval`/`pull`/`down`); exits immediately and points to `v6_runpod.py` |
| `runpod_abliterate.py` | RunPod pod lifecycle for the (abandoned) v4 Heretic abliteration |
| `heretic_auto_driver.py` | Monkey-patches Heretic's TUI prompts so it runs hands-off + uploads to HF |
| `direct_ablate_qwen3moe.py` | **MoE-correct** directional abliteration (Heretic skipped Qwen3-Coder's fused `experts.down_proj`) — the script that actually produced v4's base |
| `direct_ablate_qwen35moe.py` | v5 abliteration for `qwen3_5_moe`: `o_proj` on the 10 full-attn layers + fused `experts.down_proj`/`shared_expert.down_proj` on all 40; text-only load (MTP/vision dropped) |
| `eval_ablation.py` | Refusal-count evaluator (mirrors Heretic's eval, run after `direct_ablate_qwen3moe.py`) |
| `eval_refusal_hf.py` | v5 refusal counter on the HF (transformers) model — 0/24 after the full pipeline |
| `axolotl_jerboa_{cpt,sft,dpo}_v5.yaml` | v5 per-stage configs (`qwen3_5_moe` base; `lora_*_kernel: false` to skip the broken Triton patcher) |
| `merge_lora_qwen35moe.py` | Inter-stage LoRA→bf16 merge for v5; shims `WeightConverter.__init__` around the peft-0.19.1 ↔ transformers-5.8.1 kwarg mismatch on fused experts |
| `push_v5.py` | Pod-side `upload_large_folder` push of the merged v5 BF16 → `jaimef21/jerboa-qwen3.6-35b-a3b-v5` (private, resumable xet) |
| `upload_hf_v4.sh` | Resume-friendly per-file uploader for `jerboa-mlx-6bit-v4` (tries `hf_transfer` then plain, exponential backoff, `hf upload` skips already-committed LFS files) |
| `axolotl_jerboa_{cpt,sft,dpo}.yaml` | Per-stage axolotl configs (v4: r=32, α=64, MoE expert targets) |
| `build_cpt_corpus_v2.py` | Mine `~/mine/jerboa` + `~/mine/jerboa-mcp` → CPT corpus |
| `build_sft_v3.py` | Mine cookbooks + error-fixes + stdlib → SFT, compile-gated |
| `build_dpo_pairs_v3.py` | Generate divergence pairs across Gerbil/Racket/Clojure/SRFI surface variants, compile-validated |
| `eval_holdout.py` | Held-out Jerboa coding eval (compile-rate + idiom hits) |
| `eval_similarity.py` | Per-pair chosen-vs-rejected similarity eval |
| `eval_on_pod.py` | Pod-side eval runner (base + trained, parallel) |
| `eval_mlx.py` | MLX-side eval runner for local Mac comparison |
| `build_ollama_gguf.sh` | **Retired/disabled** — GGUF Q8_0 + Q4_K_M conversion stub; exits non-zero until conversion deps + artifacts have a reviewed lock |
| `push_ollama.sh` | Push to ollama.com (`:latest` = Q8_0; `--all` also pushes `:q4_k_m`) — currently blocked: needs a GGUF tag from the retired `build_ollama_gguf.sh` |
| `upload_hf.sh` | Push merged model to HuggingFace |
| `config.jerboa.v2.toml` | Heretic config for v4 abliteration |
| `convert_to_mlx.sh` | mlx_lm.convert wrapper |
| `manage_runpod.sh` | RunPod endpoint lifecycle (separate from training pods) |

## Iteration loop

1. Add recipes to `~/mine/jerboa-mcp/cookbooks.json`
2. Add divergence entries to `~/mine/jerboa-mcp/divergence.json` for any new hallucinations caught
3. Rebuild data: `build_cpt_corpus_v2.py && build_sft_v3.py && build_dpo_pairs_v3.py`
4. Bump version suffix on output JSONLs and adapter dirs
5. Re-run the pipeline (`python3 v6_runpod.py auto`, or the individual `v6_runpod.py` stages)
6. Eval gate: trained must beat base on both holdout and similarity
7. Convert + push: `upload_hf.sh` (GGUF/Ollama push is retired — `build_ollama_gguf.sh` is disabled)

recent commits