Security hardening and release readiness
ober
29f36d619c4beee73bf22cf4a06d6c695d6377e2
new file mode 100644 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,26 @@ +name: Private Split Gate + +on: + push: + branches: [main, master] + pull_request: + workflow_dispatch: + +permissions: + contents: read + +jobs: + verify-private: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - uses: actions/setup-python@v5 + with: + python-version: "3.12" + + - name: Verify private split gate + run: make verify + + - name: Release evidence smoke + run: make release-evidence --- a/.gitignore +++ b/.gitignore @@ -41,14 +41,26 @@ gguf/ # Logs *.log +dist/ +.pytest_cache/ # Local state (file/job IDs, pod IDs, ssh hosts — keep local) .together_state.json .together_state_v2.json .together_state_v3.json .runpod_state.json +.runpod_state_v6.json .runpod_abliterate_state.json +# v6 adapter-first artifacts +runpod-v6-adapter-peft/ +runpod-v6-smoke-adapter/ +runpod-v6-logs/ +jerboa-v6-base-mlx-*/ +jerboa-v6-adapter-mlx/ +jerboa-v6-adapter-smoke/ +mlx_data_v6*/ + # Editor / agent state .claude/ @@ -57,6 +69,7 @@ __pycache__/ *.pyc *.egg-info/ .venv/ +.venv-mlx/ # Large generated data (track the generator, not the output) # Uncomment these if you want to track the data in git: --- a/.jerboa/security.json +++ b/.jerboa/security.json @@ -1,6 +1,8 @@ { "version": 1, "repo": "jerboa-lora", + "lane": "D/split", + "publication": "private-only; do not publish as-is", "extends": ["jerboa:cli", "jerboa:generated-heavy"], "paths": { "production": ["*.py", "*.ss", "*.sls", "src/**", "scripts/**", "Makefile"], @@ -18,5 +20,18 @@ "eval": { "stringEval": "deny", "bareRead": "deny", "allowReadEval": false }, "secrets": { "scanGeneratedCorpora": true, "forbidTrainingSecrets": true } }, + "required_gates": [ + "make verify", + "make release-evidence" + ], + "public_release_blockers": [ + "dataset license/provenance review", + "base-model license/redistribution review", + "model-card and dataset-card review", + "split provenance proof with reviewed source/data/weight boundaries", + "history secret scan", + "split source-only package", + "remove private paths, accounts, endpoints, and token-file references" + ], "suppressions": [] } new file mode 100644 --- /dev/null +++ b/2026-06-09-163320-local-command-caveatcaveat-the-messages-below.txt @@ -0,0 +1,120 @@ + ▐▛███▜▌ Claude Code v2.1.170 +▝▜█████▛▘ Fable 5 · Claude Max + ▘▘ ▝▝ ~/mine/jerboa-lora + + ▎ Fable 5 is here! Our newest model for complex, long-running work + ▎ Included in your plan limits until Jun 22, then switch to usage credits to continue. + +❯ /model + ⎿ Set model to Fable 5 and saved as your default for new sessions + +❯ Please review this repository and scripts. I've spent about $400 on runpod, + and have almost no real success in training, as quantization seems to remove + all the stuff trained from here. we use a rag/mcp to enforce the language as + much as possible. is it possible we missed something that could have helped? + previously my target was a ~30b parameter model for a 48gb mac pro, this + host, but I now have a 128gb m5 max and am looking at 80GB target mlx + optimized. any ideas to make this more successful? + +⏺ I'll review the repo first — pipeline scripts, quantization steps, and evals — + then give you my take. + + Read 2 files, listed 1 directory (ctrl+o to expand) + +⏺ Explore(Survey quant experiments and pipeline) + ⎿ Done (28 tool uses · 46.6k tokens · 1m 11s) + (ctrl+o to expand) + + Read 2 files (ctrl+o to expand) + +⏺ I read the README/TODO/lora.md, the v5 configs, the quant probe logs (incl. + mixed3 — its P1/P2 are confabulated: spawn-task, the invented + actor-send-system-* list), and had an agent sweep the pipeline. The diagnosis + is clear, and yes — you missed three standard tools, all of which become + available with the 128GB M5 Max. + + Why every quant failed, structurally: the pipeline merges all Jerboa knowledge + into BF16 weights, then quantizes round-to-nearest with zero calibration data + (mlx_lm.convert -q is data-free RTN). A niche fine-tune is a tiny weight + delta; RTN noise is the same order of magnitude, and MoE makes it worse + (knowledge sharded across 256 experts, router brittle). You tuned where to + spend bits (mixed1-3) but never how the bits are chosen. + + The three misses: + 1. Keep knowledge out of the quantization path entirely. Quantize the base, + train the adapter on the quantized model (mlx_lm.lora does QLoRA natively), + serve with --adapter-path, never fuse. The fp16 adapter holds all Jerboa + knowledge — quantization can't erase what it never touches. Bonus: 3B-active + QLoRA trains fine on the M5 Max → no more $400 RunPod cycles (pod only needed + to abliterate a new base). + 2. Calibrated quantization exists in mlx-lm: mlx_lm.dwq (distills the BF16 + teacher into the quantized weights), mlx_lm.awq, mlx_lm.dynamic_quant + (per-layer sensitivity). DWQ with your Jerboa corpus as calibration is + purpose-built for "quant erased my fine-tune." Was impossible on 48GB (teacher + alone is 69GB); fits in 128GB. + 3. 8-bit uniform was never knowledge-tested — it only Metal-OOM'd under this + host's 32GB wired cap. On the M5 Max it loads with ~90GB to spare and may + simply pass. Don't carry the 32GB sysctl over; that cap was this machine's + panic threshold. + + Cheapest-first plan on the M5 Max: (1) copy jerboa-mlx-8bit-v5 + eval over, + eval as-is — $0; (2) if marginal, DWQ-requant from the BF16 with Jerboa + calibration; (3) belt-and-suspenders: short QLoRA "healing" SFT on the + quantized model (data self-distilled from your BF16), served unfused. For + v6/80GB: Qwen3-Next-80B-A3B at 6-bit (~64GB, same ~3B-active speed) — but + trained as QLoRA on the quantized base from day one; dense Qwen3-32B is the + slower fallback that degrades gracefully under quant. + + Want me to expand the QLoRA-unfused recipe, the DWQ step, or set up the M5 + transfer/eval? + +✻ Churned for 4m 54s + +❯ Please write a new plan for a v6 with all your recommendations, and steps, to + hand this off to another model to implement! be as detailed as possible. Add + all the gotchas of what bit us and ways to avoid that going forward! + +⏺ I'll gather the exact configs, flags, eval harness, and current MLX tooling so + the plan is concrete and correct, then write the handoff doc. + + Read 6 files (ctrl+o to expand) + +⏺ Fable 5's safety measures flagged this message for cybersecurity or biology topics. They may flag safe, normal content as well. These measures let us bring you Mythos-level capability in other areas sooner, and we're working to refine them. Switched to Opus 4.8. Send feedback with /feedback or learn more + ⎿ Tip: You can configure model switch behavior in /config + +⏺ Now the eval grounding and the current MLX toolchain — I need to confirm which + calibrated-quant tools are actually installed. + + Read 1 file (ctrl+o to expand) + +⏺ Bash(cd /Users/user/mine/jerboa-lora + echo "=== mlx-lm version (venv-mlx) ==="…) + ⎿ === mlx-lm version (venv-mlx) === + 0.31.3 + === mlx core version === + … +20 lines (ctrl+o to expand) + ⎿ Interrupted · What should Claude do instead? + +❯ /copy + ⎿ Copied to clipboard (125 characters, 1 lines) + Also written to /tmp/claude-501/response.md + +❯ /model + ⎿ Set model to Fable 5 and saved as your default for new sessions + +❯ Please write a new plan for a v6 with all your recommendations, and steps, to + hand this off to another model to implement! be as detailed as possible. Add + all the gotchas of what bit us and ways to avoid that going forward! + +⏺ Bash(cd /Users/user/mine/jerboa-lora + echo "=== latest mlx-lm on PyPI ==="…) + ⎿ === latest mlx-lm on PyPI === + /bin/bash: line 8: .venv-mlx/bin/pip: No such file or directory + === mlx_lm.server adapter support === + … +38 lines (ctrl+o to expand) +⏺ Fable 5's safety measures flagged this message for cybersecurity or biology topics. They may flag safe, normal content as well. These measures let us bring you Mythos-level capability in other areas sooner, and we're working to refine them. Switched to Opus 4.8. Send feedback with /feedback or learn more + ⎿ Tip: You can configure model switch behavior in /config + ⎿ Interrupted · What should Claude do instead? + +❯ /feedback + ⎿ Feedback / bug report submitted \ No newline at end of file new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. new file mode 100644 --- /dev/null +++ b/Makefile @@ -0,0 +1,42 @@ +SHELL := /bin/sh + +PYTHON ?= python3 +EVIDENCE_DIR ?= dist/release-evidence +PROVENANCE_DIR ?= dist/provenance + +.PHONY: security test provenance-evidence verify release-evidence clean + +security: + sh scripts/security-check.sh + +test: + files="$$(git ls-files --cached --others --exclude-standard '*.py')"; \ + if [ -n "$$files" ]; then $(PYTHON) -m py_compile $$files; fi + +provenance-evidence: + JLORA_PROVENANCE_DIR="$(PROVENANCE_DIR)" \ + JLORA_SPLIT_PROVENANCE_PROOF_FILE="$(JLORA_SPLIT_PROVENANCE_PROOF_FILE)" \ + JLORA_REQUIRE_SPLIT_PROVENANCE_PROOF="$(JLORA_REQUIRE_SPLIT_PROVENANCE_PROOF)" \ + sh scripts/provenance-evidence.sh + +verify: security test provenance-evidence + +release-evidence: + rm -rf $(EVIDENCE_DIR) + mkdir -p $(EVIDENCE_DIR) + $(MAKE) security >$(EVIDENCE_DIR)/security.log 2>&1 + $(MAKE) test >$(EVIDENCE_DIR)/python-syntax.log 2>&1 + $(MAKE) provenance-evidence >$(EVIDENCE_DIR)/provenance-evidence.log 2>&1 + git rev-parse HEAD >$(EVIDENCE_DIR)/git-commit.txt + git status --short --ignored >$(EVIDENCE_DIR)/git-status-ignored.txt + git diff --stat >$(EVIDENCE_DIR)/diff-stat.txt + git ls-files >$(EVIDENCE_DIR)/tracked-files.txt + git ls-files --cached --others --exclude-standard >$(EVIDENCE_DIR)/working-files.txt + git status --ignored --short | sed -n '/^!!/p' >$(EVIDENCE_DIR)/ignored-artifacts.txt + git ls-files --cached --others --exclude-standard | grep -E '(\.py|\.sh|\.yaml|\.toml|\.json|\.md|Modelfile|LICENSE|SECURITY.md)$$' | xargs shasum -a 256 >$(EVIDENCE_DIR)/source-sha256.txt + rm -rf $(EVIDENCE_DIR)/provenance + cp -R $(PROVENANCE_DIR) $(EVIDENCE_DIR)/provenance + { $(PYTHON) --version; uname -a; } >$(EVIDENCE_DIR)/build-env.txt + +clean: + rm -rf dist new file mode 100644 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,55 @@ +# Security Policy + +`jerboa-lora` is a private training and model-artifact workspace. It contains +training corpora, LoRA pipeline scripts, remote GPU orchestration helpers, and +references to private model repositories. It is **not publishable as-is**. + +## Supported Status + +Lane: `D/split`. + +No public production-support commitment exists for this repository. The +production-worthy posture is to keep this repo private and split any future +public release into separately reviewed artifacts: + +- source-only training/pipeline code; +- documented dataset manifests with license/provenance review; +- model cards and weight artifacts with explicit redistribution rights; +- split-provenance evidence proving source, data, and weights were reviewed as + separate artifacts; +- sanitized examples that do not contain private checkout paths, pod IDs, + endpoint URLs, account names, local state, or token-file details. + +## Required Gate + +Run: + +```sh +make verify +make release-evidence +``` + +The gate proves only that this private workspace has basic safety controls. It +does not authorize public release. Public release remains blocked until dataset +licenses, base-model licenses, generated outputs, private account references, +and history secrets are reviewed. + +`make provenance-evidence` records private model/data cards, source/data/config +hashes, local model artifact inventory, and split-review proof status. Required +split proof fails closed unless it includes reviewed base-model, dataset, model +card, dataset card, source split, and weight-publication markers. + +## Secret And Artifact Rules + +- Never commit token files, pod state, endpoint state, SSH keys, Hugging Face + credentials, Together/RunPod credentials, logs, model weights, adapters, GGUF + files, or local virtual environments. +- Keep `*.jsonl` training data private unless every record has provenance and + license clearance. +- Do not publish model outputs from this repo without a model card, dataset + card, safety/eval report, and redistribution decision. + +## Reporting + +Before any public split, report issues privately to the repository owner. A +public advisory contact must be added to the split artifact before release. new file mode 100644 --- /dev/null +++ b/axolotl_jerboa_v6_cpt.yaml @@ -0,0 +1,75 @@ +# v6 Stage 1: CPT adapter on Jerboa source/context. +# +# Invariant: this file trains a PEFT adapter only. Do not add merge settings. + +base_model: __V6_BASE__ +model_type: AutoModelForCausalLM +tokenizer_type: AutoTokenizer +trust_remote_code: true + +load_in_4bit: true +adapter: qlora +bnb_4bit_quant_type: nf4 +bnb_4bit_compute_dtype: bfloat16 +bnb_4bit_use_double_quant: true +quantize_moe_experts: true +bf16: true +tf32: true + +datasets: + - path: /workspace/data/cpt_v6.jsonl + type: completion + field: text + +dataset_prepared_path: /workspace/cache/v6_cpt +val_set_size: 0.02 +output_dir: /workspace/output_v6_adapter_cpt +dataset_processes: 16 + +sequence_len: 2048 +sample_packing: true +pad_to_sequence_len: true + +lora_r: 32 +lora_alpha: 64 +lora_dropout: 0 +lora_target_modules: + - q_proj + - k_proj + - v_proj + - o_proj + - in_proj_qkvz + - in_proj_ba + - out_proj + - gate_proj + - up_proj + - down_proj +lora_target_parameters: + - mlp.experts.gate_up_proj + - mlp.experts.down_proj + +lora_mlp_kernel: false +lora_qkv_kernel: false +lora_o_kernel: false + +gradient_accumulation_steps: 8 +micro_batch_size: 1 +num_epochs: 2 +optimizer: paged_adamw_8bit +lr_scheduler: cosine +learning_rate: 2.0e-5 +warmup_ratio: 0.05 +weight_decay: 0.0 + +flash_attention: true +gradient_checkpointing: true +gradient_checkpointing_kwargs: + use_reentrant: false + +logging_steps: 10 +saves_per_epoch: 1 +save_total_limit: 1 +evals_per_epoch: 2 + +special_tokens: + pad_token: <|endoftext|> new file mode 100644 --- /dev/null +++ b/axolotl_jerboa_v6_dpo.yaml @@ -0,0 +1,75 @@ +# v6 Stage 3: continue the SFT adapter through DPO. +# +# Invariant: this continues from lora_model_dir and writes the final PEFT +# adapter. Do not fuse this adapter into the base for the v6 artifact. + +base_model: __V6_BASE__ +model_type: AutoModelForCausalLM +tokenizer_type: AutoTokenizer +trust_remote_code: true + +load_in_4bit: true +adapter: qlora +bnb_4bit_quant_type: nf4 +bnb_4bit_compute_dtype: bfloat16 +bnb_4bit_use_double_quant: true +quantize_moe_experts: true +bf16: true +tf32: true + +lora_model_dir: __PREVIOUS_ADAPTER__ +rl: dpo + +datasets: + - path: /workspace/data/dpo_v6.jsonl + type: chatml.icr + +dataset_prepared_path: /workspace/cache/v6_dpo +output_dir: /workspace/output_v6_adapter_dpo +dataset_processes: 8 + +sequence_len: 2048 + +lora_r: 32 +lora_alpha: 64 +lora_dropout: 0 +lora_target_modules: + - q_proj + - k_proj + - v_proj + - o_proj + - in_proj_qkvz + - in_proj_ba + - out_proj + - gate_proj + - up_proj + - down_proj +lora_target_parameters: + - mlp.experts.gate_up_proj + - mlp.experts.down_proj + +lora_mlp_kernel: false +lora_qkv_kernel: false +lora_o_kernel: false + +gradient_accumulation_steps: 4 +micro_batch_size: 1 +num_epochs: 3 +optimizer: paged_adamw_8bit +lr_scheduler: cosine +learning_rate: 5.0e-6 +warmup_ratio: 0.1 +weight_decay: 0.0 +max_grad_norm: 1.0 + +flash_attention: true +gradient_checkpointing: true +gradient_checkpointing_kwargs: + use_reentrant: false + +logging_steps: 2 +saves_per_epoch: 1 +save_total_limit: 1 + +special_tokens: + pad_token: <|endoftext|> new file mode 100644 --- /dev/null +++ b/axolotl_jerboa_v6_sft.yaml @@ -0,0 +1,84 @@ +# v6 Stage 2: continue the CPT adapter through SFT. +# +# Invariant: this continues from lora_model_dir and writes a new adapter. +# The release artifact remains an adapter directory, not fused weights. + +base_model: __V6_BASE__ +model_type: AutoModelForCausalLM +tokenizer_type: AutoTokenizer +trust_remote_code: true + +load_in_4bit: true +adapter: qlora +bnb_4bit_quant_type: nf4 +bnb_4bit_compute_dtype: bfloat16 +bnb_4bit_use_double_quant: true +quantize_moe_experts: true +bf16: true +tf32: true + +lora_model_dir: __PREVIOUS_ADAPTER__ + +datasets: + - path: /workspace/data/sft_v6.jsonl + type: chat_template + chat_template: tokenizer_default + field_messages: messages + message_field_role: role + message_field_content: content + +dataset_prepared_path: /workspace/cache/v6_sft +val_set_size: 0.02 +output_dir: /workspace/output_v6_adapter_sft +dataset_processes: 16 + +sequence_len: 2048 +sample_packing: true +pad_to_sequence_len: true + +lora_r: 32 +lora_alpha: 64 +lora_dropout: 0 +lora_target_modules: + - q_proj + - k_proj + - v_proj + - o_proj + - in_proj_qkvz + - in_proj_ba + - out_proj + - gate_proj + - up_proj + - down_proj +lora_target_parameters: + - mlp.experts.gate_up_proj + - mlp.experts.down_proj + +lora_mlp_kernel: false +lora_qkv_kernel: false +lora_o_kernel: false + +gradient_accumulation_steps: 8 +micro_batch_size: 1 +num_epochs: 2 +optimizer: paged_adamw_8bit +lr_scheduler: cosine +learning_rate: 1.0e-4 +warmup_ratio: 0.05 +weight_decay: 0.0 + +train_on_inputs: false +group_by_length: false + +flash_attention: true +gradient_checkpointing: true +gradient_checkpointing_kwargs: + use_reentrant: false + +logging_steps: 5 +saves_per_epoch: 1 +save_total_limit: 1 +evals_per_epoch: 2 + +special_tokens: + pad_token: <|endoftext|> new file mode 100644 --- /dev/null +++ b/build_v6_prep_data.py @@ -0,0 +1,729 @@ +#!/usr/bin/env python3 +""" +Build v6 prep datasets from suggestions.md. + +This intentionally does not overwrite the v3 datasets. It emits add-on files +and combined v6 prep files: + + sft_v6_suggestions.jsonl + dpo_v6_suggestions.jsonl + cpt_corpus_v6_suggestions.jsonl + sft_v6_prep.jsonl + dpo_pairs_v6_prep.jsonl + cpt_corpus_v6_prep.jsonl + training_data_together_v6_prep.jsonl + +The harvested data focuses on the failure modes called out in suggestions.md: +jcode repair/tool discipline, jerboa-qt API use, wrong-dialect tripwires, +small Tetris ladder rungs, and arity/signature flashcards. +""" + +from __future__ import annotations + +import hashlib +import json +import os +import re +from pathlib import Path +from typing import Iterable + + +REPO = Path(__file__).resolve().parent +MINE = Path.home() / "mine" +JERBOA = MINE / "jerboa" +JERBOA_CODE = MINE / "jerboa-code" +JERBOA_QT = MINE / "jerboa-qt" +JERBOA_MCP = MINE / "jerboa-mcp" + +SFT_ADDON = REPO / "sft_v6_suggestions.jsonl" +DPO_ADDON = REPO / "dpo_v6_suggestions.jsonl" +CPT_ADDON = REPO / "cpt_corpus_v6_suggestions.jsonl" + +SFT_COMBINED = REPO / "sft_v6_prep.jsonl" +DPO_COMBINED = REPO / "dpo_pairs_v6_prep.jsonl" +DPO_AXOLOTL = REPO / "dpo_pairs_v6_axolotl.jsonl" +CPT_COMBINED = REPO / "cpt_corpus_v6_prep.jsonl" +TRAINING_TOGETHER_COMBINED = REPO / "training_data_together_v6_prep.jsonl" + +SYSTEM_JERBOA = ( + "You are an expert in Jerboa Scheme, a Chez-Scheme-based dialect with a " + "Gerbil-flavored prelude. You provide accurate, idiomatic Jerboa code " + "with correct imports, function names, and arities. Module paths use " + "the (jerboa ...) and (std ...) forms -- never :std/foo (Gerbil) or " + "(srfi :NN) (R7). When writing code, always include required (import ...) " + "statements." +) + +SYSTEM_DPO = ( + "You are an expert in Jerboa Scheme, a Chez-Scheme-based dialect with a " + "Gerbil-flavored prelude. Use only valid Jerboa forms -- never reach for " + "Racket, Gerbil, Gambit, Clojure, Common Lisp, or SRFI surface syntax. " + "Module paths use the (jerboa ...) and (std ...) forms -- never :std/foo " + "(Gerbil) or (srfi :NN) (R7). When writing code, always include required " + "(import ...) statements." +) + +SYSTEM_JCODE = ( + "You are using jcode verified workflow tools. Follow the verified " + "edit -> verify -> repair loop, use only tools exposed by the workflow, " + "and call done only after verify passes." +) + +SYSTEM_QT = ( + "You are an expert in Jerboa Scheme and jerboa-qt. Use (import " + "(jerboa-qt qt)), with-qt-app, correct qt-* arities, numeric RGBA painter " + "arguments, and verifier-owned screenshots for tests." +) + + +def dumps(obj: object) -> str: + return json.dumps(obj, ensure_ascii=False, separators=(",", ":")) + + +def scheme_block(code: str) -> str: + return "```scheme\n" + code.strip() + "\n```" + + +def sh_block(code: str) -> str: + return "```sh\n" + code.strip() + "\n```" + + +def chat(user: str, assistant: str, system: str = SYSTEM_JERBOA) -> dict: + return { + "messages": [ + {"role": "system", "content": system}, + {"role": "user", "content": user}, + {"role": "assistant", "content": assistant}, + ] + } + + +def pref(instruction: str, chosen: str, rejected: str, system: str = SYSTEM_DPO) -> dict: + return { + "system": system, + "instruction": instruction, + "chosen_response": chosen, + "rejected_response": rejected, + } + + +def read_text(path: Path) -> str: + try: + return path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + return "" + + +def extract_scheme_body(text: str) -> str: + """Drop the shell polyglot prelude from jerboa-qt examples.""" + marker = "|#" + if marker in text and text.lstrip().startswith("#!/bin/sh"): + return text.split(marker, 1)[1].lstrip() + return text.strip() + + +def one_line(s: str, limit: int = 120) -> str: + s = re.sub(r"\s+", " ", s.strip()) + return s if len(s) <= limit else s[: limit - 3] + "..." + + +def stable_key(obj: dict) -> str: + return hashlib.sha256(dumps(obj).encode("utf-8")).hexdigest() + + +def instruction_variants(base: str) -> list[str]: + return [ + base, + "Choose the better next action. " + base, + "Reject the tempting but wrong behavior. " + base, + ] + + +def write_jsonl(path: Path, rows: Iterable[dict]) -> int: + seen: set[str] = set() + count = 0 + with path.open("w", encoding="utf-8") as f: + for row in rows: + key = stable_key(row) + if key in seen: + continue + seen.add(key) + f.write(dumps(row) + "\n") + count += 1 + return count + + +def iter_jsonl(path: Path) -> Iterable[dict]: + if not path.exists(): + return + with path.open(encoding="utf-8") as f: + for line in f: + line = line.strip() + if line: + yield json.loads(line) + + +def combine_jsonl(out: Path, inputs: list[Path]) -> int: + seen: set[str] = set() + count = 0 + with out.open("w", encoding="utf-8") as f: + for path in inputs: + if not path.exists(): + continue + for row in iter_jsonl(path): + key = stable_key(row) + if key in seen: + continue + seen.add(key) + f.write(dumps(row) + "\n") + count += 1 + return count + + +def write_axolotl_dpo_jsonl(src: Path, out: Path) -> int: + rows = [] + for row in iter_jsonl(src): + rows.append({ + "system": row.get("system", ""), + "input": row["instruction"], + "chosen": row["chosen_response"], + "rejected": row["rejected_response"], + }) + return write_jsonl(out, rows) + + +# --------------------------------------------------------------------------- +# jcode repair and tool discipline + +REPAIR_FAILURES = [ + ( + "Unexpected close ) at line 157", + "Call balance(path), read a small span around line 157, then use " + "line_edit for one bad line or replace_range for the enclosing " + "unbalanced region. Verify again after the edit.", + "Call sed/head/tail repeatedly and keep trying old_str replacements.", + ), + ( + "old_str not found while replacing a generated function", + "Read the exact line span, then use replace_def(path,name,content) if " + "the function is balanced. If it is unbalanced, use replace_range with " + "the line numbers.", + "Retry the same brittle exact edit with small whitespace changes.", + ), + ( + "write is not an available tool in this verified workflow", + "Use edit with path and content to create or replace the scoped file. " + "Use content as the canonical argument name.", + "Call write again or probe the shell to create the file.", + ), + ( + "verify failed after the file was written", + "Repair the code. Use balance/read to locate the failure, edit the " + "smallest useful region, and call verify again.", + "Summarize the failure or call done without a passing verify.", + ), + ( + "missing whitespace after set! in (set!*score* 10)", + "Use line_edit or replace_range to write (set! *score* 10), then " + "verify.", + "Keep the malformed call shape and explain that it is probably okay.", + ), + ( + "empty directory: target repo has no files yet", + "Create the requested scoped file with edit(path, content). Do not " + "loop on ls/read of the empty directory.", + "Repeatedly list the directory or ask which command exists.", + ), + (