first commit
ober
f94581e1275aa1e6571e47ae59d388908d885c73
new file mode 100644 --- /dev/null +++ b/README.md @@ -0,0 +1,219 @@ +# Jerboa Scheme LoRA + +A fine-tuned Qwen 2.5 7B model that knows Jerboa Scheme. Run locally with Ollama, host on RunPod, or pull from the registry. + +Jerboa is a Chez-Scheme-based dialect with a Gerbil-flavored prelude. This LoRA teaches the base model the Jerboa-specific module syntax (`(std foo)`, `(jerboa prelude)` — not `:std/foo`), the standard library, the actor/fiber system, the FFI, and how Jerboa diverges from Gerbil/Racket/Clojure/SRFI. + +## Quick Start (Local) + +```bash +ollama pull jaimef/jerboa-qwen +ollama run jaimef/jerboa-qwen "How do I import the Jerboa prelude and parse JSON?" +``` + +## Deployment Options + +| Option | Cost | Speed | Setup | +|--------|------|-------|-------| +| **Local Ollama (GPU)** | Free | 30–40 tok/s | `ollama pull jaimef/jerboa-qwen` | +| **Local Ollama (CPU)** | Free | 5–10 tok/s | Same as above | +| **RunPod Serverless** | $0 idle, ~$0.39/hr active | 30–40 tok/s | `./deploy_runpod.sh` | +| **Together AI Endpoint** | $6.60/hr always-on | Fast | Not recommended | + +### Estimated RunPod monthly costs + +| Usage | Hours/month | Cost/month | +|-------|-------------|------------| +| Idle (scale-to-zero) | 0 | **$0** | +| Light (1hr/day) | ~30 | **~$10** | +| Moderate (3hr/day) | ~90 | **~$31** | +| Heavy (8hr/day) | ~240 | **~$82** | + +## Use with OpenCode + +### Option A: Local Ollama + +Add to `~/.config/opencode/opencode.json`: + +```json +{ + "$schema": "https://opencode.ai/config.json", + "provider": { + "ollama": { + "npm": "@ai-sdk/openai-compatible", + "name": "Ollama (local)", + "options": { + "baseURL": "http://localhost:11434/v1" + }, + "models": { + "jerboa-qwen": { + "name": "Jerboa Qwen" + } + } + } + } +} +``` + +Or run `./configure_opencode.sh ollama` to write that for you. + +### Option B: RunPod Serverless (recommended if no local GPU) + +```json +{ + "$schema": "https://opencode.ai/config.json", + "provider": { + "runpod": { + "npm": "@ai-sdk/openai-compatible", + "name": "RunPod (serverless)", + "options": { + "baseURL": "https://api.runpod.ai/v2/<ENDPOINT_ID>/openai/v1", + "apiKey": "<RUNPOD_API_KEY>" + }, + "models": { + "jerboa-qwen": { + "name": "Jerboa Qwen 7B" + } + } + } + } +} +``` + +Replace `<ENDPOINT_ID>` and `<RUNPOD_API_KEY>` with your values. + +## Deploy to RunPod + +One script handles everything: downloads merged model, uploads to HuggingFace, creates RunPod endpoint via API. + +```bash +# Prerequisites +pip install together huggingface_hub +export TOGETHER_API_KEY="your-key" +export RUNPOD_API_KEY="your-key" +hf auth login + +# Deploy (after training completes) +./deploy_runpod.sh jaimef21/jerboa-qwen-7b +``` + +The script reads the Together AI job ID from `.together_state.json` (set by `train_together.py train`), so you don't need to paste it in. + +Endpoint URL: `https://api.runpod.ai/v2/<ENDPOINT_ID>/openai/v1` + +## Build from Source + +### 1. Generate training data + +```bash +# These should already exist on your machine: +# ~/mine/jerboa (the Jerboa source repo) +# ~/mine/jerboa-mcp (the Jerboa MCP server with cookbooks/api/divergence) + +python3 convert_training_data.py +# → training_data.jsonl (~9MB, 4,622 entries) +``` + +### 2. Train on Together AI (~$3, ~7 minutes) + +```bash +pip install together +export TOGETHER_API_KEY="your-key" + +python3 train_together.py upload +python3 train_together.py train +python3 train_together.py status +``` + +### 3. Deploy + +**Local (with GPU or slow CPU):** +```bash +./download_and_convert.sh +``` + +**Hosted (RunPod serverless):** +```bash +export RUNPOD_API_KEY="your-key" +hf auth login +./deploy_runpod.sh YOUR_USERNAME/jerboa-qwen-7b +``` + +### 4. Verify + +```bash +python3 verify_model.py \ + --base-url http://localhost:11434/v1 \ + --model jerboa-qwen -v +``` + +### 5. Push to Ollama registry + +```bash +./push_ollama.sh YOUR_USERNAME +``` + +## Training Data + +**4,622 entries** generated from the Jerboa source tree and the jerboa-mcp knowledge base. + +| Source | Count | Description | +|--------|-------|-------------| +| doc | 2,320 | jerboa/docs/*.md (architecture, fiber, capability, FFI, ...) + top-level docs (JERBOA-LANG, README, CLAUDE) | +| cookbook | 924 | Verified working code recipes from cookbooks.json | +| api | 626 | Per-module export catalogs from api-signatures.json | +| test | 270 | Real usage examples from jerboa/tests/test-*.ss | +| divergence | 238 | Cross-dialect "wrong → right" pairs (Racket/Gerbil/Clojure/SRFI → Jerboa) | +| errorfix | 112 | Error pattern → fix mappings | +| security | 83 | Vulnerability patterns and remediation | +| convention | 24 | Hand-written Jerboa idiom teaching examples (3× weighted) | +| std-source | 21 | Source of key stdlib modules (prelude, fiber, error, sort, ...) | +| source | 4 | Tutorial examples from jerboa/examples/ | + +### Output formats + +| File | Format | Use with | +|------|--------|----------| +| `training_data_together.jsonl` | Together AI messages | Together AI fine-tuning | +| `training_data.jsonl` | ChatML/ShareGPT | LLaMA-Factory, Axolotl, Unsloth | +| `training_data_alpaca.jsonl` | Alpaca JSONL | Unsloth, HuggingFace | + +## Scripts + +| Script | Purpose | +|--------|---------| +| `convert_training_data.py` | Generate training data from jerboa + jerboa-mcp | +| `train_together.py` | Upload, train, and monitor on Together AI | +| `download_and_convert.sh` | Download adapter, convert to GGUF, set up Ollama | +| `deploy_runpod.sh` | Upload merged model to HuggingFace, create RunPod endpoint | +| `manage_runpod.sh` | RunPod endpoint lifecycle (list, health, delete, purge) | +| `push_ollama.sh` | Tag and push model to Ollama registry | +| `configure_opencode.sh` | Generate OpenCode config for Ollama/RunPod | +| `verify_model.py` | Run 10 Jerboa-specific test prompts | +| `train_unsloth.py` | Local GPU training with Unsloth | +| `merge_and_export.py` | Merge adapter + base to GGUF (needs 32GB RAM or GPU) | +| `train_runpod.sh` | One-shot training on rented GPU | +| `Modelfile` | Ollama model definition | + +## Iterating + +To improve the model with more training data: + +1. Add recipes to `~/mine/jerboa-mcp/cookbooks.json` +2. Add divergence entries to `~/mine/jerboa-mcp/divergence.json` for any new hallucinations you catch +3. `python3 convert_training_data.py` +4. `python3 train_together.py upload` +5. `python3 train_together.py train` +6. `./download_and_convert.sh` (local) or `./deploy_runpod.sh` (hosted) + +## Why a Jerboa-specific model? + +Out-of-the-box LLMs hallucinate Gerbil/Racket/Clojure/SRFI forms when asked for Jerboa code: +- `(import :std/sort)` instead of `(import (std sort))` +- `hash-has-key?` instead of `hash-key?` +- `define-condition-type` instead of the Jerboa `(std error)` helpers +- `(srfi :NN)` instead of `(srfi NN)` + +The 238 divergence entries + 626 API-signature entries + 924 cookbook recipes train the model on Jerboa's actual surface area, so generated code compiles and runs. +# jerboa-lora +# jerboa-lora