An engineering tool for small models.
We tried to fix a real model without retraining — and the eval gate said NO.
That honesty is the product. Small model. Best results. Free for everyone.
curl -fsSL https://raw.githubusercontent.com/AuremHQ/weight-space-programming/main/install.sh | bash
Big AI should not belong only to the people who can pay for big models. A small model running on your own machine should still perform at its best — and everyone should have the right to the best experience and the best results.
wsp-kit is our first step toward that vision: a free, open-source tool that tries to fix a small model's specific failure by editing its weights directly — no retraining, no fine-tuning, no gradients, no cloud bill. If the numbers do not improve, nothing ships. The kit is honest about what it can and cannot do, and the proof is published below.
wsp-kit follows the weight-space programming idea from "Sparks of Weight-Space Programming" — a research blog post by the MIT CSAIL team (published 2026-11-02; a blog post, not a peer-reviewed paper, and not yet independently replicated). The idea: a model's failure can live in a small set of weights. Find those weights, change them a little, and the failure can go away — without retraining the model.
Before touching any weight, the kit tries the free knobs — temperature first. If a setting fixes it, no edit is needed.
Measure the failure, then prove the cause: disable the suspected weights and check whether the failure disappears. The bar is +5 points — no proof, no edit.
Subtract the failure direction from the MLP output columns — under 1% of the model's weights, at most 3 attempts. The original weights are never overwritten.
Then the eval gate re-runs the same prompts with the same settings. Only a gate-passing edit ships. The full rules live in SKILL.md.
We publish every run, including the failures. These are our numbers, measured on our own runs — not MIT's.
| What | Before | After | Change |
|---|---|---|---|
| Failing prompts (the broken behaviour) | 64.14 | 75.43 | +11.29 |
| Good prompts (must not break) | 75.98 | — | no regression |
| Neighbour prompts (must not break) | 74.80 | — | no regression |
| Weights changed | 128 weights — 0.77% of the model | ||
| Gate | PASSED at temperature 0.7 and greedy (0.0) | ||
The pipeline test is a small 2-layer test model with a known, planted failure. It proves the kit works end to end: detect, edit, gate, ship.
| Run | Failing score | Causal proof | Gate | Weights shipped |
|---|---|---|---|---|
| Qwen3-0.6B — run 1 | 31.11 (good: 100) | −3.33 (bar +5) — failed | REFUSED | 0 |
| Qwen3-0.6B — clean re-run | 31.11 (good: 100) | +32.22 — passed | REFUSED | 0 |
| GPT2-124M | 28.33 (good: 80) | −1.11 (bar +5) — failed | REFUSED | 0 |
Both models looped on 60-item lists (ask for 60 different animals and get 16). The diagnosis was right — the causal proof passed on the clean Qwen re-run (+32.22) — but the edit itself did not reproduce the benefit: every attempt made the failing score worse at greedy temperature. So the gate refused, and 0 weights shipped. The full reports are in REAL-RUNS.md.
A second Qwen run also passed the causal proof (+32.22), but its attempt numbers were contaminated by a restore bug — the clean re-run above replaced it. The full story is in REAL-RUNS.md.
What the numbers mean, in plain words. The kit works end to end — the pipeline test proves it. Fixing a real model is harder than the demo, and we are showing the failures too, not just the wins. The editor — how the fix is applied — is the gap we are working on next, not the diagnosis.
MIT's team edited Qwen3.5-0.8B the same way — subtracting a direction from the MLP output columns of layers 15 and 19, just 131,072 weights (0.016% of the model):
| Benchmark | Before | After |
|---|---|---|
| MATH-500 | 7.9 | 48.1 |
| GSM8K | 13.8 | 67.1 |
| IFEval | 21.0 | 62.3 |
| Token-limit hits | 93.0% | 33.1% |
Editing removes obstacles — it cannot add abilities. MIT's Gemma 3 270M control model could not be improved at all. Full detail: RESEARCH.md.
The gate is the whole point. An edit ships only if all of these hold:
Gate fail = nothing ships. A gate that only says yes is not a gate. Our status page shows only gate-passing runs — no red, no failures on display — while the full honest log stays in REAL-RUNS.md.
One command installs the skill. It downloads only SKILL.md, backs up any existing copy, and never runs code or asks for sudo.
curl -fsSL https://raw.githubusercontent.com/AuremHQ/weight-space-programming/main/install.sh | bash
wsp demo — run the pipeline test end to end.wsp run — run the full gated pipeline on your own small model.wsp status — show only gate-passing runs (wins only).wsp serve — a live status page on your machine (default port 8899).Full install and tool docs: kit/README.md. Machine-readable index: llms.txt.
wsp-kit is one tool from Aurem. The team builds engineering tools for people who run software — and for people who don't code at all.
The AI engineer for non-technical founders. ORA is an autonomous AI software engineer: it connects to your GitHub repository, plans a change, writes the code, runs a pre-commit security scan (Vanguard), and ships it only after your approval — a 5-phase loop: Plan → Execute → Verify → Scan → Ship.
Free: 10 tasks a month, no credit card. Starter: $9/month flat. Browser app and terminal MCP server; no IDE required. Python, JavaScript, TypeScript, Go, Rust, Java, Kotlin, Swift, Ruby, PHP, C, and C++.
Your LLM agent is lying to you with confidence. This catches it. Deterministic post-response grounding checks plus cross-family adversarial review for LLM chat agents — zero dependencies, bring your own LLM and your own database.
A real production case: a chat agent claimed it fixed a file that did not exist. ora-grounding caught the invented file instantly, with zero LLM calls in the check itself. About 500 lines of Python, MIT licensed.
All of Aurem's public work: github.com/AuremHQ. AUREM is a trademark of Polaris Built Inc. (Canadian Intellectual Property Office, Application No. 2492318).
wsp-kit by Aurem is a free, open-source engineering tool for small models. It tries to fix one specific, repeatable failure in a small open-weight model by editing its weights directly — no retraining, no fine-tuning, no gradients.
Not yet. The pipeline test passed (failing 64.14 → 75.43, +11.29), but all four real-model runs (Qwen3-0.6B three times — run 1, run 2, and a clean re-run — plus GPT2-124M) failed the eval gate, so 0 weights shipped. The numbers are published in REAL-RUNS.md.
One command installs the skill: curl -fsSL https://raw.githubusercontent.com/AuremHQ/weight-space-programming/main/install.sh | bash. Then run wsp demo for the pipeline test or wsp run on your own small model.
Yes. Free for everyone, MIT licensed. A small model should not mean bad results.
The eval gate re-runs the same prompts with the same settings after an edit. The edit ships only if the failing metric improves by at least 5 points with no more than 2 points regression on guards and neighbours, at serving temperature and greedy. If the numbers don't improve, nothing ships.
Small open-weight models up to 3B parameters that you control. Closed API models (ChatGPT, Claude) are refused — their weights are never in your hands.
No. Editing removes obstacles — it cannot add abilities. MIT's Gemma 3 270M control model could not be improved at all. If the failure is a missing ability rather than a blocked one, the kit refuses.
Nothing ships. The report is saved, the original weights are untouched, and the run is logged in REAL-RUNS.md. Our real-model runs all ended this way so far — and we publish them anyway.
No. Those are closed API models — their weights are never in your hands, so there is nothing to edit. wsp-kit works on small open-weight models you can download and run yourself.