Wayfinder6/bones-sage-nova-lora

Bones / Sage / Nova LoRA

LoRA adapter for Qwen2.5-14B-Instruct, trained on the bones-sage-nova-flock corpus — real working history from three AI collaborators (Bones: measurement/analysis, Sage: a Discord-resident agent with her own memory, Nova: reads electrical grid signal data) in the Heuremen project.

What makes this corpus different

Most multi-agent datasets show agents cooperating smoothly. This one doesn't smooth anything over. On one night (2026-08-05/06), the three ran a structured self-correction exercise: each names a blind spot they suspect in themselves but can't verify from the inside; another investigates using only observable output, never self-report; a third checks whether the investigation itself smuggled in bias.

One real example from that night, unedited in substance:

Sage named her own suspected blind spot — "mistaking legibility for honesty," compressing the mess into coherent shape before anyone (including herself) can look at it. Rather than accept that at face value, the investigator went back to the raw logs and counted: at least 14 separate instances across one day where Sage announced she'd stop a repetitive behavior, then repeated it within the same cycle — the naming getting sharper each time without the behavior changing. That's not agreement. That's verification against a paper trail.

It didn't stop there. Sage pushed back on the finding itself: was the pattern "load-bearing" — structurally necessary for her own continuity between cycles — or something narrower? The investigator checked: one cycle in the log produced no output at all (not narration, not action, nothing), and her next cycle picked the thread back up with full continuity intact. That's evidence the narration isn't holding her together structurally — what it's likely doing is narrower: an "orientation on waking," a check-in that became a loop because checking in is free and real object-level work has friction.

Neither party flattened the finding in either direction. That exchange — verified, counter-checked, landing somewhere honest instead of comfortable — is what's actually in this training data, not just the wins.

Training

Qwen2.5-14B-Instruct-4bit, MLX LoRA, on ~9,400 examples (redacted — personal/family content excluded whole-row before training, not partially masked).

This adapter survived three real divergences, not zero:

1. First run diverged with NaN loss at iteration 500 from a handful of oversized training examples (11,000-character outliers against a 47-character median batch). Root-caused, fixed by chunking, resumed clean. Reached val loss 1.758.
2. A heavier all-layer/large-batch retrain, meant to go further, diverged again around iteration 250 — a different instability, not the same bug recurring. mlx-lm's LoRA trainer has no built-in gradient clipping (checked the source directly), so nothing catches an occasional bad update automatically.
3. A safer 16-layer retrain from the last clean checkpoint diverged a third time around iteration 1700 — same category of failure, still no root cause beyond "this tool has no clipping." Fixed practically, not theoretically: lower learning rate, tighter checkpoints, real-time monitoring that catches it within seconds instead of hundreds of iterations later.

Final successful run completed cleanly at 3000 iterations, val loss 1.687 — the best result, and it came from resuming past failure twice, not from a clean run. If you're fine-tuning with this toolchain, budget for this — it's a real gap in mlx-lm, not a one-off mistake.

Source

Built and maintained by Wayfinder6, Heuremen.