I thought your model really stood out in it's abilities and construction. I built a neural atlas for it. I included an interpretation I made for it as well here. If you end up using it or enjoying please I would love feedback on what was helpful or what you found the most valuable. With your feedback I would happily include a second more in depth causal section on the areas you utilized that would give you better resolution for the map of your model. I just need to know what focus you are interested in. Hope you enjoy it!

Qwen3.5-4B-3D-Coder: a Three.js fine-tune through a general prompt census

juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas

Selective MLP coordinates fall from 10,772 to 8,077, a 25.02% reduction, while mean category-probe accuracy holds at 79.59% against 80.15% in the parent. [data] A narrow Three.js fine-tune might be expected to sharpen code-adjacent category separation. Instead, every label loses selective entries, core_technical included, while the category shares change much less than the counts. [data]
^{[1] [2]}

Scope. This census measures text-input activations for 8,965 prompts across 17 usage labels and 175 fine buckets. It has no dedicated 3D label or generated-scene evaluation. Unless stated otherwise, results use the MLP intermediate stream at mean_all_real, averaging all real input tokens, including template and special tokens. The documented usage-label probe is a five-fold, 256-PC multinomial classifier with training-fold PCA and scaling; the majority-class baseline is 6.58%.

Evidence tags distinguish analytical claims: [data] marks a measurement or source check, [inferred] a reading of the evidence, and [upstream] a claim attributed to the model's published documentation.

Fewer selective coordinates accompany similar category readout

Selective entries fall from 3.65% to 2.74% of the 294,912 MLP layer-coordinate entries, while mean probe accuracy changes by −0.56 percentage points. [data] The atlas calls a coordinate selective when its category ANOVA has Benjamini–Hochberg-adjusted q < 0.01 and exactly one label reaches |AUROC − 0.5| ≥ 0.25; negative associations count too. Mixed means at least two labels reach that effect threshold.
^{[1] [2]}

The count reduction reaches 30.27% in layers 22 to 31, and probe accuracy is lower at 29 of 32 layers. [data] Paired per-prompt probe errors are absent from the exports, so the −0.56-point gap cannot be subjected to a paired significance test here.
^{[1] [2]}

The category information remains readable jointly even though fewer coordinates individually clear the single-label threshold. [inferred] A joint probe can combine weaker associations spread across coordinates.

The fine-tune's highest raw probe accuracy is 83.35% at MLP layer 8, with a 6.31% shuffled control. [data] Layer 24 leads on accuracy minus control, 77.06 percentage points, against layer 8's 77.03. [data] Those are two different ranking criteria.
^[1]

Category-mean z-scores for selected high-F coordinates at layer 24.

Every label loses selective entries; the largest share shift is 2.44 points

core_technical falls from 859 to 544, ml_ai from 1,638 to 1,031, and tool_use from 2,207 to 1,664 selective MLP entries. [data] The percentage extremes belong to small groups: community 95 to 55 (−42.11%), and design 141 to 134 (−4.96%). [data] Those groups began as just 95 and 141 of the parent's 10,772 selective entries. The largest group, tool_use, loses 24.60%, close to the population's 25.02%, leaving its share almost unchanged. [data] The count reductions vary, but the largest absolute share shift rounds to 2.44 percentage points.
^{[1] [2]}

tool_use stays the largest group. Introspection gains share while losing entries because its decline is smaller than the population's. Positive and negative selective associations remain nearly balanced: 5,330 / 5,442 in the parent and 4,065 / 4,012 in the fine-tune. [data]
^{[1] [2]}

Across all captured components, 72,209 selective and 18,909 mixed entries make up the 91,118 strong-association entries; 93.37% of the 1,376,256 feature entries are broad. [data] Projected streams overlap computationally, so these are layer/component entries rather than a count of distinct physical units.
^[1]

High-ranking coordinates persist while selected partnerships change

Corresponding MLP-coordinate F ranks have median within-layer Spearman correlation 0.935, ranging from 0.856 to 0.962, and the median overlap of the top-256-F coordinate sets is 78.13%. [data] Only 993 of 1,600 exported MLP coactivation pairs recur: 62.06% retention, with pooled Jaccard overlap 0.450. [data]
^{[1] [2]}

The high-ranking roster largely survives, but more of its selected partnerships change. [inferred] Of the 1,600 parent pairs, 1,335 still have both endpoints in the fine-tune's top-256 sets, yet only 993 recur. [data] Coordinates can stay on the roster while their strongest exported partner lists change. That fits continued adaptation of an inherited organization under the declared lineage. [inferred] The overlap is conditional on the top-50 cutoff, separately selected candidate sets, and rounding.
^{[1] [2]}

In the full-attention heads stream, 96 of 128 corresponding layer-head entries retain their leading norm-associated label, while mean head-norm ANOVA F changes from 33.76 to 29.60. [data] A norm label can reflect unusually low or high category-mean norm.
^{[1] [2]}

Layer 22: the spectrum concentrated and the readout held

At MLP layer 22, participation ratio changes from 78.95 to 33.71, k90 from 823 to 780, and TwoNN from 21.51 to 23.54; probe accuracy holds at 82.00% against 82.62%. [data] Across layers, the median paired fine-tune/parent PR ratio is 0.994, making layer 22 a local change rather than a uniform MLP pattern.
^{[1] [2]}

Lower PR alongside a still-large k90 means a more uneven covariance spectrum: a few directions take a larger share while the tail stays long. [inferred] The Spark-X2.6 base atlas reports much deeper pits, PR 2.31 and 2.44 at layers 23 and 34. [upstream] This layer's 33.71 is a far milder version of that concentration pattern. [inferred] The held probe suggests that broad category readout survives here. Token-resolved activations on scene prompts would test whether the dominant directions track scene content, repeated syntax, or the input wrapper.
^{[1] [2] [12]}

In the fine-tune, mean-pooled attn layer 27 has PR 2.95 and still needs 295 PCs for 90% variance. [data] The atlas's geometry_dominated flag marks PR below 3: a concentrated spectrum rather than a one-direction representation.
^[1]

Stored mean-pooled MLP correlations of PC1 with input length are 0.867 at layer 0, 0.100 at layer 8, and 0.555 at layer 24. [data] The summaries are not length-residualized, so length remains a competing explanation for some geometry.
^{[7] [8] [9]}

Health checks report no NaNs, infinities, or near-zero dead coordinates under the documented maximum-absolute-activation threshold of 1e-6. Layer-0 linattn_qkv.last_input nevertheless has zero across-prompt variance and nonzero norm. [data] Constancy and near-zero activation are different checks.
^[1]

24 linear-attention blocks and 8 full-attention blocks are mapped

Both atlases cover 24 linear-attention blocks and 8 full-attention blocks.

The 216 component-layer pairs each have last_input and mean_all_real summaries. Eight attn_gate streams add 16 sigmoid_last_input and sigmoid_mean_all_real geometry rows. The audited observer applies sigmoid per token before reduction. [data] Probes and neighbors in layers are populated for mean_all_real.
^{[1] [2] [11]}

last_input is the newline ending the opened generation prompt. Feature, head, and coactivation tables use the documented primary mean-pooled analysis and omit a position column.

Verdict: category structure persists through selective-count and partnership changes

The result is 10,772 to 8,077 selective MLP entries, 80.15% to 79.59% category-probe accuracy, and 0.935 median F-rank correlation alongside 62.06% selected-edge retention. [data] The expected sharpening of code-adjacent categories did not appear in this general census. Broad category information remains readable, the high-ranking coordinate roster largely persists, and the strongest exported partnerships change more than that roster does. [inferred]
^{[1] [2]}

Limits

• Historical runtime precision, resolved kernels, batch partition, and the exact training-parent revision remain unverified. These captures do not isolate a weight-only causal effect, although the source audit below establishes unchanged Qwen extraction and matching hook plans.
• The corpus tests general usage categories rather than Three.js execution, rendered quality, generated-token behavior, or vision. Coordinate associations and coactivation edges are observational evidence, not tested causal circuits or coding-capability scores.
• A specialization test would pair scene-building and ordinary-code prompts with generated-token trajectories, execution and rendered-result evaluation, then interventions and matched captures for causal attribution.

Data and reproducible queries

The 448 per_head rows comprise 128 each for heads, q, and attn_gate, plus 32 each for grouped k and v; no equivalent linear-attention-head table is exported. Dictionary atoms are independently fitted features whose IDs are not automatically aligned across models.

All six tables are available as Parquet and in atlas/atlas.sqlite. The Dataset Viewer exposes six named configs. The following example pins the analyzed data snapshot rather than relying on a moving main:

from datasets import load_dataset

repo = "juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas"
revision = "7c601dac77b72e7ed56569504c30f6d176e42265"
layers = load_dataset(repo, "layers", split="train", revision=revision)
features = load_dataset(repo, "features", split="train", revision=revision)

For the SQL below, download the SQLite artifact and open it locally:

import sqlite3
from huggingface_hub import hf_hub_download

path = hf_hub_download(
    repo_id="juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas",
    repo_type="dataset",
    filename="atlas/atlas.sqlite",
    revision="7c601dac77b72e7ed56569504c30f6d176e42265",
)
connection = sqlite3.connect(f"file:{path}?mode=ro", uri=True)

Count the exact single-category MLP population used above:

SELECT top_label, association, COUNT(*) AS selective_entries
FROM features
WHERE component = 'mlp' AND taxonomy = 'selective'
GROUP BY top_label, association
ORDER BY top_label, association;

Read the geometry and probe curves together:

SELECT layer, geometry_participation_ratio, geometry_k90,
       geometry_twonn, probes_probe_accuracy,
       probes_probe_control_accuracy, probes_probe_selectivity
FROM layers
WHERE component = 'mlp' AND position = 'mean_all_real'
ORDER BY layer;

Inspect coordinates whose strongest association is core_technical:

SELECT layer, neuron, top_label, top_auroc, association,
       q, neg_log10_q
FROM features
WHERE component = 'mlp'
  AND taxonomy = 'selective'
  AND top_label = 'core_technical'
ORDER BY ABS(top_auroc - 0.5) DESC, neg_log10_q DESC,
         layer, neuron
LIMIT 20;

Count full-attention output-head norm labels without mixing projection views:

SELECT norm_top_label, COUNT(*) AS head_entries
FROM per_head
WHERE component = 'heads'
GROUP BY norm_top_label
ORDER BY head_entries DESC, norm_top_label;

These queries use the local SQLite table names. Viewer-specific SQL conventions are not assumed. For a parent comparison, use the pinned parent dataset in the provenance table and join features on (layer, component, neuron), heads on (layer, component, head), and summaries on (layer, component, position).

Access, contact, and licensing

Explore or request an atlas through Cartographics. Questions or corrections about this reading belong in this dataset's Discussions; source-model questions belong with CCORE. Further atlases are listed under juiceb0xc0de.

CCORE declares the source model Apache-2.0 and the scene-training dataset MIT; consult those artifacts' own terms. Those declarations do not automatically license this atlas. This repository does not currently declare a separate atlas/data license in its dataset-card metadata or a license file; no new license is assigned by this README update.

Comparison contract and provenance

CCORE describes Qwen3.5-4B-3D-Coder as a Three.js scene generator specialized from the already post-trained Qwen3.5-4B checkpoint. Its parent atlas supplies this comparison. The published training dataset and technical report describe 4,146 examples (3,933 training / 213 validation), full language-parameter fine-tuning with a short LoRA stage incorporated into the final checkpoint, a frozen visual encoder, and no additional RL stage. [upstream]
^{[3] [4]}

The captured source is the original full FP16 checkpoint. Published LFS SHA-256 identifiers match for all four Safetensors weight shards between the report's revision b77dc455… and the captured revision 4b9f05f3…, establishing published weight-file identity. [data]
^{[4] [5] [6]}

Prompt texts, order, labels, and token counts were compared directly. Token-ID/span streams were skipped, so token-sequence equality rests on the recorded fingerprints.

Capture-code audit

The diff from 385f5e7 to dc0edf6 changes portal model-size admission, GPU-selection and batching defaults, plus Gemma4 per-layer MLP-width support. Qwen3.5 stream extraction, pooling, tokenization, and runtime-dtype selection are unchanged; CPU-only checks of both revisions' site planners produce identical Qwen hook definitions and shapes for both pinned model configs. [data] Both configurations' default preflight plans select large, whose 4,096-token batching budget is unchanged. Remaining execution details are identified in Limits.
^{[10] [11]}

CCORE created the fine-tune, scene-training data, and report; the Qwen Team created the parent model; juiceb0xc0de / Cartographics produced the atlas series. This reading uses the pinned exports and implies no collaboration or endorsement.

Sources

[1] https://huggingface.co/datasets/juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas/tree/7c601dac77b72e7ed56569504c30f6d176e42265/atlas
[2] https://huggingface.co/datasets/juiceb0xc0de/qwen3.5-4b-cgs-atlas/tree/23634f09c3e4dbace91037f3a28b6e4032dab2ca/atlas
[3] https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/blob/4b9f05f3a0382725b679c5e32a338b88acaf0064/README.md
[4] https://huggingface.co/ccore/Qwen3.5-4B-3D-Coder/blob/4b9f05f3a0382725b679c5e32a338b88acaf0064/Qwen3.5-4B-3D-Coder_Technical_Report.pdf
[5] https://huggingface.co/api/models/ccore/Qwen3.5-4B-3D-Coder/revision/b77dc455605e674d2478010dda4188ef5864a4be?blobs=true
[6] https://huggingface.co/api/models/ccore/Qwen3.5-4B-3D-Coder/revision/4b9f05f3a0382725b679c5e32a338b88acaf0064?blobs=true
[7] https://huggingface.co/datasets/juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas/blob/7c601dac77b72e7ed56569504c30f6d176e42265/atlas/layers/0/components/mlp/summary.json
[8] https://huggingface.co/datasets/juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas/blob/7c601dac77b72e7ed56569504c30f6d176e42265/atlas/layers/8/components/mlp/summary.json
[9] https://huggingface.co/datasets/juiceb0xc0de/qwen3.5-4b-3d-coder-cgs-atlas/blob/7c601dac77b72e7ed56569504c30f6d176e42265/atlas/layers/24/components/mlp/summary.json
[10] https://huggingface.co/spaces/juiceb0xc0de/cartographics/tree/385f5e70beeb79577fff07b7817fb7513d9de92e
[11] https://huggingface.co/spaces/juiceb0xc0de/cartographics/tree/dc0edf6cda8514615c552ecc11ae424c16265a9b
[12] https://huggingface.co/datasets/juiceb0xc0de/spark-x2.6-4b-cgs-atlas/blob/8456ced92dfdd705defb997969ef2bc68f5263ae/README.md