fastino

/

GLiNER2.5-Decide

Like 429

Follow

fastino 552

Hey, I am also looking to train model for different domain.

@haukelicht @vijaymaurya The training rows use the same keys as the classify_text schemas on this card, so each card example becomes one JSONL row. I checked this against gliner2 2.0.0 (the current PyPI release) and ran a short CPU fine-tune of this checkpoint.

The card dict goes through _classification_schema into Schema.classification; training rows load through Classification. More row examples: tutorial/8-train_data.md. Two of the six rows I trained on:

{"input": "Battery dies before lunch, but the keyboard and the screen are great.", "output": {"classifications": [{"task": "aspects", "labels": ["battery", "keyboard", "screen", "camera", "price"], "true_label": ["battery", "keyboard", "screen"], "multi_label": true}]}}
{"input": "I finished it in two nights. The ending is earned, the middle drags, and I would still recommend it.", "output": {"classifications": [{"task": "rating", "labels": ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10"], "true_label": ["7"]}]}}

# pip install "gliner2[train]==2.0.0"  (plain `pip install gliner2` has no torch)
from gliner2 import AutoExtractor
from gliner2.training.trainer import ExtractorTrainer, TrainingConfig
model = AutoExtractor.from_pretrained("fastino/GLiNER2.5-Decide")
cfg = TrainingConfig(output_dir="out", max_steps=6, batch_size=2, num_workers=0, eval_strategy="no", save_best=False)
ExtractorTrainer(model, cfg).train(train_data="train.jsonl")
tuned = AutoExtractor.from_pretrained("out/final")

All six rows passed validate(); six training steps took about a minute on an 8-core desktop CPU, and out/final reloaded and answered card-style schemas. That checks format and pipeline, not accuracy.

Two things to know first:

1. No ordinal loss. Each candidate label is its own binary target (model.py#L385); multi_label only changes decoding (runtime.py#L404). The loss treats these labels as categories and includes no penalty based on numeric distance. Check an ordinal head with mean absolute error alongside exact match.
2. Default label augmentation can mark the right answer wrong. In training, SamplingConfig renames labels to label 1, label 2… in half the views and, in half of those, re-adds the true label under its real name. The target was renamed, so the real name is trained as a negative. That is about 1 in 4 training views, for 2-, 4- and 11-label rows alike, on 2.0.0 and current main (55656fb). Mapping the re-added label through real2syn fixes it; the same 295 processor, classification and training tests pass before and after (one training test file fails to import either way and was excluded):

                         for tl in true_label:
+                            tl = real2syn.get(tl, tl)
                             if tl not in cls_labels:

The string true_label branch below it needs the same mapping. Check:

import random
from gliner2.processor import SchemaTransformer
p = SchemaTransformer("fastino/GLiNER2.5-Decide"); p.is_training = True; random.seed(0)
for labels in (["yes", "no"], ["a", "b", "c", "d"], [str(i) for i in range(11)]):
    bad = 0
    for _ in range(1000):
        s = {"classifications": [{"task": "t", "labels": labels, "true_label": [labels[-1]]}]}
        p._process_classifications(s, [], [], [], p.sampling_config)
        c = s["classifications"][0]; bad += labels[-1] in c["labels"] and labels[-1] not in c["true_label"]
    print(len(labels), "labels:", bad, "/ 1000")

On 2.0.0 and main it prints 258, 257 and 255 per 1,000; with the change, 0. I did not measure how much this costs in accuracy.

@haukelicht @vijaymaurya thanks for asking, and @CyberNativeAI thanks for the careful write-up. The row format above is right, and so is the bug report.

Format. Each schema on the card maps to one object in output.classifications, with the same task, labels, prompt and multi_label you pass to classify_text, plus true_label. Descriptions go in label_descriptions. An ordinal scale is a normal single-label row. #12 adds a Fine-tuning section to this card, with one example row for each schema type on the card. All of them pass TrainingDataset.validate().

The augmentation bug is real, and it has been in every gliner2 release with label augmentation. With the default SamplingConfig, about 1 in 4 classification training views re-added the correct label under its real name after the candidates had been renamed to label N, so the right answer was trained as wrong. A string true_label was also split into characters on the same path. The fix, with a regression test, is in https://github.com/fastino-ai/GLiNER2/pull/187.

To measure the cost, we fine-tuned this checkpoint on SST-5: 5-class sentiment, the full 8,544-row train set, one epoch, scored on the 2,210-row test set. We ran three seeds each for the current code and the fix:

The fix won on every seed, by 0.7 to 0.9 points of accuracy. It is a modest gain, but it costs nothing.

Until a release includes the fix, you can avoid the bug with SamplingConfig(synthetic_label_prob=0.0). That turns off label renaming, which is the half of the augmentation that triggers it:

from gliner2.processor import SamplingConfig

model.processor.sampling_config = SamplingConfig(synthetic_label_prob=0.0)

Ordinal labels. As noted above, the loss scores each label on its own and does not know that "6" is closer to "7" than "0" is. Report mean absolute error alongside accuracy for an ordinal head.