supreethrao/instructNER_fewnerd_xl

SpanMarker

This is a SpanMarker model trained on the DFKI-SLT/few-nerd dataset that can be used for Named Entity Recognition.

Model Details

Model Description

• Model Type: SpanMarker
• Maximum Sequence Length: 256 tokens
• Maximum Entity Length: 8 words
• Training Dataset: DFKI-SLT/few-nerd

Model Sources

• Repository: SpanMarker on GitHub
• Thesis: SpanMarker For Named Entity Recognition

Model Labels

Label	Examples
art-broadcastprogram	"Street Cents", "Corazones", "The Gale Storm Show : Oh , Susanna"
art-film	"L'Atlantide", "Shawshank Redemption", "Bosch"
art-music	"Champion Lover", "Atkinson , Danko and Ford ( with Brockie and Hilton )", "Hollywood Studio Symphony"
art-other	"Aphrodite of Milos", "The Today Show", "Venus de Milo"
art-painting	"Production/Reproduction", "Cofiwch Dryweryn", "Touit"
art-writtenart	"Time", "Imelda de ' Lambertazzi", "The Seven Year Itch"
building-airport	"Sheremetyevo International Airport", "Luton Airport", "Newark Liberty International Airport"
building-hospital	"Yeungnam University Hospital", "Memorial Sloan-Kettering Cancer Center", "Hokkaido University Hospital"
building-hotel	"Radisson Blu Sea Plaza Hotel", "Flamingo Hotel", "The Standard Hotel"
building-library	"British Library", "Berlin State Library", "Bayerische Staatsbibliothek"
building-other	"Communiplex", "Henry Ford Museum", "Alpha Recording Studios"
building-restaurant	"Carnegie Deli", "Trumbull", "Fatburger"
building-sportsfacility	"Sports Center", "Boston Garden", "Glenn Warner Soccer Facility"
building-theater	"Sanders Theatre", "Pittsburgh Civic Light Opera", "National Paris Opera"
event-attack/battle/war/militaryconflict	"Vietnam War", "Jurist", "Easter Offensive"
event-disaster	"1990s North Korean famine", "the 1912 North Mount Lyell Disaster", "1693 Sicily earthquake"
event-election	"1982 Mitcham and Morden by-election", "Elections to the European Parliament", "March 1898 elections"
event-other	"Eastwood Scoring Stage", "Union for a Popular Movement", "Masaryk Democratic Movement"
event-protest	"French Revolution", "Iranian Constitutional Revolution", "Russian Revolution"
event-sportsevent	"World Cup", "National Champions", "Stanley Cup"
location-GPE	"Mediterranean Basin", "the Republic of Croatia", "Croatian"
location-bodiesofwater	"Arthur Kill", "Atatürk Dam Lake", "Norfolk coast"
location-island	"Staten Island", "new Samsat district", "Laccadives"
location-mountain	"Miteirya Ridge", "Ruweisat Ridge", "Salamander Glacier"
location-other	"Northern City Line", "Victoria line", "Cartuther"
location-park	"Painted Desert Community Complex Historic District", "Gramercy Park", "Shenandoah National Park"
location-road/railway/highway/transit	"NJT", "Newark-Elizabeth Rail Link", "Friern Barnet Road"
organization-company	"Church 's Chicken", "Texas Chicken", "Dixy Chicken"
organization-education	"Barnard College", "MIT", "Belfast Royal Academy and the Ulster College of Physical Education"
organization-government/governmentagency	"Diet", "Supreme Court", "Congregazione dei Nobili"
organization-media/newspaper	"Al Jazeera", "Clash", "TimeOut Melbourne"
organization-other	"Defence Sector C", "4th Army", "IAEA"
organization-politicalparty	"Al Wafa ' Islamic", "Shimpotō", "Kenseitō"
organization-religion	"Jewish", "UPCUSA", "Christian"
organization-showorganization	"Mr. Mister", "Lizzy", "Bochumer Symphoniker"
organization-sportsleague	"NHL", "First Division", "China League One"
organization-sportsteam	"Arsenal", "Luc Alphand Aventures", "Tottenham"
other-astronomything	"Algol", "Zodiac", "`` Caput Larvae ''"
other-award	"Order of the Republic of Guinea and Nigeria", "GCON", "Grand Commander of the Order of the Niger"
other-biologything	"Amphiphysin", "BAR", "N-terminal lipid"
other-chemicalthing	"sulfur", "uranium", "carbon dioxide"
other-currency	"$", "Travancore Rupee", "lac crore"
other-disease	"hypothyroidism", "bladder cancer", "French Dysentery Epidemic of 1779"
other-educationaldegree	"BSc ( Hons ) in physics", "Master", "Bachelor"
other-god	"El", "Raijin", "Fujin"
other-language	"Latin", "English", "Breton-speaking"
other-law	"United States Freedom Support Act", "Thirty Years ' Peace", "Leahy–Smith America Invents Act ( AIA"
other-livingthing	"insects", "monkeys", "patchouli"
other-medical	"pediatrician", "Pediatrics", "amitriptyline"
person-actor	"Edmund Payne", "Tchéky Karyo", "Ellaline Terriss"
person-artist/author	"Gaetano Donizett", "George Axelrod", "Hicks"
person-athlete	"Tozawa", "Jaguar", "Neville"
person-director	"Bob Swaim", "Frank Darabont", "Richard Quine"
person-other	"Holden", "Richard Benson", "Campbell"
person-politician	"Rivière", "Emeric", "William"
person-scholar	"Stalmine", "Wurdack", "Stedman"
person-soldier	"Krukenberg", "Joachim Ziegler", "Helmuth Weidling"
product-airplane	"EC135T2 CPDS", "Spey-equipped FGR.2s", "Luton"
product-car	"100EX", "Corvettes - GT1 C6R", "Phantom"
product-food	"yakiniku", "V. labrusca", "red grape"
product-game	"Airforce Delta", "Splinter Cell", "Hardcore RPG"
product-other	"X11", "Fairbottom Bobs", "PDP-1"
product-ship	"Essex", "HMS `` Chinkara ''", "Congress"
product-software	"Wikipedia", "Apdf", "AmiPDF"
product-train	"High Speed Trains", "Royal Scots Grey", "55022"
product-weapon	"ZU-23-2M Wróbel", "AR-15 's", "ZU-23-2MR Wróbel II"

Evaluation

Metrics

Label	Precision	Recall	F1
all	0.7034	0.7027	0.7031
art-broadcastprogram	0.6024	0.5904	0.5963
art-film	0.7761	0.7533	0.7645
art-music	0.7825	0.7551	0.7685
art-other	0.4193	0.3327	0.3710
art-painting	0.5882	0.5263	0.5556
art-writtenart	0.6819	0.6488	0.6649
building-airport	0.8064	0.8352	0.8205
building-hospital	0.7282	0.8022	0.7634
building-hotel	0.7033	0.7245	0.7138
building-library	0.7550	0.7380	0.7464
building-other	0.5867	0.5840	0.5853
building-restaurant	0.6205	0.5216	0.5667
building-sportsfacility	0.6113	0.7976	0.6921
building-theater	0.7060	0.7495	0.7271
event-attack/battle/war/militaryconflict	0.7945	0.7395	0.7660
event-disaster	0.5604	0.5604	0.5604
event-election	0.4286	0.1484	0.2204
event-other	0.4885	0.4400	0.4629
event-protest	0.3798	0.4759	0.4225
event-sportsevent	0.6198	0.6162	0.6180
location-GPE	0.8157	0.8552	0.8350
location-bodiesofwater	0.7268	0.7690	0.7473
location-island	0.7504	0.6842	0.7158
location-mountain	0.7352	0.7298	0.7325
location-other	0.4427	0.3104	0.3649
location-park	0.7153	0.6856	0.7001
location-road/railway/highway/transit	0.7090	0.7324	0.7205
organization-company	0.6963	0.7061	0.7012
organization-education	0.7994	0.7986	0.7990
organization-government/governmentagency	0.5524	0.4533	0.4980
organization-media/newspaper	0.6513	0.6656	0.6584
organization-other	0.5978	0.5375	0.5661
organization-politicalparty	0.6793	0.7315	0.7044
organization-religion	0.5575	0.6131	0.5840
organization-showorganization	0.6035	0.5839	0.5935
organization-sportsleague	0.6393	0.6610	0.6499
organization-sportsteam	0.7259	0.7796	0.7518
other-astronomything	0.7794	0.8024	0.7907
other-award	0.7180	0.6649	0.6904
other-biologything	0.6864	0.6238	0.6536
other-chemicalthing	0.5688	0.6036	0.5856
other-currency	0.6996	0.8423	0.7643
other-disease	0.6591	0.7410	0.6977
other-educationaldegree	0.6114	0.6198	0.6156
other-god	0.6486	0.7181	0.6816
other-language	0.6507	0.8313	0.7300
other-law	0.6934	0.7331	0.7127
other-livingthing	0.6019	0.6605	0.6298
other-medical	0.5124	0.5214	0.5169
person-actor	0.8384	0.8051	0.8214
person-artist/author	0.7122	0.7531	0.7321
person-athlete	0.8318	0.8422	0.8370
person-director	0.7083	0.7365	0.7221
person-other	0.6833	0.6737	0.6785
person-politician	0.6807	0.6836	0.6822
person-scholar	0.5397	0.5209	0.5301
person-soldier	0.5053	0.5920	0.5452
product-airplane	0.6617	0.6692	0.6654
product-car	0.7313	0.7132	0.7222
product-food	0.5787	0.5787	0.5787
product-game	0.7364	0.7140	0.7250
product-other	0.5567	0.4210	0.4795
product-ship	0.6842	0.6842	0.6842
product-software	0.6495	0.6648	0.6570
product-train	0.5942	0.5924	0.5933
product-weapon	0.6435	0.5353	0.5844

Uses

Direct Use for Inference

from span_marker import SpanMarkerModel

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("supreethrao/instructNER_fewnerd_xl")
# Run inference
entities = model.predict("The Sunday Edition is a television programme broadcast on the ITV Network in the United Kingdom focusing on political interview and discussion, produced by ITV Productions.")

Downstream Use

You can finetune this model on your own dataset.

Click to expand

from span_marker import SpanMarkerModel, Trainer

# Download from the 🤗 Hub
model = SpanMarkerModel.from_pretrained("supreethrao/instructNER_fewnerd_xl")

# Specify a Dataset with "tokens" and "ner_tag" columns
dataset = load_dataset("conll2003") # For example CoNLL2003

# Initialize a Trainer using the pretrained model & dataset
trainer = Trainer(
    model=model,
    train_dataset=dataset["train"],
    eval_dataset=dataset["validation"],
)
trainer.train()
trainer.save_model("supreethrao/instructNER_fewnerd_xl-finetuned")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Sentence length	1	24.4945	267
Entities per sentence	0	2.5832	88

Training Hyperparameters

• learning_rate: 5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3
• mixed_precision_training: Native AMP

Framework Versions

• Python: 3.10.13
• SpanMarker: 1.5.0
• Transformers: 4.35.2
• PyTorch: 2.1.1
• Datasets: 2.15.0
• Tokenizers: 0.15.0

Citation

BibTeX

@software{Aarsen_SpanMarker,
    author = {Aarsen, Tom},
    license = {Apache-2.0},
    title = {{SpanMarker for Named Entity Recognition}},
    url = {https://github.com/tomaarsen/SpanMarkerNER}
}