Leaderboard

Official SemEval Results on the Test Set

English - Subtask 1

1	DUTIR	0.41280	0.48100	0.43020	0.45420	0.94890
2	PATeam	0.38300	0.46150	0.43020	0.44530	0.88940
3	DEMON	0.37450	0.44870	0.39620	0.42080	0.91910
4	gowithnlp	0.37020	0.44260	0.39250	0.41600	0.93620
5	TartanTritons	0.35740	0.11440	0.10190	0.10780	0.72340
6	Fane	0.34470	0.40850	0.36230	0.38400	0.89360
7	QUST	0.32770	0.39040	0.36980	0.37980	0.92340
8	LATeIIMAS	0.31060	0.36710	0.32830	0.34660	0.83830
8	NlpUned	0.31060	0.37870	0.33580	0.35600	0.84680
9	m1nadzuki	0.25960	0.31490	0.27920	0.29600	0.87660
10	LTG	0.25530	0.07490	0.07550	0.07520	0.58300
11	BERTastic	0.25110	0.31490	0.27920	0.29600	0.86380
11	adithjrajeev	0.25110	0.32140	0.30570	0.31330	0.86810
12	Mekky	0.21700	0.29480	0.27920	0.28680	0.80000
13	NarrativeMiners	0.21280	0.26810	0.23770	0.25200	0.68940
14	FromProblemImportSolve	0.20430	0.29030	0.30570	0.29780	0.77450
15	YNUzwt	0.20000	0.26380	0.23400	0.24800	0.52340
16	Cimba	0.18720	0.23950	0.21510	0.22660	0.09790
17	NarrativeNexus	0.18300	0.20850	0.18490	0.19600	0.71910
18	Rosetta	0.17870	0.22920	0.21890	0.22390	0.60430
19	Dhananjaya	0.17450	0.22880	0.20380	0.21560	0.81280
20	UMZNLP	0.14040	0.21430	0.23770	0.22540	0.70640
21	Tuebingen	0.13190	0.20580	0.21510	0.21030	0.74040
22	YNUHPCC	0.08940	0.11490	0.10190	0.10800	0.75320
23	north	0.08510	0.25870	0.42260	0.32090	0.94890
24	HowardUniversityAI4PC	0.08090	0.08320	0.17740	0.11330	0.69360
25	kzeky	0.06810	0.08090	0.07170	0.07600	0.56600
26	bumblebeeTransformer	0.06380	0.13640	0.16980	0.15130	0.78300
26	eevvgg	0.06380	0.10550	0.17360	0.13120	0.78720
27	Baseline	0.03830	0.04680	0.04150	0.04400	0.28510
28	Team12	0.02130	0.07670	0.09810	0.08610	0.72340
29	cocoa	0.01700	0.08750	0.12450	0.10280	0.82550
30	SemanticaInnovators	0.01280	0.04680	0.16600	0.07300	0.25110
31	TechDacians	0.00000	1.00000	0.00000	0.00000	0.09790
32	teamninebonn	0.00000	0.03260	0.08680	0.04740	0.17870

English - Subtask 2

1	GATENLP	0.59000	0.35300	0.43800	0.33300
2	COGNAC	0.55400	0.40000	0.42600	0.39100
3	INSALyon2	0.51300	0.37800	0.40600	0.38200
4	23	0.49300	0.39200	0.37700	0.38400
5	NCLteam	0.48600	0.36300	0.34500	0.36000
6	Narrlangen	0.44400	0.41100	0.34400	0.38600
7	PATeam	0.52100	0.35600	0.33900	0.29100
8	YNUzwt	0.48500	0.37800	0.32100	0.33200
9	iLostTheCode	0.46700	0.35600	0.32000	0.32100
10	Narrengers	0.45700	0.36300	0.31800	0.31800
11	UNEDTeam	0.51200	0.36400	0.31300	0.29400
12	CtrlAltElite	0.47400	0.42200	0.31100	0.44600
13	NotMyNarrative	0.40300	0.41400	0.29800	0.39300
14	IRNLP	0.51600	0.40200	0.28700	0.45200
15	NLPPraktikumWS2025	0.44800	0.38500	0.25800	0.33700
16	KostasThesis2025	0.40900	0.31400	0.23900	0.24300
17	NarrativeMiners	0.36900	0.42400	0.23800	0.42600
18	nlptuducd	0.33800	0.36400	0.22600	0.31700
19	ammd7	0.29400	0.35500	0.22200	0.34300
20	UniBonn187	0.40900	0.33000	0.20600	0.22400
21	Irapuarani	0.33500	0.20100	0.18800	0.14100
22	DUTtask10	0.31000	0.40500	0.16500	0.36800
23	LATeIIMAS	0.22200	0.33400	0.16300	0.32300
24	GeorgeSnape	0.19300	0.34800	0.15600	0.34800
25	GrammarPolice	0.16800	0.25800	0.06300	0.12400
26	Baseline	0.03000	0.12700	0.01300	0.07000
27	NarrativeNexus	0.00000	0.00000	0.00000	0.00000
27	bbStar	0.00000	0.00000	0.00000	0.00000

English - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	KyuHyunChoi	0.76686	0.73517	0.75040
2	WordWiz	0.75464	0.73705	0.74551
3	GPLSICORTEX	0.75375	0.73274	0.74280
4	TechSSN	0.73886	0.74568	0.74203
5	NarrativeNexus	0.71991	0.74267	0.73085
6	NarrativeMiners	0.71139	0.74827	0.72910
7	clujteam	0.72350	0.72647	0.72464
8	PATeam	0.72371	0.72589	0.72433
9	TartanTritons	0.72396	0.70200	0.71245
10	YNUzwt	0.69707	0.70112	0.69885
11	LATeIIMAS	0.70540	0.68674	0.69558
12	bbStar	0.66436	0.72023	0.69103
13	Synapse	0.65986	0.69175	0.67527
14	Baseline	0.65144	0.68344	0.66690
15	DUTtask10	0.00000	0.00000	0.00000
15	Mendel292A	0.00000	0.00000	0.00000
15	UMZNLP	0.00000	0.00000	0.00000
15	ftd	0.00000	0.00000	0.00000

Portuguese - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	DUTIR	0.59260	0.64950	0.62540	0.63720	0.94610
2	PATeam	0.49160	0.55370	0.52630	0.53970	0.90570
3	QUST	0.45790	0.50990	0.47680	0.49280	0.84510
4	BERTastic	0.41750	0.47470	0.43650	0.45480	0.84510
5	LTG	0.40740	0.11420	0.11460	0.11440	0.42760
6	DEMON	0.36700	0.43240	0.39630	0.41360	0.80810
7	LATeIIMAS	0.33670	0.38720	0.35600	0.37100	0.71720
8	TartanTritons	0.33330	0.18180	0.16720	0.17420	0.50840
9	gowithnlp	0.26940	0.31650	0.29100	0.30320	0.71380
10	Cimba	0.26260	0.33000	0.30960	0.31950	0.20540
10	FromProblemImportSolve	0.26260	0.32970	0.37150	0.34930	0.65660
11	Fane	0.25590	0.30980	0.28480	0.29680	0.62960
12	Dhananjaya	0.21890	0.26320	0.24770	0.25520	0.76770
13	YNUzwt	0.16160	0.20540	0.18890	0.19680	0.39730
14	HowardUniversityAI4PC	0.13130	0.11320	0.22600	0.15080	0.51850
15	Baseline	0.04710	0.05050	0.04640	0.04840	0.36030
16	teamninebonn	0.00000	0.04040	0.11150	0.05930	0.44110

Portuguese - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	GATENLP	0.66400	0.26000	0.48000	0.25400
2	PATeam	0.54100	0.29000	0.40900	0.26900
3	23	0.48700	0.33800	0.31300	0.25700
4	KostasThesis2025	0.47800	0.20100	0.30900	0.15300
5	iLostTheCode	0.53600	0.27300	0.29300	0.20600
6	Narrlangen	0.48000	0.33900	0.29100	0.25900
7	UNEDTeam	0.53700	0.32400	0.27000	0.26200
8	YNUzwt	0.56800	0.30100	0.26600	0.27700
9	Irapuarani	0.43500	0.19500	0.22500	0.12900
10	CtrlAltElite	0.49200	0.31600	0.14900	0.21900
11	NotMyNarrative	0.22800	0.25000	0.12400	0.11100
12	DUTtask10	0.10700	0.24100	0.02600	0.13400
13	Baseline	0.03700	0.14000	0.01400	0.07000

Portuguese - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	WordWiz	0.75931	0.73885	0.74857
2	PATeam	0.75365	0.73984	0.74637
3	bbStar	0.70991	0.72924	0.71926
4	YNUzwt	0.68986	0.68603	0.68771
5	TartanTritons	0.69764	0.67419	0.68540
6	Baseline	0.67822	0.68297	0.68046
7	LATeIIMAS	0.68395	0.66339	0.67297
8	DUTtask10	0.00000	0.00000	0.00000

Russian - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	DUTIR	0.56540	0.61750	0.59030	0.60360	0.85510
2	QUST	0.51400	0.56280	0.53300	0.54750	0.83180
3	TartanTritons	0.47200	0.16200	0.15420	0.15800	0.43930
4	BERTastic	0.46730	0.50470	0.47580	0.48980	0.78970
4	DEMON	0.46730	0.51420	0.48020	0.49660	0.81310
5	gowithnlp	0.44860	0.48130	0.45370	0.46710	0.86450
6	PATeam	0.44390	0.49780	0.48900	0.49330	0.78040
7	LTG	0.42990	0.08620	0.08810	0.08710	0.35980
8	Cimba	0.38320	0.41670	0.39650	0.40630	0.33640
9	FromProblemImportSolve	0.35510	0.43700	0.51980	0.47480	0.75230
10	LATeIIMAS	0.31310	0.36040	0.35240	0.35630	0.64490
11	Dhananjaya	0.29440	0.32560	0.30840	0.31670	0.66360
12	YNUzwt	0.26640	0.29910	0.28190	0.29020	0.48600
13	Fane	0.24300	0.27570	0.25990	0.26760	0.60750
14	HowardUniversityAI4PC	0.12620	0.07590	0.12780	0.09520	0.42520
15	Baseline	0.05140	0.06070	0.05730	0.05900	0.34110

Russian - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	GATENLP	0.70900	0.27400	0.51800	0.28200
2	PATeam	0.56600	0.26800	0.43400	0.24700
3	iLostTheCode	0.61800	0.31200	0.41100	0.30800
4	Narrlangen	0.57400	0.37000	0.40500	0.32400
5	YNUzwt	0.50600	0.35100	0.33500	0.30200
6	KostasThesis2025	0.59600	0.25700	0.33300	0.23400
7	UNEDTeam	0.51300	0.32500	0.33000	0.27000
8	UniBonn187	0.55000	0.30300	0.23100	0.17800
9	Irapuarani	0.35900	0.15000	0.19100	0.09200
10	IRNLP	0.53700	0.35100	0.11600	0.25200
11	GrammarPolice	0.00000	0.00000	0.05000	0.21800
12	DUTtask10	0.04000	0.18500	0.03300	0.18000
13	Baseline	0.06500	0.21300	0.00800	0.06400
14	LATeIIMAS	0.00000	0.00000	0.00000	0.00000

Russian - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	PATeam	0.69984	0.71423	0.70639
2	WordWiz	0.70193	0.70730	0.70395
3	TartanTritons	0.68271	0.68155	0.68159
4	YNUzwt	0.66621	0.68632	0.67578
5	bbStar	0.63433	0.69813	0.66440
6	DUTtask10	0.65979	0.66845	0.66379
7	Baseline	0.62206	0.66872	0.64427
8	LATeIIMAS	0.61043	0.67674	0.64161

Bulgarian - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	PATeam	0.51610	0.53970	0.53120	0.53540	0.92740
2	DUTIR	0.50810	0.53730	0.56250	0.54960	0.94350
3	DEMON	0.45970	0.47970	0.46090	0.47010	0.88710
4	gowithnlp	0.43550	0.45970	0.44530	0.45240	0.83060
5	TartanTritons	0.41130	0.09680	0.09380	0.09520	0.55650
6	QUST	0.38710	0.39200	0.38280	0.38740	0.84680
7	BERTastic	0.35480	0.37100	0.35940	0.36510	0.87100
8	Fane	0.34680	0.36290	0.35160	0.35710	0.77420
9	LTG	0.31450	0.04930	0.05470	0.05190	0.55650
10	Cimba	0.25810	0.26980	0.26560	0.26770	0.25000
11	FromProblemImportSolve	0.20970	0.28660	0.36720	0.32190	0.80650
12	Dhananjaya	0.19350	0.21090	0.21090	0.21090	0.69350
13	HowardUniversityAI4PC	0.09680	0.07920	0.14840	0.10330	0.51610
14	Baseline	0.04030	0.04030	0.03910	0.03970	0.25810
15	TechDacians	0.00000	1.00000	0.00000	0.00000	0.25000

Bulgarian - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	PATeam	0.63100	0.33800	0.46000	0.33300
2	GATENLP	0.62900	0.33200	0.41600	0.28600
3	iLostTheCode	0.55700	0.33500	0.36900	0.30800
4	UNEDTeam	0.57400	0.35300	0.36300	0.31200
5	Narrlangen	0.49500	0.40100	0.35500	0.37600
6	KostasThesis2025	0.51000	0.32200	0.33300	0.30000
7	Irapuarani	0.36600	0.17700	0.18300	0.12000
8	NotMyNarrative	0.24900	0.30600	0.14200	0.22800
9	DUTtask10	0.17200	0.34100	0.12100	0.31200
10	LATeIIMAS	0.17700	0.26600	0.07200	0.16900
11	Baseline	0.05600	0.19100	0.02200	0.11900

Bulgarian - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	PATeam	0.71405	0.69478	0.70396
2	WordWiz	0.69034	0.67816	0.68390
3	bbStar	0.65751	0.68740	0.67198
4	TartanTritons	0.66758	0.64399	0.65531
5	Baseline	0.62739	0.64180	0.63430
6	LATeIIMAS	0.62947	0.61907	0.62406
7	DUTtask10	0.00000	0.00000	0.00000

Hindi - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	QUST	0.46840	0.59060	0.49480	0.53850	0.76900
2	TartanTritons	0.44620	0.18940	0.15970	0.17330	0.32910
3	BERTastic	0.43990	0.56560	0.47380	0.51570	0.78480
4	DEMON	0.40190	0.52530	0.43460	0.47560	0.75630
5	LTG	0.36390	0.16580	0.15970	0.16270	0.37660
6	Cimba	0.35440	0.45310	0.37960	0.41310	0.47150
7	gowithnlp	0.33540	0.43990	0.36390	0.39830	0.71520
8	DUTIR	0.29430	0.37660	0.31150	0.34100	0.59810
9	Dhananjaya	0.27850	0.37660	0.31150	0.34100	0.70890
10	LATeIIMAS	0.27220	0.37110	0.40310	0.38640	0.63610
11	PATeam	0.26900	0.35330	0.29320	0.32050	0.69620
12	FromProblemImportSolve	0.25630	0.37410	0.39270	0.38310	0.66770
13	Fane	0.23420	0.31650	0.26180	0.28650	0.65510
14	HowardUniversityAI4PC	0.16770	0.11460	0.15180	0.13060	0.35440
15	Baseline	0.05700	0.07910	0.06540	0.07160	0.32280

Hindi - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	DUTtask10	0.56900	0.48400	0.53500	0.49400
2	IRNLP	0.37500	0.46700	0.51500	0.50000
3	Narrlangen	0.39500	0.46400	0.38500	0.46500
4	UNEDTeam	0.44900	0.46000	0.37600	0.45600
5	GATENLP	0.40900	0.41000	0.32100	0.37900
6	KostasThesis2025	0.38400	0.41800	0.28200	0.40200
7	NotMyNarrative	0.31800	0.44900	0.24300	0.40800
8	PATeam	0.39200	0.39000	0.21800	0.35000
9	iLostTheCode	0.20700	0.31300	0.14700	0.29200
10	Irapuarani	0.23400	0.12500	0.11100	0.08200
11	LATeIIMAS	0.09700	0.18500	0.02900	0.10200
12	Baseline	0.08100	0.26000	0.00000	0.00000
13	bbStar	0.00000	0.00000	0.00000	0.00000

Hindi - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	PATeam	0.75097	0.76045	0.75540
2	WordWiz	0.72859	0.73914	0.73356
3	bbStar	0.71789	0.73704	0.72712
4	TartanTritons	0.69679	0.70119	0.69873
5	Baseline	0.66802	0.67188	0.66974
6	DUTtask10	0.00000	0.00000	0.00000

Contact

twitter.com/_proppy_
Register to the google group of the task to ask questions and interact with other participants

SemEval 2025 shared task on "Multilingual Characterization and Extraction of Narratives from Online News"

Leaderboard

Official SemEval Results on the Test Set

English - Subtask 1

English - Subtask 2

English - Subtask 3

Portuguese - Subtask 1

Portuguese - Subtask 2

Portuguese - Subtask 3

Russian - Subtask 1

Russian - Subtask 2

Russian - Subtask 3

Bulgarian - Subtask 1

Bulgarian - Subtask 2

Bulgarian - Subtask 3

Hindi - Subtask 1

Hindi - Subtask 2

Hindi - Subtask 3

Contact