Leaderboard

Official SemEval 2025 task 10 Test Leaderboards

Post SemEval Leaderboards on the Test set (open)

English - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	DUTIR	0.41280	0.48100	0.43020	0.45420	0.94890
2	PinganTeam	0.40430	0.49410	0.47550	0.48460	0.90210
3	PATeam	0.38300	0.46150	0.43020	0.44530	0.88940
3	PAteam	0.38300	0.46150	0.43020	0.44530	0.88940
3	PingAnAI	0.38300	0.46150	0.43020	0.44530	0.88940
3	PingAnTeam	0.38300	0.46150	0.43020	0.44530	0.88940
3	pateam	0.38300	0.46150	0.43020	0.44530	0.88940
4	DEMON	0.37450	0.44870	0.39620	0.42080	0.91910
5	gowithnlp	0.37020	0.44260	0.39250	0.41600	0.93620
6	TartanTritons	0.35740	0.11440	0.10190	0.10780	0.72340
7	Fane	0.35320	0.40850	0.36230	0.38400	0.91490
8	NlpUned	0.32770	0.37870	0.33580	0.35600	0.09790
8	QUST	0.32770	0.39040	0.36980	0.37980	0.92340
9	LATeIIMAS	0.31060	0.36710	0.32830	0.34660	0.83830
10	clujteam	0.28940	0.34470	0.30570	0.32400	0.86380
11	m1nadzuki	0.25960	0.31490	0.27920	0.29600	0.87660
12	LTG	0.25530	0.07490	0.07550	0.07520	0.58300
13	BERTastic	0.25110	0.31490	0.27920	0.29600	0.86380
13	adithjrajeev	0.25110	0.32140	0.30570	0.31330	0.86810
14	Mekky	0.21700	0.29480	0.27920	0.28680	0.80000
15	NarrativeMiners	0.21280	0.26810	0.23770	0.25200	0.68940
15	YNUzwt	0.21280	0.26380	0.23400	0.24800	0.77450
16	FromProblemImportSolve	0.20430	0.29030	0.30570	0.29780	0.77450
17	YNUHPCC	0.19150	0.20850	0.18490	0.19600	0.82550
18	Cimba	0.18720	0.23950	0.21510	0.22660	0.09790
19	NarrativeNexus	0.18300	0.20850	0.18490	0.19600	0.71910
20	Rosetta	0.17870	0.22920	0.21890	0.22390	0.60430
21	Dhananjaya	0.17450	0.22880	0.20380	0.21560	0.81280
22	Tuebingen	0.17020	0.26340	0.26040	0.26190	0.75740
23	UMZNLP	0.14040	0.21430	0.23770	0.22540	0.70640
24	north	0.08510	0.25870	0.42260	0.32090	0.94890
25	HowardUniversityAI4PC	0.08090	0.08320	0.17740	0.11330	0.69360
26	kzeky	0.06810	0.08090	0.07170	0.07600	0.56600
27	bumblebeeTransformer	0.06380	0.13640	0.16980	0.15130	0.78300
27	eevvgg	0.06380	0.10550	0.17360	0.13120	0.78720
28	Baseline	0.03830	0.04680	0.04150	0.04400	0.28510
29	Team12	0.02130	0.07670	0.09810	0.08610	0.72340
30	cocoa	0.01700	0.08750	0.12450	0.10280	0.82550
31	SemanticaInnovators	0.01280	0.04680	0.16600	0.07300	0.25110
32	TechDacians	0.00000	1.00000	0.00000	0.00000	0.09790
32	teamninebonn	0.00000	0.03260	0.08680	0.04740	0.17870

English - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	GATENLP	0.62700	0.35200	0.46300	0.34700
2	COGNAC	0.55400	0.40000	0.42600	0.39100
3	INSALyon2	0.51300	0.37800	0.40600	0.38200
4	23	0.49300	0.39200	0.37700	0.38400
5	KostasThesis2025	0.55600	0.39600	0.36200	0.37000
6	NCLteam	0.48600	0.36300	0.34500	0.36000
7	Narrlangen	0.44400	0.41100	0.34400	0.38600
8	PATeam	0.52100	0.35600	0.33900	0.29100
8	PAteam	0.52100	0.35600	0.33900	0.29100
8	PingAnAI	0.52100	0.35600	0.33900	0.29100
8	PingAnTeam	0.52100	0.35600	0.33900	0.29100
8	PinganTeam	0.52100	0.35600	0.33900	0.29100
8	pateam	0.52100	0.35600	0.33900	0.29100
9	YNUzwt	0.48500	0.37800	0.32100	0.33200
10	iLostTheCode	0.46700	0.35600	0.32000	0.32100
11	Narrengers	0.45700	0.36300	0.31800	0.31800
12	UNEDTeam	0.51200	0.36400	0.31300	0.29400
13	CtrlAltElite	0.47400	0.42200	0.31100	0.44600
14	NotMyNarrative	0.40300	0.41400	0.29800	0.39300
15	GeorgeSnape	0.28700	0.45200	0.28700	0.45200
15	IRNLP	0.51600	0.40200	0.28700	0.45200
16	INSAntive	0.44300	0.38000	0.28100	0.35200
17	NLPPraktikumWS2025	0.44800	0.38500	0.25800	0.33700
18	NarrativeMiners	0.36900	0.42400	0.23800	0.42600
19	nlptuducd	0.33800	0.36400	0.22600	0.31700
20	ammd7	0.29400	0.35500	0.22200	0.34300
21	UniBonn187	0.40900	0.33000	0.20600	0.22400
22	Irapuarani	0.33500	0.20100	0.18800	0.14100
23	DUTtask10	0.31000	0.40500	0.16500	0.36800
24	LATeIIMAS	0.22200	0.33400	0.16300	0.32300
25	GrammarPolice	0.16800	0.25800	0.06300	0.12400
26	Baseline	0.03000	0.12700	0.01300	0.07000
27	NarrativeNexus	0.00000	0.00000	0.00000	0.00000
27	bbStar	0.00000	0.00000	0.00000	0.00000

English - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	GPLSICORTEX	0.76754	0.73464	0.75040
2	KyuHyunChoi	0.76686	0.73517	0.75040
3	AITeam	0.74850	0.75229	0.75004
4	TechSSN	0.75417	0.74174	0.74767
5	pateam	0.74821	0.74742	0.74754
6	PingAnAI	0.74930	0.74283	0.74568
7	WordWiz	0.75464	0.73705	0.74551
8	PingAnTeam	0.73870	0.74376	0.74084
9	PinganTeam	0.73410	0.74832	0.74076
10	NarrativeNexus	0.71991	0.74267	0.73085
11	NarrativeMiners	0.71139	0.74827	0.72910
12	clujteam	0.72350	0.72647	0.72464
13	PAteam	0.72371	0.72589	0.72433
14	TartanTritons	0.72396	0.70200	0.71245
15	YNUzwt	0.69707	0.70112	0.69885
16	LATeIIMAS	0.70540	0.68674	0.69558
17	bbStar	0.66436	0.72023	0.69103
18	Synapse	0.65986	0.69175	0.67527
19	Baseline	0.65144	0.68344	0.66690
20	DUTtask10	0.00000	0.00000	0.00000
20	Mendel292A	0.00000	0.00000	0.00000
20	UMZNLP	0.00000	0.00000	0.00000
20	ftd	0.00000	0.00000	0.00000

Portuguese - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	DUTIR	0.59260	0.64950	0.62540	0.63720	0.94610
2	PingAnTeam	0.50510	0.56410	0.54490	0.55430	0.91250
3	PingAnAI	0.49490	0.55130	0.53250	0.54170	0.90570
4	PATeam	0.49160	0.55370	0.52630	0.53970	0.90570
4	PinganTeam	0.49160	0.55310	0.53250	0.54260	0.90240
4	pateam	0.49160	0.54950	0.53250	0.54090	0.90570
5	PAteam	0.48820	0.55020	0.52630	0.53800	0.90240
6	QUST	0.45790	0.50990	0.47680	0.49280	0.84510
7	BERTastic	0.41750	0.47470	0.43650	0.45480	0.84510
8	LTG	0.40740	0.11420	0.11460	0.11440	0.42760
9	DEMON	0.36700	0.43240	0.39630	0.41360	0.80810
10	LATeIIMAS	0.33670	0.38720	0.35600	0.37100	0.71720
11	TartanTritons	0.33330	0.18180	0.16720	0.17420	0.50840
12	gowithnlp	0.26940	0.31650	0.29100	0.30320	0.71380
13	Cimba	0.26260	0.33000	0.30960	0.31950	0.20540
13	FromProblemImportSolve	0.26260	0.32970	0.37150	0.34930	0.65660
14	Fane	0.25590	0.30980	0.28480	0.29680	0.62960
15	Dhananjaya	0.21890	0.26320	0.24770	0.25520	0.76770
16	YNUzwt	0.17510	0.22220	0.20430	0.21290	0.49490
17	HowardUniversityAI4PC	0.13130	0.11320	0.22600	0.15080	0.51850
18	Baseline	0.04710	0.05050	0.04640	0.04840	0.36030
19	teamninebonn	0.00000	0.04040	0.11150	0.05930	0.44110

Portuguese - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	GATENLP	0.66400	0.26000	0.48000	0.25400
2	PATeam	0.54100	0.29000	0.40900	0.26900
2	PAteam	0.54100	0.29000	0.40900	0.26900
2	PingAnAI	0.54100	0.29000	0.40900	0.26900
2	PingAnTeam	0.54100	0.29000	0.40900	0.26900
2	PinganTeam	0.54100	0.29000	0.40900	0.26900
2	pateam	0.54100	0.29000	0.40900	0.26900
3	KostasThesis2025	0.53900	0.21400	0.32900	0.17100
4	iLostTheCode	0.54700	0.22800	0.31900	0.18200
5	23	0.48700	0.33800	0.31300	0.25700
6	Narrlangen	0.48000	0.33900	0.29100	0.25900
7	UNEDTeam	0.53700	0.32400	0.27000	0.26200
8	YNUzwt	0.56800	0.30100	0.26600	0.27700
9	Irapuarani	0.43500	0.19500	0.22500	0.12900
10	INSAntive	0.49100	0.27500	0.21500	0.20400
11	COGNAC	0.34200	0.36200	0.18600	0.26200
12	CtrlAltElite	0.49200	0.31600	0.14900	0.21900
13	NotMyNarrative	0.22800	0.25000	0.12400	0.11100
14	DUTtask10	0.10700	0.24100	0.02600	0.13400
15	Baseline	0.03700	0.14000	0.01400	0.07000

Portuguese - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	PingAnTeam	0.76861	0.74874	0.75801
2	PinganTeam	0.76097	0.75381	0.75690
3	pateam	0.76003	0.75202	0.75548
4	PingAnAI	0.75403	0.74702	0.75005
5	WordWiz	0.75931	0.73885	0.74857
6	PAteam	0.75365	0.73984	0.74637
7	AITeam	0.74654	0.73767	0.74158
8	bbStar	0.70991	0.72924	0.71926
9	YNUzwt	0.68986	0.68603	0.68771
10	TartanTritons	0.69764	0.67419	0.68540
11	Baseline	0.67822	0.68297	0.68046
12	LATeIIMAS	0.68395	0.66339	0.67297
13	DUTtask10	0.00000	0.00000	0.00000

Russian - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	DUTIR	0.56540	0.61750	0.59030	0.60360	0.85510
2	QUST	0.51400	0.56280	0.53300	0.54750	0.83180
3	TartanTritons	0.47200	0.16200	0.15420	0.15800	0.43930
4	BERTastic	0.46730	0.50470	0.47580	0.48980	0.78970
4	DEMON	0.46730	0.51420	0.48020	0.49660	0.81310
5	gowithnlp	0.44860	0.48130	0.45370	0.46710	0.86450
6	PATeam	0.44390	0.49780	0.48900	0.49330	0.78040
6	PAteam	0.44390	0.49780	0.48900	0.49330	0.78040
6	PingAnAI	0.44390	0.50230	0.48900	0.49550	0.80840
6	PingAnTeam	0.44390	0.49780	0.48900	0.49330	0.78040
6	PinganTeam	0.44390	0.49780	0.48900	0.49330	0.78040
6	pateam	0.44390	0.49780	0.48900	0.49330	0.78040
7	LTG	0.42990	0.08620	0.08810	0.08710	0.35980
8	Cimba	0.38320	0.41670	0.39650	0.40630	0.33640
9	FromProblemImportSolve	0.35510	0.43700	0.51980	0.47480	0.75230
10	LATeIIMAS	0.31310	0.36040	0.35240	0.35630	0.64490
11	Dhananjaya	0.29440	0.32560	0.30840	0.31670	0.66360
11	YNUzwt	0.29440	0.32710	0.30840	0.31750	0.59350
12	Fane	0.24300	0.27570	0.25990	0.26760	0.60750
13	HowardUniversityAI4PC	0.12620	0.07590	0.12780	0.09520	0.42520
14	Baseline	0.05140	0.06070	0.05730	0.05900	0.34110

Russian - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	GATENLP	0.70900	0.27400	0.51800	0.28200
2	iLostTheCode	0.59700	0.27900	0.44000	0.25100
3	PATeam	0.56600	0.26800	0.43400	0.24700
3	PAteam	0.56600	0.26800	0.43400	0.24700
3	PingAnAI	0.56600	0.26800	0.43400	0.24700
3	PingAnTeam	0.56600	0.26800	0.43400	0.24700
3	PinganTeam	0.56600	0.26800	0.43400	0.24700
3	pateam	0.56600	0.26800	0.43400	0.24700
4	Narrlangen	0.57400	0.37000	0.40500	0.32400
5	KostasThesis2025	0.57100	0.34400	0.40000	0.28300
6	UNEDTeam	0.51300	0.32500	0.33000	0.27000
7	INSAntive	0.55400	0.32800	0.32300	0.34200
8	UniBonn187	0.55000	0.30300	0.23100	0.17800
9	Irapuarani	0.35900	0.15000	0.19100	0.09200
10	COGNAC	0.24800	0.35500	0.18300	0.30600
11	YNUzwt	0.21300	0.35900	0.14800	0.25700
12	IRNLP	0.53700	0.35100	0.11600	0.25200
13	GrammarPolice	0.00000	0.00000	0.05000	0.21800
14	DUTtask10	0.04000	0.18500	0.03300	0.18000
15	NotMyNarrative	0.12900	0.24600	0.01900	0.12800
16	Baseline	0.06500	0.21300	0.00800	0.06400
17	LATeIIMAS	0.00000	0.00000	0.00000	0.00000

Russian - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	PingAnAI	0.72469	0.71512	0.71904
2	PinganTeam	0.71787	0.72144	0.71888
3	PingAnTeam	0.71926	0.70627	0.71205
4	pateam	0.71232	0.71232	0.71166
5	AITeam	0.71354	0.71034	0.71136
6	PAteam	0.69984	0.71423	0.70639
7	WordWiz	0.70193	0.70730	0.70395
8	TartanTritons	0.68271	0.68155	0.68159
9	YNUzwt	0.66621	0.68632	0.67578
10	bbStar	0.63433	0.69813	0.66440
11	DUTtask10	0.65979	0.66845	0.66379
12	Baseline	0.62206	0.66872	0.64427
13	LATeIIMAS	0.61043	0.67674	0.64161

Bulgarian - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	PingAnAI	0.53230	0.54760	0.53910	0.54330	0.92740
2	PATeam	0.51610	0.53970	0.53120	0.53540	0.92740
3	DUTIR	0.50810	0.53730	0.56250	0.54960	0.94350
4	pateam	0.48390	0.51180	0.50780	0.50980	0.94350
5	PAteam	0.46770	0.49600	0.48440	0.49010	0.94350
5	PingAnTeam	0.46770	0.48820	0.48440	0.48630	0.94350
5	PinganTeam	0.46770	0.49610	0.49220	0.49410	0.94350
6	DEMON	0.45970	0.47970	0.46090	0.47010	0.88710
7	gowithnlp	0.43550	0.45970	0.44530	0.45240	0.83060
8	TartanTritons	0.41130	0.09680	0.09380	0.09520	0.55650
9	QUST	0.38710	0.39200	0.38280	0.38740	0.84680
10	BERTastic	0.35480	0.37100	0.35940	0.36510	0.87100
11	Fane	0.34680	0.36290	0.35160	0.35710	0.77420
12	LTG	0.31450	0.04930	0.05470	0.05190	0.55650
13	Cimba	0.25810	0.26980	0.26560	0.26770	0.25000
14	FromProblemImportSolve	0.20970	0.28660	0.36720	0.32190	0.80650
15	Dhananjaya	0.19350	0.21090	0.21090	0.21090	0.69350
16	HowardUniversityAI4PC	0.09680	0.07920	0.14840	0.10330	0.51610
17	Baseline	0.04030	0.04030	0.03910	0.03970	0.25810
18	TechDacians	0.00000	1.00000	0.00000	0.00000	0.25000

Bulgarian - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	PATeam	0.63100	0.33800	0.46000	0.33300
1	PingAnAI	0.63100	0.33800	0.46000	0.33300
1	PingAnTeam	0.63100	0.33800	0.46000	0.33300
1	PinganTeam	0.63100	0.33800	0.46000	0.33300
1	pateam	0.63100	0.33800	0.46000	0.33300
2	PAteam	0.57500	0.32900	0.44200	0.30700
3	GATENLP	0.62900	0.33200	0.41600	0.28600
4	iLostTheCode	0.55800	0.35100	0.39100	0.35200
5	UNEDTeam	0.57400	0.35300	0.36300	0.31200
6	KostasThesis2025	0.52300	0.37100	0.35700	0.34900
7	Narrlangen	0.49500	0.40100	0.35500	0.37600
8	INSAntive	0.52300	0.36600	0.32400	0.36000
9	Irapuarani	0.36600	0.17700	0.18300	0.12000
10	NotMyNarrative	0.24900	0.30600	0.14200	0.22800
11	DUTtask10	0.17200	0.34100	0.12100	0.31200
12	LATeIIMAS	0.17700	0.26600	0.07200	0.16900
13	Baseline	0.05600	0.19100	0.02200	0.11900

Bulgarian - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	AITeam	0.73485	0.70303	0.71818
2	PinganTeam	0.72676	0.71052	0.71815
3	pateam	0.72369	0.70837	0.71557
4	PingAnTeam	0.72954	0.70028	0.71418
5	PingAnAI	0.71217	0.69934	0.70527
6	PAteam	0.71405	0.69478	0.70396
7	WordWiz	0.69034	0.67816	0.68390
8	bbStar	0.65751	0.68740	0.67198
9	TartanTritons	0.66758	0.64399	0.65531
10	Baseline	0.62739	0.64180	0.63430
11	LATeIIMAS	0.62947	0.61907	0.62406
12	DUTtask10	0.00000	0.00000	0.00000

Hindi - Subtask 1

Rank	Team	Exact Match Ratio	micro P	micro R	micro F1	Accuracy for main role
1	QUST	0.46840	0.59060	0.49480	0.53850	0.76900
2	TartanTritons	0.44620	0.18940	0.15970	0.17330	0.32910
3	BERTastic	0.43990	0.56560	0.47380	0.51570	0.78480
4	DEMON	0.40190	0.52530	0.43460	0.47560	0.75630
5	LTG	0.36390	0.16580	0.15970	0.16270	0.37660
6	Cimba	0.35440	0.45310	0.37960	0.41310	0.47150
7	gowithnlp	0.33540	0.43990	0.36390	0.39830	0.71520
8	DUTIR	0.29430	0.37660	0.31150	0.34100	0.59810
9	Dhananjaya	0.27850	0.37660	0.31150	0.34100	0.70890
10	LATeIIMAS	0.27220	0.37110	0.40310	0.38640	0.63610
11	PATeam	0.26900	0.35330	0.29320	0.32050	0.69620
11	PAteam	0.26900	0.35330	0.29320	0.32050	0.69620
11	PingAnAI	0.26900	0.35330	0.29320	0.32050	0.69620
11	PinganTeam	0.26900	0.35330	0.29320	0.32050	0.69620
11	pateam	0.26900	0.35330	0.29320	0.32050	0.69620
12	FromProblemImportSolve	0.25630	0.37410	0.39270	0.38310	0.66770
12	PingAnTeam	0.25630	0.32490	0.26960	0.29470	0.67720
13	Fane	0.23420	0.31650	0.26180	0.28650	0.65510
14	HowardUniversityAI4PC	0.16770	0.11460	0.15180	0.13060	0.35440
15	Baseline	0.05700	0.07910	0.06540	0.07160	0.32280

Hindi - Subtask 2

Rank	Team	F1 macro coarse	F1 st. dev. coarse	F1 samples	F1 st. dev. samples
1	DUTtask10	0.56900	0.48400	0.53500	0.49400
2	IRNLP	0.37500	0.46700	0.51500	0.50000
3	Narrlangen	0.39500	0.46400	0.38500	0.46500
4	UNEDTeam	0.44900	0.46000	0.37600	0.45600
5	KostasThesis2025	0.45300	0.44100	0.34100	0.45000
6	GATENLP	0.40900	0.41000	0.32100	0.37900
7	INSAntive	0.36500	0.44000	0.26500	0.41400
8	NotMyNarrative	0.31800	0.44900	0.24300	0.40800
9	PATeam	0.39200	0.39000	0.21800	0.35000
9	PAteam	0.39200	0.39000	0.21800	0.35000
9	PingAnAI	0.39200	0.39000	0.21800	0.35000
9	PingAnTeam	0.39200	0.39000	0.21800	0.35000
9	PinganTeam	0.39200	0.39000	0.21800	0.35000
9	pateam	0.39200	0.39000	0.21800	0.35000
10	iLostTheCode	0.22700	0.37800	0.17400	0.33900
11	Irapuarani	0.23400	0.12500	0.11100	0.08200
12	LATeIIMAS	0.09700	0.18500	0.02900	0.10200
13	Baseline	0.08100	0.26000	0.00000	0.00000
14	bbStar	0.00000	0.00000	0.00000	0.00000

Hindi - Subtask 3

Rank	Team	Precision	Recall	F1 macro
1	AITeam	0.75576	0.76231	0.75845
2	pateam	0.75656	0.75959	0.75774
3	PinganTeam	0.75605	0.75613	0.75574
4	PAteam	0.75097	0.76045	0.75540
5	PingAnAI	0.74263	0.75416	0.74792
6	PingAnTeam	0.73661	0.74691	0.74145
7	WordWiz	0.72859	0.73914	0.73356
8	bbStar	0.71789	0.73704	0.72712
9	TartanTritons	0.69679	0.70119	0.69873
10	Baseline	0.66802	0.67188	0.66974
11	DUTtask10	0.00000	0.00000	0.00000

Contact

twitter.com/_proppy_
Register to the google group of the task to ask questions and interact with other participants

SemEval 2025 shared task on "Multilingual Characterization and Extraction of Narratives from Online News"

Leaderboard

Official SemEval 2025 task 10 Test Leaderboards

Post SemEval Leaderboards on the Test set (open)

English - Subtask 1

English - Subtask 2

English - Subtask 3

Portuguese - Subtask 1

Portuguese - Subtask 2

Portuguese - Subtask 3

Russian - Subtask 1

Russian - Subtask 2

Russian - Subtask 3

Bulgarian - Subtask 1

Bulgarian - Subtask 2

Bulgarian - Subtask 3

Hindi - Subtask 1

Hindi - Subtask 2

Hindi - Subtask 3

Contact