Leaderboard

Official SemEval 2025 task 10 Test Leaderboards

Post SemEval Leaderboards on the Test set (open)

English - Subtask 1

RankTeamExact Match Ratiomicro Pmicro Rmicro F1Accuracy for main role
1DUTIR0.412800.481000.430200.454200.94890
2PinganTeam0.404300.494100.475500.484600.90210
3slowders0.391500.455300.403800.428000.90210
4PATeam0.383000.461500.430200.445300.88940
4PAteam0.383000.461500.430200.445300.88940
4PingAnAI0.383000.461500.430200.445300.88940
4PingAnTeam0.383000.461500.430200.445300.88940
4pateam0.383000.461500.430200.445300.88940
5DEMON0.374500.448700.396200.420800.91910
6gowithnlp0.370200.442600.392500.416000.93620
7TartanTritons0.357400.114400.101900.107800.72340
8Fane0.353200.408500.362300.384000.91490
9NlpUned0.327700.378700.335800.356000.09790
9QUST0.327700.390400.369800.379800.92340
10LATeIIMAS0.310600.367100.328300.346600.83830
11clujteam0.289400.344700.305700.324000.86380
12m1nadzuki0.259600.314900.279200.296000.87660
13LTG0.255300.074900.075500.075200.58300
14BERTastic0.251100.314900.279200.296000.86380
14adithjrajeev0.251100.321400.305700.313300.86810
15Cimba0.225500.278500.249100.262900.83830
16Mekky0.217000.294800.279200.286800.80000
17NarrativeMiners0.212800.268100.237700.252000.68940
17YNUzwt0.212800.263800.234000.248000.77450
18FromProblemImportSolve0.204300.290300.305700.297800.77450
19YNUHPCC0.191500.208500.184900.196000.82550
20NarrativeNexus0.183000.208500.184900.196000.71910
21Rosetta0.178700.229200.218900.223900.60430
22Dhananjaya0.174500.228800.203800.215600.81280
23Tuebingen0.170200.263400.260400.261900.75740
24Chrissy0.148900.187200.166000.176000.83400
25UMZNLP0.140400.214300.237700.225400.70640
26north0.085100.258700.422600.320900.94890
27HowardUniversityAI4PC0.080900.083200.177400.113300.69360
28kzeky0.068100.080900.071700.076000.56600
29bumblebeeTransformer0.063800.136400.169800.151300.78300
29eevvgg0.063800.105500.173600.131200.78720
30Baseline0.038300.046800.041500.044000.28510
31Team120.021300.076700.098100.086100.72340
32cocoa0.017000.087500.124500.102800.82550
33SemanticaInnovators0.012800.046800.166000.073000.25110
34TechDacians0.000001.000000.000000.000000.09790
34teamninebonn0.000000.032600.086800.047400.17870

English - Subtask 2

RankTeamF1 macro coarseF1 st. dev. coarseF1 samplesF1 st. dev. samples
1GATENLP0.627000.352000.463000.34700
2INSALyon20.547000.369000.433000.33300
3COGNAC0.554000.400000.426000.39100
4NarrativesHunter0.509000.395000.400000.40200
5230.493000.392000.377000.38400
6iLostTheCode0.498000.363000.373000.37300
7KostasThesis20250.556000.396000.362000.37000
8NCLteam0.486000.363000.345000.36000
9NarrativesHunter20.488000.384000.344000.36700
9Narrlangen0.444000.411000.344000.38600
10PATeam0.521000.356000.339000.29100
10PAteam0.521000.356000.339000.29100
10PingAnAI0.521000.356000.339000.29100
10PingAnTeam0.521000.356000.339000.29100
10PinganTeam0.521000.356000.339000.29100
10pateam0.521000.356000.339000.29100
11UNEDTeam0.467000.363000.327000.34600
12YNUzwt0.485000.378000.321000.33200
13Narrengers0.457000.363000.318000.31800
14CtrlAltElite0.474000.422000.311000.44600
15NotMyNarrative0.403000.414000.298000.39300
16GeorgeSnape0.287000.452000.287000.45200
16IRNLP0.516000.402000.287000.45200
17INSAntive0.329000.412000.259000.38900
18NLPPraktikumWS20250.448000.385000.258000.33700
19NarrativeMiners0.369000.424000.238000.42600
20nlptuducd0.338000.364000.226000.31700
21ammd70.294000.355000.222000.34300
22UniBonn1870.409000.330000.206000.22400
23Irapuarani0.335000.201000.188000.14100
24DUTtask100.310000.405000.165000.36800
25LATeIIMAS0.222000.334000.163000.32300
26GrammarPolice0.168000.258000.063000.12400
27Baseline0.030000.127000.013000.07000
28AlAnood0.000000.000000.000000.00000
28NarrativeNexus0.000000.000000.000000.00000
28bbStar0.000000.000000.000000.00000

English - Subtask 3

RankTeamPrecisionRecallF1 macro
1GPLSICORTEX0.767540.734640.75040
2KyuHyunChoi0.766860.735170.75040
3AITeam0.748500.752290.75004
4TechSSN0.754170.741740.74767
5pateam0.748210.747420.74754
6PingAnAI0.749300.742830.74568
7WordWiz0.754640.737050.74551
8PingAnTeam0.738700.743760.74084
9PinganTeam0.734100.748320.74076
10NarrativeNexus0.719910.742670.73085
11NarrativeMiners0.711390.748270.72910
12clujteam0.723500.726470.72464
13PAteam0.723710.725890.72433
14Dhananjaya0.707000.727880.71689
15TartanTritons0.723960.702000.71245
16YNUzwt0.697070.701120.69885
17LATeIIMAS0.705400.686740.69558
18bbStar0.664360.720230.69103
19Synapse0.659860.691750.67527
20Baseline0.651440.683440.66690
21DUTtask100.000000.000000.00000
21Mendel292A0.000000.000000.00000
21UMZNLP0.000000.000000.00000
21ftd0.000000.000000.00000

Portuguese - Subtask 1

RankTeamExact Match Ratiomicro Pmicro Rmicro F1Accuracy for main role
1DUTIR0.592600.649500.625400.637200.94610
2PingAnTeam0.505100.564100.544900.554300.91250
3PingAnAI0.494900.551300.532500.541700.90570
4PATeam0.491600.553700.526300.539700.90570
4PinganTeam0.491600.553100.532500.542600.90240
4pateam0.491600.549500.532500.540900.90570
5PAteam0.488200.550200.526300.538000.90240
6QUST0.457900.509900.476800.492800.84510
7BERTastic0.417500.474700.436500.454800.84510
8LTG0.407400.114200.114600.114400.42760
9DEMON0.367000.432400.396300.413600.80810
10LATeIIMAS0.336700.387200.356000.371000.71720
11TartanTritons0.333300.181800.167200.174200.50840
12gowithnlp0.269400.316500.291000.303200.71380
13Cimba0.262600.330000.309600.319500.20540
13FromProblemImportSolve0.262600.329700.371500.349300.65660
14Fane0.255900.309800.284800.296800.62960
15Dhananjaya0.218900.263200.247700.255200.76770
16YNUzwt0.175100.222200.204300.212900.49490
17HowardUniversityAI4PC0.131300.113200.226000.150800.51850
18Baseline0.047100.050500.046400.048400.36030
19teamninebonn0.000000.040400.111500.059300.44110

Portuguese - Subtask 2

RankTeamF1 macro coarseF1 st. dev. coarseF1 samplesF1 st. dev. samples
1GATENLP0.664000.260000.480000.25400
2INSALyon20.679000.263000.433000.20900
3PATeam0.541000.290000.409000.26900
3PAteam0.541000.290000.409000.26900
3PingAnAI0.541000.290000.409000.26900
3PingAnTeam0.541000.290000.409000.26900
3PinganTeam0.541000.290000.409000.26900
3pateam0.541000.290000.409000.26900
4KostasThesis20250.539000.214000.329000.17100
5iLostTheCode0.547000.228000.319000.18200
6230.487000.338000.313000.25700
7Narrlangen0.480000.339000.291000.25900
8UNEDTeam0.537000.324000.270000.26200
9YNUzwt0.568000.301000.266000.27700
10Irapuarani0.435000.195000.225000.12900
11INSAntive0.491000.275000.215000.20400
12COGNAC0.342000.362000.186000.26200
13CtrlAltElite0.492000.316000.149000.21900
14NotMyNarrative0.228000.250000.124000.11100
15DUTtask100.107000.241000.026000.13400
16Baseline0.037000.140000.014000.07000

Portuguese - Subtask 3

RankTeamPrecisionRecallF1 macro
1PingAnTeam0.768610.748740.75801
2PinganTeam0.760970.753810.75690
3pateam0.760030.752020.75548
4PingAnAI0.754030.747020.75005
5WordWiz0.759310.738850.74857
6PAteam0.753650.739840.74637
7AITeam0.746540.737670.74158
8bbStar0.709910.729240.71926
9Dhananjaya0.709790.723180.71614
10YNUzwt0.689860.686030.68771
11TartanTritons0.697640.674190.68540
12Baseline0.678220.682970.68046
13LATeIIMAS0.683950.663390.67297
14DUTtask100.000000.000000.00000

Russian - Subtask 1

RankTeamExact Match Ratiomicro Pmicro Rmicro F1Accuracy for main role
1DUTIR0.565400.617500.590300.603600.85510
2QUST0.514000.562800.533000.547500.83180
3TartanTritons0.472000.162000.154200.158000.43930
4BERTastic0.467300.504700.475800.489800.78970
4DEMON0.467300.514200.480200.496600.81310
5gowithnlp0.448600.481300.453700.467100.86450
6PATeam0.443900.497800.489000.493300.78040
6PAteam0.443900.497800.489000.493300.78040
6PingAnAI0.443900.502300.489000.495500.80840
6PingAnTeam0.443900.497800.489000.493300.78040
6PinganTeam0.443900.497800.489000.493300.78040
6pateam0.443900.497800.489000.493300.78040
7LTG0.429900.086200.088100.087100.35980
8Cimba0.383200.416700.396500.406300.33640
9FromProblemImportSolve0.355100.437000.519800.474800.75230
10LATeIIMAS0.313100.360400.352400.356300.64490
11Dhananjaya0.294400.325600.308400.316700.66360
11YNUzwt0.294400.327100.308400.317500.59350
12Fane0.243000.275700.259900.267600.60750
13HowardUniversityAI4PC0.126200.075900.127800.095200.42520
14Baseline0.051400.060700.057300.059000.34110

Russian - Subtask 2

RankTeamF1 macro coarseF1 st. dev. coarseF1 samplesF1 st. dev. samples
1GATENLP0.709000.274000.518000.28200
2iLostTheCode0.597000.279000.440000.25100
3PATeam0.566000.268000.434000.24700
3PAteam0.566000.268000.434000.24700
3PingAnAI0.566000.268000.434000.24700
3PingAnTeam0.566000.268000.434000.24700
3PinganTeam0.566000.268000.434000.24700
3pateam0.566000.268000.434000.24700
4INSALyon20.589000.293000.410000.24400
5Narrlangen0.574000.370000.405000.32400
6KostasThesis20250.571000.344000.400000.28300
7UNEDTeam0.513000.325000.330000.27000
8INSAntive0.554000.328000.323000.34200
9UniBonn1870.550000.303000.231000.17800
10Irapuarani0.359000.150000.191000.09200
11COGNAC0.248000.355000.183000.30600
12YNUzwt0.213000.359000.148000.25700
13IRNLP0.537000.351000.116000.25200
14GrammarPolice0.000000.000000.050000.21800
15DUTtask100.040000.185000.033000.18000
16NotMyNarrative0.129000.246000.019000.12800
17Baseline0.065000.213000.008000.06400
18AlAnood0.000000.000000.000000.00000
18LATeIIMAS0.000000.000000.000000.00000

Russian - Subtask 3

RankTeamPrecisionRecallF1 macro
1PingAnAI0.724690.715120.71904
2PinganTeam0.717870.721440.71888
3PingAnTeam0.719260.706270.71205
4pateam0.712320.712320.71166
5AITeam0.713540.710340.71136
6PAteam0.699840.714230.70639
7WordWiz0.701930.707300.70395
8TartanTritons0.682710.681550.68159
9YNUzwt0.666210.686320.67578
10bbStar0.634330.698130.66440
11DUTtask100.659790.668450.66379
12Dhananjaya0.636680.693080.66324
13Baseline0.622060.668720.64427
14LATeIIMAS0.610430.676740.64161

Bulgarian - Subtask 1

RankTeamExact Match Ratiomicro Pmicro Rmicro F1Accuracy for main role
1PingAnAI0.532300.547600.539100.543300.92740
2PATeam0.516100.539700.531200.535400.92740
3DUTIR0.508100.537300.562500.549600.94350
4pateam0.483900.511800.507800.509800.94350
5PAteam0.467700.496000.484400.490100.94350
5PingAnTeam0.467700.488200.484400.486300.94350
5PinganTeam0.467700.496100.492200.494100.94350
6DEMON0.459700.479700.460900.470100.88710
7gowithnlp0.435500.459700.445300.452400.83060
8TartanTritons0.411300.096800.093800.095200.55650
9QUST0.387100.392000.382800.387400.84680
10BERTastic0.354800.371000.359400.365100.87100
11Fane0.346800.362900.351600.357100.77420
12LTG0.314500.049300.054700.051900.55650
13Cimba0.258100.269800.265600.267700.25000
14FromProblemImportSolve0.209700.286600.367200.321900.80650
15Dhananjaya0.193500.210900.210900.210900.69350
16HowardUniversityAI4PC0.096800.079200.148400.103300.51610
17Baseline0.040300.040300.039100.039700.25810
18TechDacians0.000001.000000.000000.000000.25000

Bulgarian - Subtask 2

RankTeamF1 macro coarseF1 st. dev. coarseF1 samplesF1 st. dev. samples
1PATeam0.631000.338000.460000.33300
1PingAnAI0.631000.338000.460000.33300
1PingAnTeam0.631000.338000.460000.33300
1PinganTeam0.631000.338000.460000.33300
1pateam0.631000.338000.460000.33300
2PAteam0.575000.329000.442000.30700
3GATENLP0.629000.332000.416000.28600
4iLostTheCode0.558000.351000.391000.35200
5INSALyon20.590000.311000.381000.24800
6UNEDTeam0.574000.353000.363000.31200
7KostasThesis20250.523000.371000.357000.34900
8Narrlangen0.495000.401000.355000.37600
9INSAntive0.523000.366000.324000.36000
10Irapuarani0.366000.177000.183000.12000
11NotMyNarrative0.249000.306000.142000.22800
12DUTtask100.172000.341000.121000.31200
13LATeIIMAS0.177000.266000.072000.16900
14Baseline0.056000.191000.022000.11900
15AlAnood0.000000.000000.000000.00000

Bulgarian - Subtask 3

RankTeamPrecisionRecallF1 macro
1AITeam0.734850.703030.71818
2PinganTeam0.726760.710520.71815
3pateam0.723690.708370.71557
4PingAnTeam0.729540.700280.71418
5PingAnAI0.712170.699340.70527
6PAteam0.714050.694780.70396
7Dhananjaya0.676950.693780.68508
8WordWiz0.690340.678160.68390
9bbStar0.657510.687400.67198
10TartanTritons0.667580.643990.65531
11Baseline0.627390.641800.63430
12LATeIIMAS0.629470.619070.62406
13DUTtask100.000000.000000.00000

Hindi - Subtask 1

RankTeamExact Match Ratiomicro Pmicro Rmicro F1Accuracy for main role
1QUST0.468400.590600.494800.538500.76900
2TartanTritons0.446200.189400.159700.173300.32910
3BERTastic0.439900.565600.473800.515700.78480
4DEMON0.401900.525300.434600.475600.75630
5LTG0.363900.165800.159700.162700.37660
6Cimba0.354400.453100.379600.413100.47150
7gowithnlp0.335400.439900.363900.398300.71520
8DUTIR0.294300.376600.311500.341000.59810
9Dhananjaya0.278500.376600.311500.341000.70890
10LATeIIMAS0.272200.371100.403100.386400.63610
11PATeam0.269000.353300.293200.320500.69620
11PAteam0.269000.353300.293200.320500.69620
11PingAnAI0.269000.353300.293200.320500.69620
11PinganTeam0.269000.353300.293200.320500.69620
11pateam0.269000.353300.293200.320500.69620
12FromProblemImportSolve0.256300.374100.392700.383100.66770
12PingAnTeam0.256300.324900.269600.294700.67720
13Fane0.234200.316500.261800.286500.65510
14HowardUniversityAI4PC0.167700.114600.151800.130600.35440
15Baseline0.057000.079100.065400.071600.32280

Hindi - Subtask 2

RankTeamF1 macro coarseF1 st. dev. coarseF1 samplesF1 st. dev. samples
1DUTtask100.569000.484000.535000.49400
2IRNLP0.375000.467000.515000.50000
3INSALyon20.515000.433000.435000.42400
4Narrlangen0.395000.464000.385000.46500
5UNEDTeam0.449000.460000.376000.45600
6KostasThesis20250.453000.441000.341000.45000
7GATENLP0.409000.410000.321000.37900
8INSAntive0.365000.440000.265000.41400
9NotMyNarrative0.318000.449000.243000.40800
10PATeam0.392000.390000.218000.35000
10PAteam0.392000.390000.218000.35000
10PingAnAI0.392000.390000.218000.35000
10PingAnTeam0.392000.390000.218000.35000
10PinganTeam0.392000.390000.218000.35000
10pateam0.392000.390000.218000.35000
11iLostTheCode0.227000.378000.174000.33900
12Irapuarani0.234000.125000.111000.08200
13LATeIIMAS0.097000.185000.029000.10200
14Baseline0.081000.260000.000000.00000
15bbStar0.000000.000000.000000.00000

Hindi - Subtask 3

RankTeamPrecisionRecallF1 macro
1AITeam0.755760.762310.75845
2pateam0.756560.759590.75774
3PinganTeam0.756050.756130.75574
4PAteam0.750970.760450.75540
5PingAnAI0.742630.754160.74792
6PingAnTeam0.736610.746910.74145
7WordWiz0.728590.739140.73356
8bbStar0.717890.737040.72712
9Dhananjaya0.721430.721560.72125
10TartanTritons0.696790.701190.69873
11Baseline0.668020.671880.66974
12DUTtask100.000000.000000.00000

Contact