Partition strategy on Lichess

Qwen3.5-9B · N=4 · 400 held-out puzzles (rating 800–1199) × 4 samples

Partition strategy on Lichess010203040506070ScoreSolved (pass@1)Mean rewardGRPO 24.84GRPO 34.234.6937.4444.6942.5545.4451.93
GRPO baselineFull dataset random seedsRandomly partitionedSemantic partitions
View data table
SeriesCategoryScoreInterval / statistics
GRPOSolved (pass@1)24.84
Full dataset random seedsSolved (pass@1)34.69
Randomly partitionedSolved (pass@1)37.44
Semantic partitionsSolved (pass@1)44.69
GRPOMean reward34.2
Full dataset random seedsMean reward42.55
Randomly partitionedMean reward45.44
Semantic partitionsMean reward51.93