Validating the source before calculating rankings.
Amortized agent time and cost per pass
The table applies the selected time/cost weighting; the chart shows the underlying trade-off. 4-run success is supplemental and does not affect value or Pareto status. The bottom-left is more efficient.
Models
Leaderboard
| Model | Value | Time/passTime/ Pass |
Cost/passCost/ Pass |
1-run success1-run success |
4-run success4-run success |
|---|
No configurations meet the current table filters.
Data and methodology
- Data generated
- —
- Data fetched
- —
- Formula
- Formula v1 methodology
Amortized time and cost divide aggregate attempt averages by the 1-run success rate (Pass@1); they do not model stopping after a successful retry. Coverage may vary because the source excludes some errors.
Data published by DeepSWE by DataCurve. This leaderboard is independent and unaffiliated.