Skip to content
IonWarpTry for free

Cost review benchmark: models compared (Sep 2026)

DeepSeek V4.1 Flash has the highest F1 of the 3 models graded on every PR: 57.1%, at $0.07 a review. No app was graded on these 20 PRs.

Reviewer / modelF1PrecisionRecall$ per reviewp50 timePRs
IonWarp Cost · GLM 5.3 Flash55.2%53.3%57.1%$0.012m 23s20/20
IonWarp Cost · DeepSeek V4.1 Flash57.1%47.6%71.4%$0.0710m 39s20/20
IonWarp Cost · GPT-6 Luna Pro50.0%60.0%42.9%$0.0227 sec20/20
CodeRabbitN/AN/AN/A~$1.50N/AN/A
Cursor BugbotN/AN/AN/A~$1.00–1.50N/AN/A
Greptile v4.1N/AN/AN/A~$1.50–2.00N/AN/A
Qodo v2N/AN/AN/A~$1.67N/AN/A

Sorted by Reviewer Score (F1 counts 3×, time and cost 1× each). Rows not graded on every PR come last. Review times come from runs on Sep 22–24, 2026 (models). App prices (~) are list-price estimates; model prices are measured inference cost per review. Provisional labels: IonWarp chose which issues count as Cost Review findings, and no independent party has checked that choice.

20 PRs · snapshot · How we measure