Skip to content
IonWarpTry for free

AI architecture review benchmark: models vs apps (Sep 2026)

DeepSeek V4.1 Flash and GLM 5.3 tie for the highest F1 of the 8 models graded on every PR (47.1%); DeepSeek V4.1 Flash is the cheaper, at $0.05 a review in 6m 14s. An app leads: CodeRabbit has the highest F1 of the 4 apps graded, 72.7%.

Reviewer / modelF1PrecisionRecall$ per reviewp50 timePRs
CodeRabbit72.7%80.0%66.7%~$1.507m 43s11/11
IonWarp AI Architecture · GLM 5.347.1%36.4%66.7%$0.071m 29s11/11
IonWarp AI Architecture · GPT 5.6 Luna Pro35.3%27.3%50.0%$0.0551 sec11/11
IonWarp AI Architecture · DeepSeek V4 Pro44.4%33.3%66.7%$0.101m 52s11/11
Greptile v4.160.0%75.0%50.0%~$1.50–2.005m 6s11/11
IonWarp AI Architecture · DeepSeek V4.1 Flash47.1%36.4%66.7%$0.056m 14s11/11
IonWarp AI Architecture · GLM 5.3 Flash15.4%14.3%16.7%$0.011m 2s11/11
Cursor Bugbot50.0%40.0%66.7%~$1.00–1.507m 46s11/11
IonWarp AI Architecture · Grok 4.634.8%23.5%66.7%$0.283m 29s11/11
IonWarp AI Architecture · Muse Spark 1.328.6%20.0%50.0%$0.192m 4s11/11
IonWarp AI Architecture · Gemini 3.8 Flash33.3%25.0%50.0%$0.243m 50s11/11
Qodo v225.0%50.0%16.7%~$1.675m 52s11/11

Sorted by Reviewer Score (F1 counts 3×, time and cost 1× each). Review times come from runs on Sep 8–24, 2026 (models) and Jan 23–Apr 7, 2026 (apps). App prices (~) are list-price estimates; model prices are measured inference cost per review. Provisional labels: IonWarp chose which issues count as AI Architecture Review findings, and no independent party has checked that choice.

11 PRs · snapshot · How we measure