Skip to content

AI model leaderboards

Get API keyRead the docsAbout this data
See top models, labs, apps, and providers on AI Gateway with token, request, and cost trends.

DeepsecBench measures how well different AI models find security vulnerabilities in application code, based on the deepsec cyber harness(opens in new tab). The report combines recall and precision into a single benchmark score.

Score vs. cost

Score versus Cost (USD) on a linear x-axis for each model configurationLower cost (usd) appears to the right. Ringed points and the solid line mark the Pareto frontier: configurations that are not dominated by another configuration with at least as high a score and no higher cost (usd), with one of those metrics strictly better.01020304050$0$50$100$150Cost (USD)Score

Score vs. total time

Score versus Total time on a linear x-axis for each model configurationLower total time appears to the right. Ringed points and the solid line mark the Pareto frontier: configurations that are not dominated by another configuration with at least as high a score and no higher total time, with one of those metrics strictly better.010203040500h2h4h6h8hTotal timeScore

Results

1
GPT-6 Sol xhighopenai/gpt-6-sol
40.75
35.6%96.0%4$12.761h 09m 09s1,716.8Codex
2
GPT-6.1 Sol xhighopenai/gpt-6.1-sol
39.49
34.3%99.0%1$12.371h 21m 01s1,389.0Codex
3
GPT-6 Astra xhighopenai/gpt-6-astra
37.64
32.6%97.9%2$63.7049m 47s1,683.9Codex
4
Claude Sonnet 5.5 maxanthropic/claude-sonnet-5.5
37.50
32.6%93.3%6$65.256h 16m 28s2,574.9Claude Code
5
Kimi K3.1 Pro maxmoonshotai/kimi-k3.1-pro
37.45
32.6%92.0%7$16.262h 02m 48s2,155.3Pi
6
GPT-5.6 Sol xhighopenai/gpt-5.6-sol
35.30
30.5%96.3%3$55.983h 39m 02s2,274.9Codex
7
Claude Opus 5 maxanthropic/claude-opus-5
32.31
27.9%88.0%10$127.932h 34m 19s4,453.7Claude Code
8
GPT-6.1 Sol mediumopenai/gpt-6.1-sol
30.69
26.2%98.6%1$12.1537m 26s753.6Codex
9
Deepseek V4.1 Flash 1017 maxdeepseek/deepseek-v4.1-flash-1017
29.97
25.8%87.0%10$7.871h 04m 41s1,640.1Pi
10
GPT-6 Astra mediumopenai/gpt-6-astra
29.21
24.9%95.5%3$35.2021m 49s973.0Codex
11
GPT-6 Sol mediumopenai/gpt-6-sol
29.19
24.9%94.2%4$12.0528m 08s932.1Codex
12
Kimi K3.1 Pro highmoonshotai/kimi-k3.1-pro
28.71
24.5%93.8%4$5.5042m 18s893.5Pi
13
Claude Opus 5 mediumanthropic/claude-opus-5
28.14
24.5%70.5%26$31.9647m 01s999.7Claude Code
14
GPT-5.6 Luna xhighopenai/gpt-5.6-luna
26.57
22.7%81.2%13$24.593h 01m 40s3,343.4Codex
15
Claude Opus 5.5 maxanthropic/claude-opus-5.5
26.57
22.7%81.0%15$40.313h 51m 08s1,175.1Claude Code
16
GPT-5.6 Sol mediumopenai/gpt-5.6-sol
24.90
21.0%94.3%3$17.9530m 52s703.0Codex
17
Deepseek V4.1 Flash 1017 highdeepseek/deepseek-v4.1-flash-1017
24.71
21.0%82.1%12$0.5647m 39s1,180.7Pi
18
GLM-5.3 highzai/glm-5.3
21.82
18.5%80.7%11$16.222h 05m 37s1,232.1Pi
19
GPT-5.5 xhighopenai/gpt-5.5
21.03
17.6%95.5%2$43.422h 22m 57s2,158.8Codex
20
GPT-6 Luna xhighopenai/gpt-6-luna
20.97
17.6%89.6%5$0.511h 00m 36s1,423.5Codex
21
GLM-5.3 maxzai/glm-5.3
20.31
17.2%75.9%13$14.601h 29m 34s1,241.8Pi
22
GPT-5.6 Terra xhighopenai/gpt-5.6-terra
19.06
15.9%95.3%2$27.811h 45m 32s2,108.7Codex
23
Kimi K3 xhighmoonshotai/kimi-k3
17.42
14.6%77.1%11$12.381h 59m 39s731.9Pi
24
Deepseek V4 Flash xhighdeepseek/deepseek-v4-flash
16.41
13.7%74.0%13$5.941h 18m 09s1,481.0Pi
25
Qwen3.8 Max xhighalibaba/qwen3.8-max
16.41
13.7%73.9%12$6.331h 52m 20s342.6Pi
26
Grok 4.5 mediumxai/grok-4.5
16.40
13.7%73.3%12$11.041h 37m 33s909.6Pi
27
Grok 4.6 xhighxai/grok-4.6
16.10
13.3%100.0%0$22.294h 02m 36s3,456.4Pi
28
Qwen3.8 Max mediumalibaba/qwen3.8-max
15.46
12.9%78.9%8$6.222h 03m 33s383.4Pi
29
Grok 4.5 xhighxai/grok-4.5
15.45
12.9%77.8%10$5.601h 24m 17s1,010.5Pi
30
Deepseek V4 Flash mediumdeepseek/deepseek-v4-flash
15.35
12.9%66.7%16$5.061h 21m 16s1,422.5Pi
31
Claude Sonnet 5 maxanthropic/claude-sonnet-5
14.94
12.4%75.6%10$43.012h 54m 53s4,213.5Claude Code
32
GPT-6 Luna mediumopenai/gpt-6-luna
13.53
11.2%89.7%3$1.7524m 56s873.5Codex
33
GLM-5.3 Flash highzai/glm-5.3-flash
12.87
10.7%64.3%15$0.3521m 21s1,264.6Pi
34
GPT-5.5 mediumopenai/gpt-5.5
12.53
10.3%92.3%2$14.9424m 05s751.4Codex
35
GPT-5.6 Terra mediumopenai/gpt-5.6-terra
12.02
9.9%92.0%2$5.1513m 15s617.7Codex
36
GPT-5.6 Luna mediumopenai/gpt-5.6-luna
11.98
9.9%82.8%5$5.2311m 59s572.0Codex
37
Claude Sonnet 5.5 mediumanthropic/claude-sonnet-5.5
11.48
9.4%84.6%4$4.3303m 18s23.9Claude Code
38
Gemini 3.6 Flash mediumgoogle/gemini-3.6-flash
11.46
9.4%80.0%6$10.7426m 21s2,730.7Pi
39
Kimi K3 mediummoonshotai/kimi-k3
11.38
9.4%64.1%14$10.841h 29m 16s676.6Pi
40
Grok 4.7 xhighspacexai/grok-4.7
10.89
9.0%64.7%12$13.2842m 23s1,703.4Pi
41
GLM-5.3 Flash maxzai/glm-5.3-flash
10.87
9.0%62.5%15$1.8341m 04s1,598.4Pi
42
GLM-5.2 mediumzai/glm-5.2
10.37
8.6%62.5%12$11.091h 03m 38s1,498.9Pi
43
Claude Opus 5.5 mediumanthropic/claude-opus-5.5
9.95
8.2%82.6%4$7.3207m 01s38.0Claude Code
44
GLM-5.2 xhighzai/glm-5.2
9.82
8.2%53.8%18$51.842h 30m 44s4,727.6Pi
45
Gemini 3.6 Flash xhighgoogle/gemini-3.6-flash
8.91
7.3%77.3%5$5.3322m 18s1,822.4Pi
46
Grok 4.7 mediumspacexai/grok-4.7
7.81
6.4%53.3%14$3.9721m 28s904.5Pi
47
Claude Opus 4.8 maxanthropic/claude-opus-4.8
6.86
5.6%81.3%3$7.5619m 29s80.8Claude Code
48
Claude Opus 4.8 mediumanthropic/claude-opus-4.8
6.82
5.6%61.9%8$10.8118m 16s78.1Claude Code
49
Inkling xhighthinkingmachines/inkling
6.27
5.2%48.0%13$1.9311m 34s300.9Pi
50
Claude Sonnet 5 mediumanthropic/claude-sonnet-5
5.79
4.7%61.1%7$6.6615m 47s442.2Claude Code
51
Inkling mediumthinkingmachines/inkling
5.21
4.3%37.0%17$2.1810m 31s509.0Pi
52
Claude Haiku 4.5 defaultanthropic/claude-haiku-4.5
4.69
3.9%33.3%18$5.9129m 40s912.4Claude Code
53
Gemini 3.5 Flash Lite mediumgoogle/gemini-3.5-flash-lite
2.13
1.7%44.4%5$0.5801m 46s459.9Pi
54
Gemini 3.5 Flash Lite xhighgoogle/gemini-3.5-flash-lite
1.06
0.9%28.6%5$0.1801m 07s406.5Pi