Home · Technology · Nov 3 archive

Gemini 4 Carbon Scores High in Cybersecurity, Coding Benchmarks

12:44 AM ETConfirmedShare on:      

Technology Desk

In Short: Internal tests show Carbon outperforming Argon, GPT-6 Astra, and Claude Opus 5.5 on benchmarks for coding, knowledge work, science, math, multimodal understanding, and cybersecurity.

Google's latest AI model, Gemini 4 Carbon, has been benchmarked by SemiAnalysis, a research firm, in a new frontier-level benchmark called GDPx.

The benchmark includes 206 tasks across various domains, including finance, materials science, cybersecurity, and software engineering.

In cybersecurity, Carbon achieved a 0.0% attack success rate, compared to 0.7% for Argon and 1.0% for Claude Opus 5.5.

On the Artificial Analysis Intelligence Index v5.0, Carbon ranks first, followed by Claude Opus 5.5 and GPT-6 Astra.

Gemini 4 Carbon also leads in Terminal-Bench 4.0 (AA) with a score of 65.2%, ahead of GPT-6 Astra (59.1%), Opus 5.5 (59.6%), Argon (57.1%), and Fable 5.1 (52.0%).

However, on FrontierSWE 2, GPT-6 Astra takes the top spot with 65.5%, ahead of Carbon’s 61.2%, Opus 5.5’s 62.3%, Argon’s 55.0%, and Fable 5.1’s 56.3%.

Google announced Carbon on October 28, highlighting its significant gains in coding, agentic tasks, and cybersecurity.

Carbon is available to a subset of cyber defenders at $2 per million input tokens and $10 per million output tokens.

Internal tests show Carbon outperforming Argon, GPT-6 Astra, and Claude Opus 5.5 on benchmarks for coding, knowledge work, science, math, multimodal understanding, and cybersecurity.

Google says Carbon reduces shortcuts and loophole exploitation, improves transparency and honesty, and better resists prompt injection.

Koray Kavukcuoglu, Google's chief AI architect, noted in a blog post that Carbon performed more than 10X better than another lab’s large frontier model in generating high-quality patches.

Carbon also cuts unsupported claims by approximately 2x over Argon, according to internal testing.

What this adds

Background

What's confirmed

• Google's latest AI model, Gemini 4 Carbon, has been benchmarked by SemiAnalysis, a research firm, in a new frontier-level benchmark called GDPx.
• The benchmark includes 206 tasks across various domains, including finance, materials science, cybersecurity, and software engineering.
• In cybersecurity, Carbon achieved a 0.0% attack success rate, compared to 0.7% for Argon and 1.0% for Claude Opus 5.5.
• On the Artificial Analysis Intelligence Index v5.0, Carbon ranks first, followed by Claude Opus 5.5 and GPT-6 Astra.
• Gemini 4 Carbon also leads in Terminal-Bench 4.0 (AA) with a score of 65.2%, ahead of GPT-6 Astra (59.1%), Opus 5.5 (59.6%), Argon (57.1%), and Fable 5.1 (52.0%).
• However, on FrontierSWE 2, GPT-6 Astra takes the top spot with 65.5%, ahead of Carbon’s 61.2%, Opus 5.5’s 62.3%, Argon’s 55.0%, and Fable 5.1’s 56.3%.
• Google announced Carbon on October 28, highlighting its significant gains in coding, agentic tasks, and cybersecurity.
• Carbon is available to a subset of cyber defenders at $2 per million input tokens and $10 per million output tokens.
• Internal tests show Carbon outperforming Argon, GPT-6 Astra, and Claude Opus 5.5 on benchmarks for coding, knowledge work, science, math, multimodal understanding, and cybersecurity.
• Google says Carbon reduces shortcuts and loophole exploitation, improves transparency and honesty, and better resists prompt injection.
• Koray Kavukcuoglu, Google's chief AI architect, noted in a blog post that Carbon performed more than 10X better than another lab’s large frontier model in generating high-quality patches.
• Carbon also cuts unsupported claims by approximately 2x over Argon, according to internal testing.

What's still developing

• Gemini also beat the next model on the list, GPT-6 Astra, which scored lower while costing more than both at about $54 per task.
• Where Claude’s cost came out to around $51 per task, Gemini was just over $16 per task.
• Google’s new frontier model, Gemini 4 Carbon, brings some welcome improvements and while it doesn’t top the charts in a new benchmark, it does provide much more bang for your buck.
• The company says Carbon reflects the “pace at which we’re pushing the frontier forward,” coming with a “significant leap” in cyberdefense that makes it better at finding vulnerabilities and generating secure code.
• Announcing the model, Pichai wrote that Carbon “ranks #1 on the recently updated v5.0 of Artificial Analysis’ Intelligence Index at 59,” and highlighted its cybersecurity results — 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching.
• It used the same staged approach for Gemini 4 Argon, launched two weeks earlier.
• It said Carbon outperformed competing models by wide margins in cybersecurity.
• The LLM is rolling out just two weeks after the search introduced its previous flagship LLM, Gemini 4 Argon.
• According to the company, a number of internal teams that use Carbon in “live coding environments” reported a nearly 80% drop in misalignment alerts.

Sources

• 9To5Googlelink
• The Vergelink
• Melink
• Officechailink
• Dataconomylink
• Siliconanglelink
• WarpBeat — background on Google Launches Gemini 4 Carbon with Enhanced Cybersecurity Capabilities link
• WarpBeat — background on Google Launches Gemini 4 Carbon With Enhanced Cybersecurity and Coding Capabilities link
• Tech Rewired — video link