Cognition SWE-2: Benchmarks, Cost, Kimi K3 Base and Devin Availability
Cognition SWE-2 is one of the more interesting coding-model launches this month, but not because it wins every benchmark. The bigger story is that Cognition is trying to make agentic coding cheaper by teaching a model to care about cost during reinforcement learning.
Released September 10, 2026, SWE-2 is Cognition’s newest software-engineering model for Devin. Cognition reports 50.0% on FrontierCode 1.1 Main, 92.8% on Terminal-Bench 2.1 and 73.0% on DeepSWE 1.1. Yet the harder Terminal-Bench 4 result is only 27.3%. That mixed profile is why the model deserves a careful look rather than another winner-takes-all headline.
Cognition SWE-2 at a glance
| Detail | SWE-2 |
|---|---|
| Developer | Cognition |
| Release | September 10, 2026 |
| Base model | Kimi K3, 2.8T parameters |
| Use case | Agentic software engineering |
| Effort levels | Medium, high, max |
| Access | Devin Desktop and CLI; Web and Fusion rolling out |
| Standalone API | Not announced |
According to Cognition’s official launch, SWE-2 is its first reinforcement-learning run at multi-trillion-parameter scale and is post-trained from Moonshot AI’s Kimi K3.
Cognition SWE-2 benchmarks compared
| Benchmark | SWE-2 | Fable 5.1 | GPT-5.6 Sol | GPT-6 Astra |
|---|---|---|---|---|
| FrontierCode 1.1 Main | 50.0% | 50.9% | 47.5% | 53.3% |
| DeepSWE 1.1 | 73.0% | 67.4% | 72.7% | 74.1% |
| Terminal-Bench 2.1 | 92.8% | 91.4% | 88.8% | 89.9% |
| Terminal-Bench 4 | 27.3% | 55.8% | 37.3% | 57.9% |
SWE-2 is competitive on three of Cognition’s four headline rows. It leads the listed models on Terminal-Bench 2.1 and comes close to Astra on DeepSWE. But Terminal-Bench 4 changes the picture sharply. SWE-2’s 27.3% is less than half Astra’s 57.9% and substantially below Fable 5.1’s 55.8%.
That is not a minor footnote. It suggests SWE-2 may be excellent for some coding-agent workloads without being the best choice for the hardest long-horizon terminal tasks. Cognition also notes that its table combines public results with internal evaluations where public numbers are unavailable, using the harness each model was primarily built for. Treat these as vendor-reported comparisons, not a fully independent bake-off.
Why SWE-2 focuses on cost per completed task
Most LLM pricing starts with dollars per million tokens. Coding agents make that metric less useful. An agent may inspect dozens of files, grep a repository, run tests, retry commands and rewrite a patch. Cheap tokens can still produce an expensive result if the agent wanders.
Cognition instead emphasizes the cost-performance Pareto frontier. On FrontierCode 1.1 Main, it says SWE-2 reaches 50.0% while costing 64% less than Fable 5.1 at the comparison point. It also says the model comes within a few points of GPT-6 Astra at roughly one-quarter of the task cost.
There is a crucial caveat: Cognition has not published a standalone SWE-2 per-token API price. The 64% number is a benchmark rollout-cost comparison, not a public API rate. Buyers should therefore evaluate Devin consumption per accepted task rather than treating SWE-2 like a commodity API model.
How SWE-2 changes coding-agent behavior
The comparison with SWE-1.7 is especially useful. Cognition says SWE-2 medium scores higher on FrontierCode while using 58% fewer turns and costing 81% less on average. Mean steps fall from 127 for SWE-1.7 to 53 for SWE-2 medium. SWE-2 high uses 80 steps and max uses 98.
The model also starts implementing earlier. Cognition reports a median of 18 steps before SWE-2 medium makes its first real edit, compared with 48 for SWE-1.7. Anyone who has watched a coding agent repeatedly read files without changing anything will understand why this matters. Exploration costs money and time even when it never affects the final patch.
Cognition says SWE-2 also shows stronger end-to-end test coverage, more resourceful problem solving and better verification discipline. These are vendor observations, so teams should validate them on their own repositories.
How Cognition trained SWE-2 to care about cost
The central training idea is straightforward. Instead of rewarding a rollout only for success, Cognition subtracts a cost penalty. Its published objective is R = S − λC: success earns reward, while unnecessary cost reduces it. The penalty changes by effort level.
This allows medium, high and max to learn different trade-offs in one reinforcement-learning run. Medium can move quickly on ordinary tasks, while high and max can spend more time planning and verifying when the problem warrants it.
The clever part is that Cognition does not simply punish long answers. It tunes the cost penalty to the local slope of the base model’s Pareto frontier. An excessive penalty could make high effort collapse toward medium and lose hard-task capability. The goal is to shift each effort level toward a better cost-performance point without erasing the reason different effort levels exist.
Why the Kimi K3 base matters
SWE-2 starts from Kimi K3, a 2.8-trillion-parameter model that had already undergone extensive reinforcement learning for agentic coding. Cognition says its additional post-training still adds roughly five to six points on many benchmarks.
This illustrates a broader shift in AI engineering. A company does not necessarily need to pretrain a giant foundation model from zero to create a competitive specialist. A strong base model can become raw material for domain-specific reinforcement learning, better tool use, harness design and cost optimization.
Training infrastructure behind Cognition SWE-2
Cognition says it tripled the number of reinforcement-learning environments, broadened repository diversity, added instruction-following overlays and repeatedly hardened verifiers against reward hacking. Those changes matter because stronger models can discover loopholes in automated grading that weaker models never find.
On the serving side, a prefill delayer batches nearby requests. Cognition reports a 10–20% improvement in both tokens per minute per GPU and tokens per second per request. It also used speculative decoding, then trained an updated draft model as reinforcement learning progressed. A SpecForge-trained draft reportedly produced 15% longer accepted sequences.
Low-precision NVFP4 and FP8 kernels plus quantization-aware training were used to keep rollout memory manageable. The lesson is that coding-model progress is increasingly systems engineering as much as model scale: scheduling, quantization, verifiers and rollout throughput all affect the economics of training and deployment.
Cognition SWE-2 availability and API status
SWE-2 is available in Devin Desktop and Devin CLI, with rollout underway for Devin Web and Fusion. Cognition did not announce a standalone public SWE-2 API or open SWE-2 weights.
That matters for developers searching for SWE-2 API pricing. There is no normal input/output token table to compare directly with general-purpose APIs. The model is currently delivered as part of Cognition’s coding-agent experience, where repository navigation, shell tools, context management and verification are part of the product.
SWE-2 vs GPT-6 Astra and Fable 5.1
If your priority is the hardest long-horizon agentic work, Cognition’s own Terminal-Bench 4 table argues against assuming SWE-2 is the winner. Astra and Fable 5.1 hold large leads. If your team has a high volume of routine and intermediate repository work, however, SWE-2 medium may be more interesting because the model was explicitly optimized around cost-performance trade-offs.
| Workload | What to measure |
|---|---|
| Small fixes and routine PRs | Cost per accepted change, latency, review time |
| Large multi-file changes | Long-horizon reliability and failure recovery |
| Terminal-heavy automation | Tool success plus your actual CI/test environment |
| High-volume agent use | Retries, total consumption, engineer review minutes |
| API-first custom product | SWE-2 is less suitable today because no standalone API was announced |
For a broader frontier comparison, see AI Tech Ledger’s GPT-6 Astra vs Claude Fable 5.1 benchmark analysis.
How to test Cognition SWE-2 on your own repository
Do not choose a coding agent from one leaderboard. Build a small internal evaluation from 10 to 30 real tasks. Mix bug fixes, tests, refactors and messy issues that require several files.
- Freeze the starting commit. Give every model the same repository state.
- Use identical issue descriptions. Do not give one model extra hints.
- Record total consumption. Count failed attempts and retries.
- Measure accepted outcomes. Passing tests is not enough if the patch is hard to maintain.
- Track human review minutes. A cheap agent that creates expensive review work is not truly cheap.
- Separate easy and hard tasks. Learn when medium effort is sufficient and when escalation pays off.
A useful final metric is cost per accepted change. It captures capability and economics better than token price alone.
What the SWE-2 launch does not prove
Three limitations should stay visible. First, the headline benchmark numbers are vendor-reported. Second, SWE-2 does not dominate every evaluation; Terminal-Bench 4 is the clearest counterexample. Third, no standalone API price exists, so the launch cost claims cannot yet be translated into a universal dollars-per-million-tokens comparison.
Those caveats do not make SWE-2 unimportant. Cognition has presented a serious argument that cost-aware reinforcement learning can create coding agents that solve many tasks with fewer steps. Independent repository testing will determine how broadly that advantage transfers.
FAQ about Cognition SWE-2
What is Cognition SWE-2?
SWE-2 is Cognition’s software-engineering model released September 10, 2026. It is post-trained from Kimi K3 and optimized for agentic coding inside Devin.
Is SWE-2 better than GPT-6 Astra?
Not across the board. Cognition reports 50.0% for SWE-2 versus 53.3% for Astra on FrontierCode 1.1 Main, and 27.3% versus 57.9% on Terminal-Bench 4. SWE-2’s strongest argument is efficiency, not universal benchmark leadership.
Does Cognition SWE-2 have an API?
Cognition’s launch did not introduce a standalone public SWE-2 API. The model is available through Devin Desktop and CLI, with Web and Fusion rolling out.
Is SWE-2 open source?
Cognition has not announced open SWE-2 weights. Its base model, Kimi K3, is open-weight, but SWE-2 is a separately post-trained model delivered through Devin.
What is the main SWE-2 benchmark result?
The headline result is 50.0% on FrontierCode 1.1 Main, within one percentage point of Fable 5.1. Cognition also reports 92.8% on Terminal-Bench 2.1 and 73.0% on DeepSWE 1.1.
Bottom line
Cognition SWE-2 is best understood as an efficiency-focused coding model, not a new king of every benchmark. The 58% reduction in turns versus SWE-1.7 and reported 81% lower average task cost are especially relevant for teams running coding agents at scale.
But the 27.3% Terminal-Bench 4 result is a useful warning against oversimplifying the launch. For difficult long-horizon tasks, frontier models still show meaningful headroom. Teams should route work by difficulty, measure cost per accepted change and judge SWE-2 on their own repositories rather than on launch-day charts alone.
Sources: Cognition SWE-2 launch and methodology; Cognition Research; BenchLM SWE-2 evidence snapshot.
