LLMWatch

51

AA v4.1
open leader

86

BenchLM
open anchor

$0.04

Cost-per-Task
cheapest open

1M

Baseline
Context

Core Signal This Week

Three forces converged this week. The closed frontier was capped by government action (Fable 5 offline). The open frontier surged upward (GLM-5.2 at AA 51). And the methodology shifted toward agentic evaluation (AA v4.1). For the first time, the best open and closed models are within 5 composite points.

Kimi K2.7 Code posts +21.8% on Kimi Code Bench v2 over K2.6. It beats Opus 4.8 on tool invocation (MCP Mark Verified 81.1 vs 76.4). All benchmarks are vendor-reported with zero independent confirmation.

GLM-5.2 is the new open-weight leader at AA v4.1: 51. It also took #1 open-weight agentic on WhatLLM (75.9). MIT license and 1M context make it a strong long-context coding alternative.

Ant Group's InclusionAI shipped three trillion-scale checkpoints under MIT. The accompanying arXiv paper focuses on instant agentic intelligence at trillion-parameter scale.

Olmo Hybrid pairs gated DeltaNet with attention in a 3:1 pattern. It matches Olmo 3 accuracy with 49% fewer training tokens. Weights, data, code, and checkpoints are all released.

NVIDIA's first diffusion language model family. It supports AR, diffusion, and self-speculation modes. Training code is released via NVIDIA Megatron Bridge.

DiffusionGemma hits 1,000+ tok/s on H100 via non-autoregressive diffusion. It is experimental and lower quality than Gemma 4, but up to 4× faster. Useful for code infilling, math graphs, and Sudoku-style tasks.

Fable 5 forced offline

The US Commerce Department ordered Anthropic to disable Claude Fable 5 / Mythos 5 on June 12. It was the first-ever government takedown of a deployed frontier model. The effective closed leader is now Opus 4.8 (AA v4.1: 56).

GLM-5.2 takes open crown

GLM-5.2 (Z.ai, MIT, 1M context) scored 51 on AA v4.1 — the open-weight leader by 7 points. It launched one day after the Fable 5 shutdown, positioned as a long-context coding alternative.

AA v4.1 agentic shift

Artificial Analysis reweighted its index toward agentic workloads. The new Cost-per-Task metric quantifies the open-value story: DeepSeek V4 Pro at $0.04/task vs. Opus 4.8 at $1.78/task.

Trillion-scale open flood

Three open releases now operate at or near 1T parameters: Kimi K2.7 Code, InclusionAI Ling/Ring 2.6, and DeepSeek V4 Pro. Combined with GLM-5.2 and MiniMax M3, the open frontier is in the trillion-parameter regime.

MIT license strengthens

GLM-5.2, InclusionAI Ling/Ring 2.6, Olmo Hybrid, DeepSeek V4 Pro/Flash, and MiMo-V2.5-Pro all ship under fully permissive MIT. This shifts the frontier toward unrestricted commercial use.

Open infra trust erodes

Google will replace its Apache-2.0 Gemini CLI with a closed-source Antigravity binary on June 18. Separately, Meta's Alexandr Wang said the old open-source playbook "didn't work." Muse Spark stays closed.

DeepSeek V4 value dominance

DeepSeek V4 Pro combines the highest independently confirmed open composite (BenchLM 86) with the lowest Cost-per-Task ($0.04). It is the standout value story even though GLM-5.2 now leads the AA composite.

Hybrid & diffusion emerge

Olmo Hybrid pairs gated DeltaNet with attention for 2× data efficiency. NVIDIA's Nemotron-Labs Diffusion family and Google's DiffusionGemma bring non-autoregressive text generation into the open ecosystem.

Frontier Coding (API)

Main: GLM-5.2

Runner-up: DeepSeek V4 Pro

GLM-5.2 leads open-weights on AA v4.1 (51) and WhatLLM Agentic (75.9), with MIT license, 1M context, and $1.40/$4.40 per M. DeepSeek V4 Pro is the strongest independently confirmed coder (BenchLM Coding 75.9, SWE-Verified ~88%).

Frontier Coding (self-hosted)

Main: DeepSeek V4 Pro

Runner-up: GLM-5.2

DeepSeek V4 Pro is the safest benchmark anchor under MIT (BenchLM 86, SWE-Verified ~88%). GLM-5.2 adds 1M context and AA v4.1 leadership but requires ~744B parameters to host.

Cost-Efficient Production

Main: DeepSeek V4 Flash

Runner-up: MiMo-V2.5

DeepSeek V4 Flash offers $0.14/$0.28 per M with BenchLM 74 and 1M context — the best value-to-quality ratio. MiMo-V2.5 is cheaper ($0.18/M blended) with 210 tok/s and 1M context.

Single-GPU Coding

Main: Devstral Small 2

Runner-up: Qwen 4 Coder 32B-A3B

Devstral Small 2 (24B dense, Apache 2.0) is purpose-built for consumer-hardware coding at 68.0% SWE-bench Verified. Qwen 4 Coder (32B/3B active, Apache 2.0) hits 82% SWE-Verified as the best open-source Mac coder.

Edge / Small Devices

Main: LFM2.5-8B-A1B

Runner-up: Gemma 4.5 12B

LFM2.5-8B-A1B (8.3B/1.5B active) is explicitly designed for edge deployment (CPU/NPU/GPU) with τ²-Telecom 88.07 and IFEval 91.84. Gemma 4.5 12B (Apache 2.0, 1M context) is the best multimodal small-model alternative.

RAG / Routing / Sub-Agents

Main: Mellum2

Runner-up: Command A+

Mellum2 (12B/2.5B active, Apache 2.0) is explicitly optimized for routing and sub-agent workflows with low latency. Command A+ adds enterprise-grade multilingual and multimodal support (Apache 2.0).

Computer Use Agents

Main: GLM-5.2

Runner-up: Kimi K2.6

GLM-5.2 is the #1 open-weight agentic model (WhatLLM Agentic 75.9, GDPval-AA 1524) with MIT license and 1M context. Kimi K2.6 adds 300 parallel sub-agents and BrowseComp 86.3% under Modified MIT.

Long Context (10M+)

Main: Llama 4 Scout

Runner-up: GLM-5.2 / DeepSeek V4 Pro

Llama 4 Scout is the open-weight long-context landmark at 10M tokens. For 1M-context use cases, GLM-5.2, DeepSeek V4 Pro, and Nemotron 3 Ultra all offer it.

Reasoning / Math

Main: GLM-5.2

Runner-up: Nemotron 3 Ultra

GLM-5.2 (AA v4.1: 51, MIT) is the strongest open composite including reasoning. Nemotron 3 Ultra (GPQA 87.0%, IOI 2025 570.0, OpenMDW-1.1) is the specialist for math/code competition.

Multilingual

Main: Command A+

Runner-up: GLM-5.2

Command A+ (Apache 2.0) emphasizes multilingual enterprise use. GLM-5.2 (MIT) adds strong Chinese-English bilingual capabilities from Z.ai.

Physical AI / Robotics

Main: Cosmos 3

Runner-up: Qwen-Robot Suite

Cosmos 3 (NVIDIA, OpenMDW-1.1) is an omnimodal foundation model covering text/images/video/sound/actions. Qwen-Robot Suite (Alibaba) adds VLA, VLN, and video world model variants — pilot testing underway.

Speed-Optimized Generation

Main: DiffusionGemma

Runner-up: MiMo-V2.5-Pro-UltraSpeed

DiffusionGemma achieves 1,000+ tok/s on H100 via non-autoregressive diffusion under Apache 2.0 — but is experimental and lower quality than Gemma 4. MiMo UltraSpeed offers 1,000+ tok/s on a 1T-param model.

Fable 5 Restoration Timeline (CRITICAL)

Anthropic engineers went to Washington on June 16. No restoration date announced. If it returns, the closed ceiling jumps back to AA v4.1: 60.

GLM-5.2 Independent Benchmark Confirmation

SWE-bench / Terminal-Bench / GPQA scores on independent leaderboards are still TBD. Confirmation would solidify GLM-5.2 as the definitive open leader.

Kimi K2.7 Code Independent Benchmarks

Every Kimi K2.7 Code benchmark is vendor-reported. Zero independent SWE-bench / LiveCodeBench / GPQA scores exist yet.

MiniMax M3 Independent Convergence

M3 vendor scores (SWE-Bench Pro 59%, GPQA 93%) still sit far above its BenchLM independent rank (#23/124). The starkest vendor-vs-independent gap.

Nex-N2 Pro License (5th Cycle)

Four consecutive sweeps found no LICENSE file. If confirmed Apache 2.0, it becomes a permissive agentic coding leader.

Gemini CLI → Antigravity Closure (Jun 18)

Free Gemini CLI access ends June 18. Watch for community forks and whether other labs follow "open code, closed infra."

Meta's Two-Tier Open/Closed Strategy

Wang said the old playbook "didn't work." Watch for Muse Spark's launch date and whether Llama 6 keeps open weights.

Microsoft FastContext License & Evaluation

FastContext-1.0-4B-SFT claims up to 60% token reduction for coding agents. License terms and independent benchmarks are pending.

AA v4.1 Methodology Settling

First cycle under the v4.1 agentic reweighting. Watch whether scores stabilize and whether Cost-per-Task gains adoption.

DeepSeek V4 Parameter Clarification

HF "862B"/"158B" labels are on-disk GB sizes (FP4+FP8), not parameter counts. A direct config.json read would settle it definitively.