Deepseek.ai is an independent website and is not affiliated with, sponsored by, or endorsed by Hangzhou DeepSeek Artificial Intelligence Co., Ltd.

    AI Architecture
    DeepSeek V4
    1.6T Parameters
    1M Context
    Open Source

    DeepSeek V4 Unveiled: Breaking the AI Efficiency Barrier with 1.6 Trillion Parameters

    May 1, 2026 · Independent DeepSeek Resource Hub

    The AI landscape is witnessing a massive paradigm shift. While the industry grapples with the immense computational power required for ultra-long contexts, DeepSeek-AI has introduced the V4 series — a Mixture-of-Experts model with 1.6 trillion parameters and native support for 1 million tokens, setting a new benchmark for open-source intelligence.

    TL;DR — DeepSeek V4 in 60 seconds

    • Two variants: V4-Pro (1.6T total / 49B active) and V4-Flash (284B / 13B active).

    • Native 1M-token context via Hybrid Attention (CSA + HCA + SWA).

    • Efficiency: ~27% of V3.2's FLOPs and ~10% of its KV cache at 1M tokens.

    • Stability: Muon optimizer replaces AdamW; mHC stabilises deep residual stacks.

    • Benchmarks: SOTA SimpleQA, perfect Putnam-2025, beats Gemini 3.1 Pro on MRCR 1M.

    • Open: Released on Hugging Face with open kernels (MegaMoE, TileLang).

    1. Architecture of a Giant: 1.6T Parameters, Intelligently Distributed

    DeepSeek-V4-Pro is a technical masterpiece. Although the model boasts a staggering 1.6 trillion (1.6T) parameters, it only activates 49 billion (49B) per token — making it not only more powerful but also unprecedentedly efficient.

    DeepSeek-V4-Pro

    1.6T total · 49B activated per token. Frontier-class reasoning rivalling proprietary models.

    DeepSeek-V4-Flash

    284B total · 13B activated. Ultra-fast and cheap — built for agents, coding assistants and real-time apps.

    The series retains the proven DeepSeekMoE framework and the Multi-Token Prediction (MTP) strategy from V3, while introducing entirely new attention and optimisation machinery on top.

    2. The Secret to 1 Million Tokens: Hybrid Attention

    The most significant innovation in V4 is its handling of ultra-long sequences. Traditional attention scales quadratically with sequence length — a hard wall at 1M tokens. DeepSeek breaks through with a Hybrid Attention architecture combining three complementary branches:

    CSA

    Compressed Sparse Attention — compresses the KV cache for every m tokens into one entry, then applies sparse attention on top.

    HCA

    Heavily Compressed Attention — even more aggressive compression for global context, with dense attention preserved.

    SWA

    Sliding Window Attention — supplementary branch that maintains razor-sharp local dependencies.
    The result? In a 1-million-token scenario, V4-Pro requires only 27% of the inference FLOPs and a mere 10% of the KV cache compared to DeepSeek V3.2.

    Want a deeper dive into the compressed-attention internals? Read DeepSeek V4 Compressed Attention: How the KV-Cache Shrinks to Just 2%.

    3. Training Stability: Muon and mHC

    Training a model of this scale often leads to numerical instability. DeepSeek V4 introduces two critical stabilisers:

    • Muon Optimizer — replaces AdamW for the majority of modules, delivering faster convergence and improved training stability at trillion-parameter scale.
    • Manifold-Constrained Hyper-Connections (mHC) — an upgrade to conventional residual connections that stabilises signal propagation across the model's many layers. Read more in our deep dive on mHC explained.
    Did you know? To prevent "loss spikes" (sudden training errors), DeepSeek uses Anticipatory Routing, where MoE routing indices are computed using historical network parameters rather than current ones.

    4. Benchmarks: Open-Source Redefines the State-of-the-Art

    The performance of DeepSeek-V4-Pro-Max (the maximum reasoning-effort mode) rivals top-tier proprietary models like GPT-5.4 and Gemini 3.1 Pro.

    CategoryBenchmarkV4-Pro-Max Score
    KnowledgeSimpleQA Verified57.9% (SOTA, open models)
    MathematicsPutnam-2025120 / 120 (proof-perfect)
    CodeLiveCodeBench57.9%
    Long ContextMRCR 1M (Retrieval)90.2% (beats Gemini 3.1 Pro)

    5. Dedicated Agent Optimisations

    DeepSeek V4 is not just a bigger model — it was explicitly tuned for agentic workflows. According to the official release notes, V4 ships as open-source SOTA on agentic coding benchmarks and is already powering DeepSeek's in-house agent stack.

    • Seamless integration with leading agent frameworks including Claude Code, OpenClaw and OpenCode.
    • V4-Flash matches V4-Pro on simple agent tasks at a fraction of the cost.
    • Both models support dual-mode inference (Thinking / Non-Thinking) on a 1M-token context — perfect for long-running tool-use loops.

    6. API Availability & Migration

    The DeepSeek V4 API is live today. Per the official announcement, existing customers can keep their base_url and only need to update the model name to deepseek-v4-pro or deepseek-v4-flash.

    • Compatible with both OpenAI ChatCompletions and Anthropic-style APIs.
    • Both models expose 1M context and dual Thinking / Non-Thinking modes.
    • Try interactively at chat.deepseek.com via Expert Mode (V4-Pro) or Instant Mode (V4-Flash).
    ⚠️ Deprecation notice: The legacy deepseek-chat and deepseek-reasoner models will be fully retired and inaccessible after July 24, 2026, 15:59 UTC. Until then they transparently route to deepseek-v4-flash (non-thinking and thinking respectively). Migrate now to avoid breakage. See our DeepSeek API guide for request examples.

    Source: official DeepSeek V4 Preview release notes · Tech report: DeepSeek_V4.pdf on Hugging Face.

    7. Why This Changes the AI Market

    DeepSeek V4 proves that we are far from the limits of test-time scaling. The massive efficiency gains in context processing mean complex agentic workflows and large cross-document analyses can now be performed routinely at 1 million tokens.

    Furthermore, DeepSeek is committed to reproducibility: high-performance kernels such as MegaMoE and the TileLang libraries are open-sourced, helping the entire AI community build more efficiently. For a hands-on look at the lighter variant, see our DeepSeek V4 Flash review.

    Frequently Asked Questions

    DeepSeek-V4-Pro has 1.6 trillion total parameters with 49 billion activated per token. The lighter DeepSeek-V4-Flash has 284 billion total parameters with only 13 billion activated per token, thanks to the DeepSeekMoE Mixture-of-Experts architecture.

    In a 1-million-token context, DeepSeek V4-Pro requires only ~27% of the inference FLOPs and ~10% of the KV cache compared to DeepSeek V3.2. That is roughly a 73% reduction in compute per token and a 90% reduction in KV-cache memory at long context.

    V4-Pro (1.6T total / 49B active) targets maximum reasoning quality and rivals proprietary frontier models. V4-Flash (284B total / 13B active) is optimised for speed and cost, ideal for high-throughput agents, coding assistants, and real-time applications.

    DeepSeek V4 uses a novel attention design combining token-wise compression with DeepSeek Sparse Attention (DSA). In practice it stacks Compressed Sparse Attention (CSA), Heavily Compressed Attention (HCA) and Sliding Window Attention (SWA) to make 1M-token context tractable.

    Muon is a new optimizer that replaces AdamW for most modules in V4, delivering faster convergence and better stability at trillion-parameter scale. mHC (Manifold-Constrained Hyper-Connections) is an upgrade to residual connections that stabilises signal propagation across very deep networks.

    DeepSeek V4 is live on chat.deepseek.com via Expert Mode (V4-Pro) and Instant Mode (V4-Flash). Developers can call it through the official API by setting the model to deepseek-v4-pro or deepseek-v4-flash — the base_url stays the same. Both OpenAI ChatCompletions and Anthropic-compatible APIs are supported.

    DeepSeek announced that the legacy deepseek-chat and deepseek-reasoner models will be fully retired and inaccessible after July 24, 2026, 15:59 UTC. Until then, those endpoints route to deepseek-v4-flash (non-thinking and thinking respectively).

    Yes. Both V4-Pro and V4-Flash expose dual modes — Thinking and Non-Thinking — across a 1M-token context window, configurable per request via the standard DeepSeek API thinking-mode parameter.

    DeepSeek-V4 model checkpoints are released through the official DeepSeek-AI collection on Hugging Face under an open license, alongside open-sourced kernels such as MegaMoE and the TileLang libraries. The full technical report is published as DeepSeek_V4.pdf on the V4-Pro repository.

    On open benchmarks DeepSeek V4-Pro-Max posts 57.9% on SimpleQA Verified (SOTA for open models), a perfect 120/120 on Putnam-2025, 57.9% on LiveCodeBench, and 90.2% on MRCR 1M retrieval — beating Gemini 3.1 Pro on long-context retrieval. DeepSeek also reports open-source SOTA on agentic coding benchmarks.

    This blog post is based on the official DeepSeek-V4 technical preview (May 2026). Independent Resource Hub — not affiliated with DeepSeek.com.

    Related Reading