DEEPSEEK / FOUNDATION MODELS / 2026

DeepSeek-V4-Pro

TOTAL1.6Tparameters

ACTIVE49Bper token

DEPTH61+1repeated blocks

HIDDEN7,168main trunk

CONTEXT1Mnative tokens

CHECKPOINT864.7GB64 shards

INTERACTIVE SYSTEM BLUEPRINT

MODEL MRI

100K INPUT + 1K OUTPUT · 11.59P arithmetic FLOPs · click any module for evidence

VERIFIED ADAPTER

INPUT100.0KTOKENS

TRANSFORMER STACK61 MAIN LAYERS31 HCA · 30 CSA · alternating after L2

mHC residual fabric · 4 streams · A/B/C dynamic mixing

OUTPUT1.0KTOKENS

wraps every repeated block

HYBRID ATTENTION SYSTEMall layers also keep SWA 128

SPARSE MoE · 1.55T CAPACITY49B active path · total capacity ≠ active compute

DEPTH61 + 1main + MTP

HIDDEN7,168mHC ×4

QUERY128 × 512partial RoPE

EXPERTS384routed

ACTIVE6 + 1per token

PRECISIONFP4 / FP8BF16 critical