Z.AI / GLM-4 FAMILY / 2026

GLM-4.x Family

TOTAL31.2Bparameters

ACTIVE≈3Bper token

DEPTH47+1repeated blocks

HIDDEN2,048main trunk

CONTEXT202Knative tokens

CHECKPOINT62.44GB48 shards · MIT

FAMILY EVOLUTION

GLM-4.5355B-A32BGLM-4.5-Air106B-A12BGLM-4.7agenticGLM-4.7-Flash30B-A3B

INTERACTIVE SYSTEM BLUEPRINT

MODEL MRI

100K INPUT + 1K OUTPUT · 5.52P arithmetic FLOPs · click any module for evidence

VERIFIED ADAPTER

INPUT100.0KTEXT

TRANSFORMER STACK47 MAIN LAYERS1 dense FFN · 46 sparse MoE · MLA throughout

pre-norm additive residual · compact 2,048-wide trunk

OUTPUT1.0KTOKENS

wraps every repeated block

MULTI-HEAD LATENT ATTENTION · 30B-A3B SCALINGGLM-4.7-Flash 用较小 hidden 和 64-way expert bank，把总容量压到 30B 级、active path 压到约 3B。

GLM Lite MoE · 31.2B CAPACITY≈3B active path · total capacity ≠ active compute

DEPTH47 + 1main + MTP

HIDDEN2,048compact trunk

MLAKV 512Q rank 768

EXPERTS64top-4 + 1

CONTEXT202Ktokens

LICENSEMITcommercial