STEPFUN / EFFICIENT VLM / 2026

Step-3.7-Flash

TOTAL201Bparameters

ACTIVE≈11Bper token

DEPTH45 + ViT47repeated blocks

HIDDEN4,096main trunk

CONTEXT256Knative tokens

CHECKPOINT402.7GB26 shards · Apache-2.0

INTERACTIVE SYSTEM BLUEPRINT

MODEL MRI

100K INPUT + 1K OUTPUT · 4.28P arithmetic FLOPs · click any module for evidence

VERIFIED ADAPTER

INPUT100.0KIMAGE · TEXT

TRANSFORMER STACK45 MAIN + 3 MTP LAYERS3 sliding : 1 full · MoE from L3 · 288 experts

pre-norm additive residual · head-wise attention gate

OUTPUT1.0KTOKENS

wraps every repeated block

SWA ⇄ FULL ATTENTION · HYBRID LOCAL / GLOBAL多数层用局部窗口控制成本，每四层的 full attention 恢复跨窗口信息。

Step MoE · 201B CAPACITY≈11B active path · total capacity ≠ active compute

TEXT45Lmain trunk

VISION47L1,536 dim

ATTN3 : 1SWA / full

EXPERTS288top-8 + shared

MTP3layers

CONTEXT256Ktokens