DEEPSEEK / UNIFIED AR MULTIMODAL / 2025

Janus-Pro-7B

TOTAL≈7Bparameters

ACTIVE≈7Bper token

DEPTH30 + encodersrepeated blocks

HIDDEN4,096main trunk

CONTEXT16Knative tokens

CHECKPOINT≈14.5GB2 shards · MIT

INTERACTIVE SYSTEM BLUEPRINT

MODEL MRI

15,384 INPUT + 1,000 OUTPUT · 295.35T arithmetic FLOPs · click any module for evidence

VERIFIED ADAPTER

INPUT15.4KIMAGE ⇄ TEXT

TRANSFORMER STACKDECOUPLED ENCODERS + 30Ltwo visual paths → shared autoregressive transformer

Shared LLM Residual

OUTPUT1.0KTOKENS

wraps every repeated block

CAUSAL SELF-ATTENTION · DECOUPLED VISUAL ENCODING理解需要语义表示，生成需要离散细节；分开编码缓解表示冲突。

Shared 7B Transformer共享主干保留理解与生成之间的知识迁移。

TRUNK30L4,096

ENCODERS2decoupled

UNDERSTANDSigLIPvision

GENERATEVQtokens

CONTEXT16Ktokens

PARAMS≈7Bsystem