LLaVA / PROJECTOR VLM / 2023

LLaVA 1.5 · 7B

TOTAL≈7.1Bparameters

ACTIVE≈7.1Bper token

DEPTH24 + 32repeated blocks

HIDDEN1,024→4,096main trunk

CONTEXT4Knative tokens

CHECKPOINT≈13.5GB2 shards · Llama + Apache code

INTERACTIVE SYSTEM BLUEPRINT

MODEL MRI

3,096 INPUT + 1,000 OUTPUT · 62.56T arithmetic FLOPs · click any module for evidence

VERIFIED ADAPTER

INPUT3.1KIMAGE + TEXT → TEXT

TRANSFORMER STACK24L VIT + 32L LLMfrozen vision tower → MLP projector → causal LLM

Separate Residual Stacks

OUTPUT1.0KTOKENS

wraps every repeated block

VISION + CAUSAL ATTENTION · TWO-LAYER MLP PROJECTOR关键创新不是复杂 fusion，而是用很小的可训练桥接层对齐两个冻结/预训练空间。

Vicuna-7B Backbone语言主干承担跨模态推理与回答生成。

VISION24LViT-L/14

PROJECTOR2LMLP

LANGUAGE32LVicuna-7B

IMAGE336²pixels

CONTEXT4Ktokens

PARAMS≈7.1Bsystem