Vision-Language-Action Models

CAC-VLA: Context-Gated Action Conditioning

A VLM-native latent-action interface that gives continuous action experts structured guidance while learning when and how strongly to use it.

Yifu Xiong, Wenhao Yu, Jiaxuan Lin, Bojun Zou, Jiahao Li, Lu Zhang, Yanyong Zhang, Jianmin Ji

arXiv:2607.04816 [cs.RO]

Explore the method Paper PDF arXiv Code pending release

98.3%

LIBERO average success

89.5%

LIBERO-Plus SFT

+48 pp

Real-world Full SR

01 / Abstract

A lightweight latent-action interface inside the VLM.

CAC-VLA trains dedicated VLM query tokens to predict coarse-to-fine latent actions encoded from future action segments. These latent actions are not executed as trajectories; instead, they provide action-structured guidance to the original continuous action expert.

A context gate calibrates how much retrieved latent-action information enters each expert layer, allowing the policy to use action guidance adaptively across task phases and generation states.

Contribution 1 Latent-action conditioning

An intermediate interface between VLM representations and continuous robot control.

Contribution 2 Context-gated expert guidance

Cross-attention retrieves action information while a gate controls residual injection.

Contribution 3 Benchmark validation

LIBERO, LIBERO-Plus, ablations, and initial real-world tabletop experiments.

02 / Motivation

The semantic-to-control gap is an interface problem.

Standard VLA systems ask the action expert to recover precise motor structure from representations optimized primarily for visual-language understanding. CAC-VLA makes that interface explicitly action-structured and adaptive.

03 / Method

Latent actions are guidance, not commands.

CAC-VLA predicts compact, coarse-to-fine representations of future action segments inside the VLM. The original action expert remains responsible for producing executable continuous action chunks.

CAC-VLA pipeline with OAT latent-action alignment and context-gated action expert conditioning — CAC-VLA predicts OAT-aligned latent actions inside the VLM and injects them into the continuous action expert through context-gated conditioning.

Expert-aware attention mask for image, prompt, state, query, and expert tokens — **Expert-aware mask.** Expert tokens cannot directly attend to latent query tokens, preventing an unstructured shortcut.

Context gate computed from action hidden states and cross-attention output — **Context gate.** The current expert state and retrieved latent-action update jointly determine the injection strength.

Predict

Dedicated VLM query tokens predict OAT-aligned latent actions from the current image and language instruction.

Retrieve

Expert hidden states use cross-attention to retrieve action information relevant to the current layer and generation state.

Calibrate

A channel-wise context gate controls the residual contribution of the retrieved latent-action update.

Training target Frozen OAT tokenizer

Encodes future action segments into raw latent-action targets for alignment and conditioning.

Deployment path VLM-predicted latents only

The OAT tokenizer is removed at inference; the expert uses latent actions predicted by VLM query tokens.

Main settings N_q = 8, D_z = 4

Expert horizon H_e = 10; latent horizon H_l = 20 for LIBERO and 10 for LIBERO-Plus.

Core update x⁺_l = x_l + g_l ⊙ u_l

Cross-attention decides what is relevant. The gate decides how much enters the expert.

04 / Adaptive Conditioning

Why a gate?

Predicted latent actions can be incomplete, noisy, or mismatched with the immediate control requirement. Their usefulness also changes across task phases and flow steps, so fixed-strength fusion is unnecessarily rigid.

Observation The learned gate varies across flow steps and task suites.

Average context-gate value across ten flow steps for four LIBERO suites — Gate values averaged over layers, episodes, and hidden channels.

05 / Simulation

Strong performance across standard and shifted environments.

CAC-VLA is evaluated on LIBERO and LIBERO-Plus, including distribution shifts in viewpoint, embodiment, language, lighting, background, noise, and layout.

LIBERO

98.3% Average success rate, ranked #2

Spatial: 98.4%
Object: 99.8%
Goal: 99.6%
Long: 95.4%

LIBERO-Plus

89.5% Supervised fine-tuning

Zero-shot: 83.8%
Lighting: 97.5%
Background: 97.1%

Design ablation

98.3% Full CAC-VLA

No latent condition: 98.0%
No context gate: 97.9%
Best horizon: H_l = 20

Standard benchmark

LIBERO average

Success rate (%)

ACoT-VLA

98.5

CAC-VLA
98.3

VLA-Adapter

97.3

π0.5

96.9

CAC-VLA ranks second overall and achieves the best Object and Goal suite scores.

Distribution shifts

LIBERO-Plus SFT

Success rate (%)

CAC-VLA
89.5

ACoT-VLA

88.0

π0.5

85.7

The strongest average result under supervised fine-tuning across seven shifts.

Direct transfer

LIBERO-Plus zero-shot

Success rate (%)

CAC-VLA
83.8

π0.5

81.5

OpenVLA-OFT

69.6

Trained on LIBERO and evaluated directly on LIBERO-Plus without shift-specific fine-tuning.

Suite breakdown

LIBERO comparison

Method	Spatial	Object	Goal	Long	Avg.
π0.5	98.8	98.2	98.0	92.4	96.9
ACoT-VLA	98.6	99.0	99.4	97.0	98.5
CAC-VLA	98.4	99.8	99.6	95.4	98.3

Shift breakdown

CAC-VLA on LIBERO-Plus

Supervised fine-tuning

Camera91.2

Robot78.4

Language83.3

Lighting97.5

Background97.1

Noise95.4

Layout87.8

Design validation

Ablation studies

Moderate future context and adaptive injection provide the best average result.

Latent-action horizon

Setting	Spatial	Object	Goal	Long	Avg.
H_l = 10	98.8	99.8	99.6	94.2	98.1
H_l = 20	98.4	99.8	99.6	95.4	98.3
H_l = 30	98.0	99.8	97.2	97.2	98.1

Conditioning mechanism

Variant	Spatial	Object	Goal	Long	Avg.
Without latent action	97.6	99.8	99.0	95.6	98.0
Without context gate	98.8	99.8	98.8	94.0	97.9
Full CAC-VLA	98.4	99.8	99.6	95.4	98.3

06 / Real World

Initial validation on tabletop pick-and-place.

CAC-VLA and the π0.5 baseline are fine-tuned on the same 50 demonstrations and evaluated over 25 independent trials using a UR7e and two RealSense D435i cameras.

Annotated UR7e tabletop setup with RealSense D435i cameras and the target object — Annotated real-world setup with the wrist and third-person RealSense D435i cameras.

Task Score 48% to 72% +24 percentage points

Full Success Rate 16% to 64% +48 percentage points

Scope. The real-world study is intentionally initial: one task, one robot embodiment, and a limited demonstration set. Broader tasks and task-adaptive latent horizons remain future work.

07 / Takeaway

CAC-VLA makes the VLM-to-expert interface action-structured and adaptive.

VLM-native latent-action prediction without a standalone executable action generator.

Expert-preserving design that leaves continuous robot control to the original action expert.

Context-gated residual modulation that calibrates guidance instead of trusting it uniformly.

08 / Citation

BibTeX

Use the arXiv identifier for the current public version.

@misc{xiong2026cacvla,
  title     = {CAC-VLA: Context-Gated Action Conditioning for
               Vision-Language-Action Models},
  author    = {Xiong, Yifu and Yu, Wenhao and Lin, Jiaxuan and
               Zou, Bojun and Li, Jiahao and Zhang, Lu and
               Zhang, Yanyong and Ji, Jianmin},
  year      = {2026},
  eprint    = {2607.04816},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO},
  url       = {https://arxiv.org/abs/2607.04816}
}