VLA论文精读目录
VLA Paper Taxonomy
Part 1:Foundation Model / Policy / Action Representation
1. Vision-Language-Action Foundation Models
这一类关注机器人基础模型:
核心问题:
- 如何将视觉语言模型迁移到机器人控制;
- 如何利用大规模数据实现跨任务泛化;
- 如何构建类似 GPT 的 embodied foundation model。
OpenVLA分类
- Vision-Language-Action Foundation Model
- Open-source VLA
- Action Token Prediction
VLA 论文精读(一)OpenVLA: An Open-Source Vision-Language-Action Model
https://arxiv.org/pdf/2406.09246
GR00T N1分类
- Humanoid Robot Foundation Model
- Generalist Robot Policy
- Embodied Foundation Model
VLA 论文精读(二)GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
https://arxiv.org/pdf/2503.14734
π0分类
- Vision-Language-Action Foundation Model
- Flow Matching Policy
- General Robot Control
VLA 论文精读(二十三)π0: A Vision-Language-Action Flow Model for General Robot Control
https://www.pi.website/download/pi0.pdf
π0.5分类
- Open-world Generalization
- VLA Scaling
- Generalist Robot Policy
VLA 论文精读(十八)π0.5: a Vision-Language-Action Model with Open-World Generalization
https://arxiv.org/pdf/2504.16054
Gemini Robotics分类
- Embodied Foundation Model
- Vision-Language-Action
- Large-scale Robot Intelligence
VLA 论文精读(二十五)Gemini Robotics: Bringing AI into the Physical World
https://arxiv.org/pdf/2503.20020
RDT-1B分类
- Diffusion Foundation Model
- Bimanual Manipulation
- Large Robot Policy
VLA 论文精读(二十一)RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
https://arxiv.org/pdf/2410.07864
SmolVLA分类
- Efficient VLA
- Small-scale Foundation Model
- Affordable Robotics
VLA 论文精读(三十一)SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
https://arxiv.org/pdf/2506.01844
VLA-0分类
- VLA Construction
- Zero Modification VLA
- Foundation Model Analysis
VLA 论文精读(三十六)VLA-0: Building State-of-the-Art VLAs with Zero Modification
https://arxiv.org/pdf/2510.13054
2. Visuomotor Policy Learning and Action Generation
这一类关注机器人如何从 observation 生成 action。
主要路线:
- Diffusion Policy
- Flow Matching
- Reactive Policy
Diffusion Policy分类
- Visuomotor Policy
- Diffusion-based Action Generation
- Imitation Learning
VLA 论文精读(三)Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
https://arxiv.org/pdf/2303.04137
Reactive Diffusion Policy分类
- Contact-rich Manipulation
- Visual-Tactile Policy
- Fast-Slow Control
VLA 论文精读(七)Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
https://arxiv.org/pdf/2503.02881
3. Action Representation and Tokenization
核心问题:机器人动作是否可以像语言一样表示为 token。
Universal Actions for Enhanced Embodied Foundation Models分类
- Action Representation
- Universal Action Space
- Embodied Foundation Model
VLA论文精读(十二)Universal Actions for Enhanced Embodied Foundation Models
https://arxiv.org/pdf/2501.10105
Survey on Vision-Language-Action Models: An Action Tokenization Perspective分类
- Survey
- Action Tokenization
- VLA Taxonomy
VLA 论文精读(三十三)A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
https://arxiv.org/pdf/2507.01925
Part 2:Data Scaling / Benchmark / 3D Representation / Dexterous Manipulation / Tactile
4. Robot Data Scaling and Simulation
这一类关注:机器人领域如何获得类似互联网规模的数据。
核心问题:
- 真实机器人数据成本高;
- simulation 是否可以扩展 robot learning;
- 如何建立统一的数据平台和 benchmark。
DemoGen分类
- Synthetic Demonstration Generation
- Data-efficient Robot Learning
- Simulation Data
VLA 论文精读(四)DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning
https://arxiv.org/pdf/2502.16932
ROBOVERSE分类
- Robot Dataset Platform
- Benchmark
- Scalable Robot Learning
VLA论文精读(十一)ROBOVERSE: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
https://roboverseorg.github.io/static/pdfs/roboverse.pdf
RoboTwin 2.0分类
- Simulation Data Generation
- Robot Benchmark
- Domain Randomization
VLA 论文精读(二十八)RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization
https://arxiv.org/pdf/2506.18088
VR-Robo分类
- Real-to-Sim-to-Real
- Simulation Transfer
- Robot Navigation and Locomotion
VLA 论文精读(二十九)VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion
https://arxiv.org/pdf/2502.01536
5. Benchmark and Evaluation
这一类关注:如何判断 VLA 是否真的具备泛化能力。
重点:
- long-horizon tasks
- language-conditioned manipulation
- generalization
VLABench分类
- Manipulation Benchmark
- Long-horizon Reasoning
- Language-conditioned Robotics
VLA 论文精读(五)VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks
https://arxiv.org/pdf/2412.18194
ROBOVERSE分类
- Benchmark Platform
- Dataset Evaluation
VLA论文精读(十一)ROBOVERSE: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
https://roboverseorg.github.io/static/pdfs/roboverse.pdf
RoboTwin 2.0分类
- Simulation Benchmark
- Manipulation Evaluation
VLA 论文精读(二十八)RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization
https://arxiv.org/pdf/2506.18088
6. 3D Representation and Geometric Understanding
这一类关注:VLA 是否需要显式三维世界理解。
核心问题:
RGB image:
↓
3D geometry:
↓
robot manipulation
PointVLA分类
- 3D Vision-Language-Action
- Point Cloud Representation
- Spatial Understanding
VLA 论文精读(十四)PointVLA: Injecting the 3D World into Vision-Language-Action Models
https://arxiv.org/pdf/2503.07511
Lift3D Foundation Policy分类
- 2D-to-3D Transfer
- Foundation Policy
- 3D Manipulation
VLA 论文精读(十五)Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation
https://arxiv.org/pdf/2411.18623
FP3分类
- 3D Foundation Policy
- Robotic Manipulation
VLA 论文精读(十六)FP3: A 3D Foundation Policy for Robotic Manipulation
https://arxiv.org/pdf/2503.08950
7. Affordance Learning
这一类关注:机器人不仅需要识别物体,还需要理解:物体可以如何被操作。
AffordDP分类
- Affordance Representation
- Generalizable Policy
- Diffusion Policy
VLA 论文精读(十七)AffordDP: Generalizable Diffusion Policy with Transferable Affordance
https://arxiv.org/pdf/2412.03142
8. Dexterous Manipulation and Bimanual Control
这一类关注从arm + gripper走向复杂双臂、灵巧手、人类级操作。
ALOHA Unleashed分类
- Bimanual Manipulation
- Low-cost Hardware
- Imitation Learning
VLA 论文精读(二十四)ALOHA Unleashed: A Simple Recipe for Robot Dexterity
https://arxiv.org/pdf/2410.13126
Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware分类
- Bimanual Manipulation
- Low-cost Robot Learning
VLA 论文精读(十九)Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
https://arxiv.org/pdf/2304.13705
ManipTrans分类
- Dexterous Manipulation Transfer
- Residual Learning
- Bimanual Control
VLA 论文精读(九)ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
https://arxiv.org/pdf/2503.21860
DexGraspVLA分类
- Dexterous Grasping
- Vision-Language-Action
- General Dexterous Manipulation
VLA论文精读(十)DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping
https://arxiv.org/pdf/2502.20900
Part 3:Tactile / Reasoning / Agentic Robotics / Fine-tuning / Surveys
9. Tactile Sensing and Contact-Rich Manipulation
这一类关注视觉模型在真实操作中的局限:
- 接触状态不可见;
- 物体形变难预测;
- 力反馈缺失;
- open-loop policy 容易失败。
核心方向:
vision
+
touch / force
+
feedback control
Beyond Sight分类
- Heterogeneous Sensor Fusion
- Vision-Language Policy
- Multi-modal Robot Learning
VLA 论文精读(六)Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
https://arxiv.org/pdf/2501.04693
Reactive Diffusion Policy分类
- Visual-Tactile Policy
- Contact-rich Manipulation
- Reactive Control
VLA 论文精读(七)Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
https://arxiv.org/pdf/2503.02881
3D-ViTac分类
- Vision-Tactile Learning
- Fine-grained Manipulation
- Sensor Fusion
VLA 论文精读(二十)3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
https://arxiv.org/pdf/2410.24091
10. Reasoning, Planning and Agentic Robotics
这一类关注VLA 如何从observe → act,发展到observe → reason → plan → act → recover
CoT-VLA分类
- Vision Chain-of-Thought
- Visual Reasoning
- Long-horizon Manipulation
VLA论文精读(十三)CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
https://arxiv.org/pdf/2503.22020
RoboBrain分类
- Robot Brain Architecture
- Embodied Intelligence
- Hierarchical Robot Reasoning
VLA 论文精读(二十七)RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
https://arxiv.org/pdf/2502.21257
TIGeR分类
- Tool-Integrated Reasoning
- Geometric Reasoning
- Vision-Language Model for Robotics
VLA论文精读(三十五)TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics
https://arxiv.org/pdf/2510.07181
FEAST分类
- Robot Assistance
- Personalization
- In-the-wild Deployment
VLA 论文精读(二十六)FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization
https://arxiv.org/pdf/2506.14968
Multi-agent Embodied AI分类
- Multi-agent Robotics
- Collaborative Embodied Intelligence
VLA 论文精读(二十二)Multi-agent Embodied AI: Advances and Future Directions
https://arxiv.org/pdf/2505.05108
11. Fine-tuning, Adaptation and Efficient VLA
这一类关注大型 VLA 如何
- 更快适配新任务;
- 降低训练成本;
- 提高推理速度。
Fine-Tuning Vision-Language-Action Models分类
- VLA Fine-tuning
- Training Strategy
- Success Optimization
VLA 论文精读(八)Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
https://arxiv.org/pdf/2502.19645
ConRFT分类
- Reinforcement Fine-tuning
- VLA Optimization
- Consistency Policy
VLA 论文精读(三十二)ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy
https://arxiv.org/pdf/2502.05450
MoLe-VLA分类
- Efficient VLA
- Dynamic Inference
- Layer Skipping
VLA 论文精读(三十)MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation
https://arxiv.org/pdf/2503.20384
SmolVLA分类
- Efficient Deployment
- Small VLA Model
VLA 论文精读(三十一)SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
https://arxiv.org/pdf/2506.01844
12. Surveys
这一类用于建立领域地图。
A Survey on Vision-Language-Action Models: An Action Tokenization Perspective分类
- Survey
- Action Representation
VLA 论文精读(三十三)A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
https://arxiv.org/pdf/2507.01925
Pure Vision Language Action (VLA) Models: A Comprehensive Survey分类
- Comprehensive Survey
- VLA Taxonomy
VLA 论文精读(三十四)Pure Vision Language Action (VLA) Models: A Comprehensive Survey
https://arxiv.org/pdf/2509.19012
Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey分类
- Efficient VLA
- Model Compression
- Deployment
VLA论文精读(三十六)Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
https://arxiv.org/pdf/2510.17111
Research Questions
Data scaling
机器人是否可以像大语言模型一样,通过扩大数据规模获得能力提升?VLA 面临的核心问题是机器人数据昂贵、embodiment 不统一、动作空间不同以及长尾失败案例稀缺。主要路线包括:
- Real Robot Data:真实机器人数据,例如 Open X-Embodiment、π0/π0.5。优势是物理真实性高,但规模有限。
- Synthetic Data:例如 DemoGen、RoboTwin 2.0、ROBOVERSE。优势是可扩展,但存在 simulation gap。
- Human Data:例如 teleoperation 和 egocentric video。优势是行为丰富,但人与机器人 embodiment 不一致。
Open Questions:
- 是否存在 robot scaling law?
- 什么样的数据最有价值?
- failure data 是否比 successful demonstration 更重要?
- 数据规模增加是否带来能力跃迁?
Representation
机器人需要什么样的世界表示?当前 VLA 主要依赖 image + language,但真实操作需要理解 geometry、object state、contact state、affordance 和 physical dynamics。主要方向包括:
- Vision Representation:利用 CLIP、DINO、VLM 等视觉表示,语义理解强,但空间精度不足。
- 3D Representation:例如 PointVLA、Lift3D、FP3,引入 depth、point cloud,提高空间理解能力。
- Affordance Representation:例如 AffordDP,关注物体如何被操作,而不仅是物体类别。
Open Questions:
- VLA 是否需要显式 3D 表示?
- latent representation 是否已经包含足够空间信息?
- robot memory 应该存储什么形式的状态?
Action generation
如何从视觉和语言生成机器人动作?
主要路线包括:
- Action Tokenization:例如 OpenVLA、Universal Actions,将连续动作转换为 token,使 Transformer 可以处理,但可能损失连续控制信息。
- Diffusion / Flow Policy:例如 Diffusion Policy、π0、RDT-1B,直接生成动作轨迹,适合多模态和复杂 manipulation。
- Reactive Policy:例如 Reactive Diffusion Policy,引入快速反馈,提高 contact-rich task 稳定性。
Open Questions:
- action token 是否会成为机器人领域类似语言 token 的表示?
- diffusion policy 是否适合作为通用控制接口?
- policy 如何结合 reasoning 和 memory?
Memory
机器人如何利用过去经验?当前 VLA 通常学习:
隐含假设当前 observation 足够,但真实任务具有长期依赖。
Memory 可以分为:
- Short-term Memory:保存最近 observation/action,用于 temporal reasoning。
- Episodic Memory:保存过去执行经验,例如类似任务的成功轨迹。
- Semantic Memory:保存抽象知识,例如物体属性和操作规律。
相关方向:
- RoboBrain;
- Robot ICL;
- MEM;
- Hi Robot。
Open Questions:
- memory 应该保存 observation、action 还是 latent state?
- 如何进行 memory retrieval?
- 如何避免错误经验影响策略?
- memory 是否可以减少机器人数据需求?
Long-horizon planning
机器人如何完成多步骤、长时间任务?
例如:
prepare breakfast
↓
find bread
↓
open package
↓
place food
当前方法包括:
- Language Planning:利用 LLM 分解任务。
- Hierarchical Policy:高层 planner 负责任务规划,低层 policy 执行动作。
- Reasoning-based Policy:例如 CoT-VLA,通过中间 reasoning 提升复杂任务能力。
Open Questions:
- VLA 是否需要独立 planner?
- skill library 是否仍然必要?
- 长任务失败后如何恢复?
- 如何判断任务完成?
Failure recovery
机器人失败后如何恢复?当前 imitation learning 主要学习成功轨迹,但真实环境包含 perception failure、action failure、planning failure 和 physical failure。
主要方向:
- Feedback Control:加入 vision、force、tactile feedback。
- Self-correction:机器人观察失败并修改策略。
- Experience Replay:将失败案例加入训练或 memory。
Open Questions:
- failure 是否应该成为主要训练数据?
- 如何自动发现和分类失败?
- 如何区分偶然失败和策略错误?
- robot 是否需要类似 debugging 的能力?
Evaluation
如何评价机器人是否真正具备通用能力?传统 benchmark 主要关注 task success rate,但通用机器人还需要评价 generalization、robustness、adaptation 和 recovery ability。
当前方向:
- Task Benchmark:VLABench、RoboTwin、ROBOVERSE。
- Generalization Evaluation:测试 unseen object、environment、instruction。
- Reliability Evaluation:测试失败率、一致性和恢复能力。
Open Questions:
- success rate 是否足够?
- 如何评价长期自主能力?
- 如何评价 memory 和 adaptation?
- 是否需要新的 robot intelligence benchmark?
Suggested Reading Path for Robot Research
Stage 1: Understand VLA Core
- Diffusion Policy
- OpenVLA
- π0
- π0.5
Stage 2: Understand Scaling
- GR00T N1
- Gemini Robotics
- RoboTwin 2.0
- ROBOVERSE
Stage 3: Understand Current Bottlenecks
- VLABench
- Reactive Diffusion Policy
- 3D-ViTac
- RoboBrain
- CoT-VLA
Stage 4: Explore Research Opportunities
- PointVLA
- Lift3D
- FP3
- DexGraspVLA
- RDT-1B
- TIGeR
CSDN按顺序
VLA 论文精读(一)OpenVLA: An Open-Source Vision-Language-Action Model
https://arxiv.org/pdf/2406.09246
VLA 论文精读(二)GR00T N1: An Open Foundation Model for Generalist Humanoid Robots
https://arxiv.org/pdf/2503.14734
VLA 论文精读(三)Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
https://arxiv.org/pdf/2303.04137
VLA 论文精读(四)DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learning
https://arxiv.org/pdf/2502.16932
VLA 论文精读(五)VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks
https://arxiv.org/pdf/2412.18194
VLA 论文精读(六)Beyond Sight: Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding
https://arxiv.org/pdf/2501.04693
VLA 论文精读(七)Reactive Diffusion Policy: Slow-Fast Visual-Tactile Policy Learning for Contact-Rich Manipulation
https://arxiv.org/pdf/2503.02881
VLA 论文精读(八)Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success
https://arxiv.org/pdf/2502.19645
VLA 论文精读(九)ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning
https://arxiv.org/pdf/2503.21860
VLA论文精读(十)DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Grasping
https://arxiv.org/pdf/2502.20900
VLA论文精读(十一)ROBOVERSE: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning
https://roboverseorg.github.io/static/pdfs/roboverse.pdf
VLA论文精读(十二)Universal Actions for Enhanced Embodied Foundation Models
https://arxiv.org/pdf/2501.10105
VLA论文精读(十三)CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models
https://arxiv.org/pdf/2503.22020
VLA论文精读(十四)PointVLA: Injecting the 3D World into Vision-Language-Action Models
https://arxiv.org/pdf/2503.07511
VLA 论文精读(十五)Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation
https://arxiv.org/pdf/2411.18623
VLA 论文精读(十六)FP3: A 3D Foundation Policy for Robotic Manipulation
https://arxiv.org/pdf/2503.08950
VLA 论文精读(十七)AffordDP: Generalizable Diffusion Policy with Transferable Affordance
https://arxiv.org/pdf/2412.03142
VLA 论文精读(十八)π0.5: a Vision-Language-Action Model with Open-World Generalization
https://arxiv.org/pdf/2504.16054
VLA 论文精读(十九)Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
https://arxiv.org/pdf/2304.13705
VLA 论文精读(二十)3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing
https://arxiv.org/pdf/2410.24091
VLA 论文精读(二十一)RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation
https://arxiv.org/pdf/2410.07864
VLA 论文精读(二十二)Multi-agent Embodied AI: Advances and Future Directions
https://arxiv.org/pdf/2505.05108
VLA 论文精读(二十三)π0: A Vision-Language-Action Flow Model for General Robot Control
https://www.pi.website/download/pi0.pdf
VLA 论文精读(二十四)ALOHA Unleashed: A Simple Recipe for Robot Dexterity
https://arxiv.org/pdf/2410.13126
VLA 论文精读(二十五)Gemini Robotics: Bringing AI into the Physical World
https://arxiv.org/pdf/2503.20020
VLA 论文精读(二十六)FEAST: A Flexible Mealtime-Assistance System Towards In-the-Wild Personalization
https://arxiv.org/pdf/2506.14968
VLA 论文精读(二十七)RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete
https://arxiv.org/pdf/2502.21257
VLA 论文精读(二十八)RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization
https://arxiv.org/pdf/2506.18088
VLA 论文精读(二十九)VR-Robo: A Real-to-Sim-to-Real Framework for Visual Robot Navigation and Locomotion
https://arxiv.org/pdf/2502.01536
VLA 论文精读(三十)MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation
https://arxiv.org/pdf/2503.20384
VLA 论文精读(三十一)SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics
https://arxiv.org/pdf/2506.01844
VLA 论文精读(三十二)ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy
https://arxiv.org/pdf/2502.05450
VLA 论文精读(三十三)A Survey on Vision-Language-Action Models: An Action Tokenization Perspective
https://arxiv.org/pdf/2507.01925
VLA 论文精读(三十四)Pure Vision Language Action (VLA) Models: A Comprehensive Survey
https://arxiv.org/pdf/2509.19012
VLA 论文精读(三十五)TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Robotics
https://arxiv.org/pdf/2510.07181
VLA 论文精读(三十六)VLA-0: Building State-of-the-Art VLAs with Zero Modification
https://arxiv.org/pdf/2510.13054
VLA 论文精读(三十六)Efficient Vision-Language-Action Models for Embodied Manipulation: A Systematic Survey
https://arxiv.org/pdf/2510.17111

浙公网安备 33010602011771号