SciTech-AI-VLK(视觉语言运动学)+SceneBot(场景化机器人🤖)

🎉 VLK 正式发布！

很开心和大家分享这段时间一直在做的工作。
VLK 探索了一种不依赖 "大量人形机器人的遥操作数据"，
而是通过 "重建真实场景自动生成Vision-Language-Kinematics 数据" 的新思路。

也想特别给 SceneBot 点个赞。
我们的 whole-body tracker 就是基于 SceneBot 搭建的，
作为第一批使用者，它真的非常强大，也让整个系统的开发顺利了很多。

• VLK 项目主页：
  https://vision-language-kinematics.github.io
• SceneBot项目主页：
  https://ericcsr.github.io/scenebot/

VLK

• STAGE1: SYNTHETIC DATA GENERATION
  

• STAGE2: VLK POLICY
  

• SYNTHETIC DATA: DATA MODES
  Data modes. Six trajectory modes are synthesized in both an Apartment and a Lab 3DGS replica.

  1. 01 Walk to
     
  2. 02Turn around, 03pick[floor]
     
  3. 04place[floor], 05pick(surface)
     

• SIM DEPLOYMENT: DEPLOYMENT IN SIMULATION
  

• REAL DEPLOYMENT

  1. TASK 1 NAVIGATION
     
  2. TASK 2 BOX LIFTING
     
  3. TASK 3 MULTI-STAGE TASKS
     
  4. TASK 4 ROBUSTNESS: Robustness to disturbance.
     
  5. TASK 5 LONG-HORIZON: Long-horizon tasks
     

• CITATION
  

• ACKNOWLEDGEMENTS
  We thank,
  Xiaoyu Huang for preparing a clean retargeted OMOMO dataset,
  Alejandro Escontrela for guidance on 3DGS training,
  Jenny Zhang and Hunter Liu for providing realistic box assets for rendering,
  and Eric Lalumiere for providing the camera mount.
  We also thank,
  Zhen Wu, Shibo Zhao, and Yunsheng Tian for valuable discussions on motion synthesis and Real2Sim pipelines.
  We are grateful to,
  Haozhi Qi, Linda Shih, Youjian Huang, and Roberto Ceja for their support with the experimental hardware,
  and to Shuying Deng, Zihan Wang, and Charlie Cheng for their valuable feedback on the design and visual presentation of the paper figures and video.
  

Github Related:

https://github.com/abaelhe/vision-language-kinematics.github.io