面向空间操作任务的视觉表征对齐与时延补偿轻量化方法
DOI:
CSTR:
作者:
作者单位:

1.五邑大学机械与自动化工程学院;2.五邑大学电子与信息工程学院

作者简介:

通讯作者:

中图分类号:

基金项目:

2024年广东省普通高校科研重点领域专项(2024ZDZX1008)


Lightweight Visual Representation Alignment and Delay Compensation for Spatial Manipulation Tasks
Author:
Affiliation:

1.School of Electronics and Information Engineering, Wuyi Universit;2.School of Mechanical and Automation Engineering, Wuyi University

Fund Project:

  • 摘要
  • |
  • 图/表
  • |
  • 访问统计
  • |
  • 参考文献
  • |
  • 相似文献
  • |
  • 引证文献
  • |
  • 资源附件
  • |
  • 文章评论
    摘要:

    针对视觉-语言-动作模型在空间操作任务中局部几何特征表征不足,以及真实机器人感知-推理-执行链路中预测区间与执行区间时空错位两类问题,文章提出一种以中间层视觉表征对齐为主、以时延补偿为辅的轻量化方法。首先,采用参数冻结的DINOv2模型作为视觉教师网络,在VLA视觉-语言骨干网络的中间层引入对齐投影层,通过余弦距离损失约束中间层视觉特征与教师特征的一致性,增强模型对局部空间结构的编码能力;同时,设计时延补偿机制,以提升系统在真实执行链路中的鲁棒性。仿真实验表明,π0基础模型引入视觉表征对齐后,任务成功率由72.60%提升至75.07%,进一步结合时延补偿训练后提升至80.87%;在π0.5基础模型上,引入视觉表征对齐后,任务成功率由94.27%提升至96.53%。真实机械臂实验表明,相比π0.5基础模型,所提部署策略在三类空间操作任务上的成功率分别由70.0%、55.0%和50.0%提升至95.0%、90.0%和85.0%,同时将成功回合的平均完成时间由17.97s缩短至16.24s。结果表明,中间层视觉表征对齐能够在不同基础模型上稳定增强空间感知能力,时延补偿更适合作为面向真实执行链路的辅助补偿手段,所提方法兼顾了空间感知增强与真实部署适应性。

    Abstract:

    To address the insufficient representation of local geometric features in Vision-Language-Action (VLA) models for spatial manipulation tasks, as well as the spatiotemporal mismatch between prediction and execution intervals in real robotic perception-inference-execution pipelines, this paper proposes a lightweight method based primarily on intermediate visual representation alignment, with delay compensation as an auxiliary component. First, a frozen DINOv2 model is adopted as the visual teacher network, and an alignment projection layer is introduced into an intermediate layer of the VLA vision-language backbone. A cosine distance loss is used to constrain the consistency between intermediate visual features and teacher features, thereby enhancing the model’s ability to encode local spatial structures. Meanwhile, a delay compensation mechanism is designed to improve the robustness of the system in real execution pipelines. Experimental results in the LIBERO-Spatial simulation environment show that, after visual representation alignment is introduced, the task success rate of the π0-based model increases from 72.60% to 75.07%, and further increases to 80.87% when combined with delay compensation training. For the π0.5-based model, visual representation alignment improves the task success rate from 94.27% to 96.53%. Real-robot experiments show that, compared with the π0.5 baseline model, the proposed deployment strategy improves the success rates of three spatial manipulation tasks from 70.0%, 55.0%, and 50.0% to 95.0%, 90.0%, and 85.0%, respectively, while reducing the average completion time of successful trials from 17.97s to 16.24s. The results demonstrate that intermediate visual representation alignment can consistently enhance spatial perception across different base models, while delay compensation is more suitable as an auxiliary compensation strategy for real execution pipelines. The proposed method achieves a balance between spatial perception enhancement and real-world deployment adaptability.

    参考文献
    相似文献
    引证文献
引用本文
分享
相关视频

文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2026-05-14
  • 最后修改日期:2026-05-22
  • 录用日期:2026-05-25
  • 在线发布日期:
  • 出版日期:
文章二维码