高级检索

    机器人声纹特征提取的Whisper模型研究

    Research on Whisper Model for Robot Voiceprint Feature Extraction

    • 摘要: 人形机器人是具身智能产业的核心领域,其在复杂开放环境下的听觉身份感知仍是难点。基于 Whisper 编码器分析语音多层隐藏特征的层级贡献,提出一种两阶段自适应融合模型,首先利用全层全局加权机制评估各隐藏层对身份判别的贡献,筛选出身份强相关的关键声学层级,随后对优选层级进行投影对齐与二次自适应加权融合,增强各层信息的互补表达。结合ECAPA-TDNN后端实验表明,该方法在标准、跨域方言及复杂场景下均优于传统基准,为高可靠机器人听觉与交互系统提供有效前端方案。

       

      Abstract: Humanoid robots represent a core area of the embodied intelligence industry, and enabling reliable auditory identity perception in complex open environments remains a major challenge. The accuracy of voiceprint feature extraction directly affects the interaction experience of robots and constitutes a key technology. This study provides an in-depth analysis of the Whisper encoder’s capability to extract multi-layer hidden speech features. Through layer-wise contribution analysis, the preferences and redundancies of different encoder layers in acoustic and semantic encoding are revealed. Accordingly, a two-stage adaptive fusion model is proposed. First, a full-layer global weighting mechanism is used to evaluate the contribution of each hidden layer to identity discrimination and to select key acoustic layers that are strongly related to speaker identity. Then, the selected layers are subjected to projection alignment and secondary adaptive weighted fusion to enhance the complementary representation of multi-layer information. Experiments based on the ECAPA-TDNN architecture show that the proposed model outperforms traditional baselines in standard, cross-domain dialect, and complex real-world noisy environments, with a significant reduction in equal error rate (EER). The findings can support the design of highly reliable robotic auditory and interaction systems.

       

    /

    返回文章
    返回