Vue3+vite项目集成科大讯飞语音合成踩坑记录

背景

本人前些时候在做公司项目（乾坤湾大屏）的时候，需要语音播报功能，用到科大讯飞的在线语音合成功能，遇到了许多坑，做了个总结，给没有用过的兄弟姐妹参考。科大讯飞语音合成 主要将文字转化为自然流畅的人声，支持多语种，可以选择体验中心，扫码进入选择语音合成体验一下

一、项目环境

vue2.7+vite

二、注册科大讯飞

注册后新建个应用，拿到APPID、APISecret、APIkey，在项目中会用到这三个参数，新用户有500的免费的服务量。

三、下载语音合成demo

因为是将合成功能集成到前端项目中，后端不参与，所以我进入 科大讯飞文档中心 中示例demo，选择了下载 “语音合成流式API demo js语言” ，该demo项目环境为webpack+js，可以在package.json中看到

下载的demo文件名为 tts_ws_js_demo，下图为vscode打开的demo文件夹结构截图：

四、集成到前端Vue项目

项目目录如下：

warning-notice文件夹为需要文字转化为语音的组件目录，transcode.worker.js直接从tts_ws_js_demo的js文件夹中获取，TTSRecorder.js为index.js修改后所得，里面逻辑去除了与本项目无关或者没用到的的所有代码。 比如没用到的download.js，vconsole等

// import {downloadPCM, downloadWAV} from 'js/download.js'
// import Enc from 'enc'
// import VConsole from 'vconsole'
// import './index.css'

还有里面所有无关的jQuery代码，

将从科大讯飞注册得到的APPID、API_SECRET、API_KEY正确填入TTSRecorder.js

然后将类TTSRecorder作为默认接口暴露出来

export default class TTSRecorder

六、 语音合成调用

import TTSRecorder from './js/TTSRecorder.js'

const ttsRecorder = new TTSRecorder()

function audioPlay(text='我是默认文本',voiceName='xiaoyan'){

  ttsRecorder.setParams({

    voiceName,

    tte: 'UTF8',

    text ,

    // speed : 50 ,

    // voice : 50

  })

  if (['init', 'endPlay', 'errorTTS'].indexOf(ttsRecorder.status) > -1) {

    ttsRecorder.start()

  } else {

    ttsRecorder.stop()

  }

}

七、踩坑记录

踩坑1：

运行过程中总是弹出 'WebSocket报错，请f12查看详情'

定位到代码TTSRecorder.js，将alert直接注释了

ttsWS.onerror = e => {

  clearTimeout(this.playTimeout)

  this.setStatus('errorTTS')

  // alert('WebSocket报错，请f12查看详情')

  console.error('WebSocket报错，请f12查看详情')

  console.error(`详情查看：${encodeURI(url.replace('wss:', 'https:'))}`)

}

踩坑2：

在科大讯飞的js-demo中，并没有用到vue或者vite，我们实际开发时都会遇到 let transWorker = new TransWorker() 代码导致的报错，意思都差不多。

譬如我在项目中遇到的：

Uncaught TypeError: _transcode_worker_js__WEBPACK_IMPORTED_MODULE_8___default.a is not a constructor

其他猿友也遇到过如下报错：

报错1：

TypeError:TransWorker is not a constructor

报错2：

Uncaught SyntaxError: The requested module '/src/until/transcode.worker.js?t=1671455993687' does not provide an export named 'default'

为什么会报错呢？

import TransWorker from './transcode.worker.js'引入了webWorker文件，而我们项目是vite+vue，无法直接使用原生的new Worker这一特性

如何解决？

1、首先安装worker-loader的2.0.0版本：

npm install worker-loader@2.0.0 -D

2、然后在vue.config.js中找到configureWebpack，添加以下配置

configureWebpack: config => {

config.module.rules.push({

    test : /\.worker.js$/,

    use : {

        loader : 'worker-loader',

        options : {

            inline : true ,

            name : 'workerName.[hash].js'

        }

    }

})

},

或者在chainWebpack中添加以下配置：

chainWebpack: config => {

config.module

    .rule('worker-loader')

    .test(/\.worker\.js$/)

    .use('worker-loader')

    .loader('worker-loader')

    .options({

        inline: true

    })

    .end()

},

并在和chainWebpack或者configureWebpack同级的地方添加

parallel : false,

chainWebpack中 添加

config.output.globalObject('this')

3、修改transcode.worker.js文件：

修改前：

修改后：

这个很简单，就是注释掉(删掉)了transcode.worker.js代码里面的立即执行函数，其余的代码维持不变。

另外一种修改方式为：

此方法核心是将立即执行函数作为一个接口变量暴露出来。

关于webWoker

webWorker相当于js中的线程，在主线程中启动一个子线程不影响ui。

web worker 是运行在后台的 JavaScript，独立于其他脚本，不会影响页面的性能。您可以继续做任何愿意做的事情：点击、选取内容等等，而此时 web worker 在后台运行。

有关webworker的配置问题，在 Vite官方中文文档 中也有叙述，

按照文档中的说明，我将以下代码

import TransWorker from './transcode.worker.js' 
let transWorker = new TransWorker()

改为

const transWorker = new Worker(new URL('./transcode.worker.js', import.meta.url))

不过很遗憾的事，在我项目中运行到这段代码后，还是会报错，反而改为如下代码，就不会报错了，

const transWorker = new Worker('./transcode.worker.js')

有谁可以为我解惑么？

项目代码:

TTSRecorder.js

/*
 * @Autor: lycheng
 * @Date: 2020-01-13 16:12:22
 */
/**
 * Created by iflytek on 2019/11/19.
 *
 * 在线语音合成调用demo
 * 此demo只是一个简单的调用示例，不适合用到实际生产环境中
 *
 * 在线语音合成 WebAPI 接口调用示例 接口文档（必看）：https://www.xfyun.cn/doc/tts/online_tts/API.html
 * 错误码链接：
 * https://www.xfyun.cn/doc/tts/online_tts/API.html
 * https://www.xfyun.cn/document/error-code （code返回错误码时必看）
 *
 */

// 1. websocket连接：判断浏览器是否兼容，获取websocket url并连接，这里为了方便本地生成websocket url
// 2. 连接websocket，向websocket发送数据，实时接收websocket返回数据
// 3. 处理websocket返回数据为浏览器可以播放的音频数据
// 4. 播放音频数据
// ps: 该示例用到了es6中的一些语法，建议在chrome下运行


//APPID，APISecret，APIKey在控制台-我的应用-语音合成（流式版）页面获取
const APPID = '项目APPID'
const API_SECRET = '项目APISecret'
const API_KEY = '项目APIKey'

import CryptoJS from 'crypto-js'
import { Base64 } from 'js-base64'
import TransWorker from './transcode.worker.js'
const transWorker = new TransWorker()

function getWebsocketUrl() {
  return new Promise((resolve, reject) => {
    var apiKey = API_KEY
    var apiSecret = API_SECRET
    var url = 'wss://tts-api.xfyun.cn/v2/tts'
    //var host = location.host
    var host = 'tts-api.xfyun.cn'
    var date = new Date().toGMTString()
    var algorithm = 'hmac-sha256'
    var headers = 'host date request-line'
    var signatureOrigin = `host: ${host}\ndate: ${date}\nGET /v2/tts HTTP/1.1`
    var signatureSha = CryptoJS.HmacSHA256(signatureOrigin, apiSecret)
    var signature = CryptoJS.enc.Base64.stringify(signatureSha)
    var authorizationOrigin = `api_key="${apiKey}", algorithm="${algorithm}", headers="${headers}", signature="${signature}"`
    var authorization = btoa(authorizationOrigin)
    url = `${url}?authorization=${authorization}&date=${date}&host=${host}`
    resolve(url)
  })
}
export default class TTSRecorder {
  constructor({
    speed = 50,
    voice = 50,
    pitch = 50,
    voiceName = 'xiaoyan',
    appId = APPID,
    text = '',
    tte = 'UTF8',
    defaultText = '请输入您要合成的文本',
  } = {}) {
    this.speed = speed
    this.voice = voice
    this.pitch = pitch
    this.voiceName = voiceName
    this.text = text
    this.tte = tte
    this.defaultText = defaultText
    this.appId = appId
    this.audioData = []
    this.rawAudioData = []
    this.audioDataOffset = 0
    this.status = 'init'
    transWorker.onmessage = (e) => {
      this.audioData.push(...e.data.data)
      this.rawAudioData.push(...e.data.rawAudioData)
    }
  }
  // 修改录音听写状态
  setStatus(status) {
    this.onWillStatusChange && this.onWillStatusChange(this.status, status)
    this.status = status
  }
  // 设置合成相关参数
  setParams({ speed, voice, pitch, text, voiceName, tte }) {
    speed !== undefined && (this.speed = speed)
    voice !== undefined && (this.voice = voice)
    pitch !== undefined && (this.pitch = pitch)
    text && (this.text = text)
    tte && (this.tte = tte)
    voiceName && (this.voiceName = voiceName)
    this.resetAudio()
  }
  // 连接websocket
  connectWebSocket() {
    this.setStatus('ttsing')
    return getWebsocketUrl().then(url => {
      let ttsWS
      if ('WebSocket' in window) {
        ttsWS = new WebSocket(url)
      } else if ('MozWebSocket' in window) {
        ttsWS = new MozWebSocket(url)
      } else {
        alert('浏览器不支持WebSocket')
        return
      }
      this.ttsWS = ttsWS
      ttsWS.onopen = e => {
        this.webSocketSend()
        this.playTimeout = setTimeout(() => {
          this.audioPlay()
        }, 1000)
      }
      ttsWS.onmessage = e => {
        this.result(e.data)
      }
      ttsWS.onerror = e => {
        clearTimeout(this.playTimeout)
        this.setStatus('errorTTS')
        // alert('WebSocket报错，请f12查看详情')
        console.error('WebSocket报错，请f12查看详情')
        console.error(`详情查看：${encodeURI(url.replace('wss:', 'https:'))}`)
      }
      ttsWS.onclose = e => {
        console.log(e)
      }
    })
  }
  // 处理音频数据
  transToAudioData(audioData) {}
  // websocket发送数据
  webSocketSend() {
    var params = {
      common: {
        app_id: this.appId, // APPID
      },
      business: {
        aue: 'raw',
        auf: 'audio/L16;rate=16000',
        vcn: this.voiceName,
        speed: this.speed,
        volume: this.voice,
        pitch: this.pitch,
        bgs: 1,
        tte: this.tte,
      },
      data: {
        status: 2,
        text: this.encodeText(
          this.text || this.defaultText,
          this.tte === 'unicode' ? 'base64&utf16le' : ''
        )
      },
    }
    this.ttsWS.send(JSON.stringify(params))
  }
  encodeText (text, encoding) {
    switch (encoding) {
      case 'utf16le' : {
        let buf = new ArrayBuffer(text.length * 4)
        let bufView = new Uint16Array(buf)
        for (let i = 0, strlen = text.length; i < strlen; i++) {
          bufView[i] = text.charCodeAt(i)
        }
        return buf
      }
      case 'buffer2Base64': {
        let binary = ''
        let bytes = new Uint8Array(text)
        let len = bytes.byteLength
        for (let i = 0; i < len; i++) {
          binary += String.fromCharCode(bytes[i])
        }
        return window.btoa(binary)
      }
      case 'base64&utf16le' : {
        return this.encodeText(this.encodeText(text, 'utf16le'), 'buffer2Base64')
      }
      default : {
        return Base64.encode(text)
      }
    }
  }
  // websocket接收数据的处理
  result(resultData) {
    let jsonData = JSON.parse(resultData)
    // 合成失败
    if (jsonData.code !== 0) {
      alert(`合成失败: ${jsonData.code}:${jsonData.message}`)
      console.error(`${jsonData.code}:${jsonData.message}`)
      this.resetAudio()
      return
    }
    transWorker.postMessage(jsonData.data.audio)

    if (jsonData.code === 0 && jsonData.data.status === 2) {
      this.ttsWS.close()
    }
  }
  // 重置音频数据
  resetAudio() {
    this.audioStop()
    this.setStatus('init')
    this.audioDataOffset = 0
    this.audioData = []
    this.rawAudioData = []
    this.ttsWS && this.ttsWS.close()
    clearTimeout(this.playTimeout)
  }
  // 音频初始化
  audioInit() {
    let AudioContext = window.AudioContext || window.webkitAudioContext
    if (AudioContext) {
      this.audioContext = new AudioContext()
      this.audioContext.resume()
      this.audioDataOffset = 0
    } 
  }
  // 音频播放
  audioPlay() {
    this.setStatus('play')
    let audioData = this.audioData.slice(this.audioDataOffset)
    this.audioDataOffset += audioData.length
    let audioBuffer = this.audioContext.createBuffer(1, audioData.length, 22050)
    let nowBuffering = audioBuffer.getChannelData(0)
    if (audioBuffer.copyToChannel) {
      audioBuffer.copyToChannel(new Float32Array(audioData), 0, 0)
    } else {
      for (let i = 0; i < audioData.length; i++) {
        nowBuffering[i] = audioData[i]
      }
    }
    let bufferSource = this.bufferSource = this.audioContext.createBufferSource()
    bufferSource.buffer = audioBuffer
    bufferSource.connect(this.audioContext.destination)
    bufferSource.start()
    bufferSource.onended = event => {
      if (this.status !== 'play') {
        return
      }
      if (this.audioDataOffset < this.audioData.length) {
        this.audioPlay()
      } else {
        this.audioStop()
      }
    }
  }
  // 音频播放结束
  audioStop() {
    this.setStatus('endPlay')
    clearTimeout(this.playTimeout)
    this.audioDataOffset = 0
    if (this.bufferSource) {
      try {
        this.bufferSource.stop()
      } catch (e) {
        console.log(e)
      }
    }
  }
  start() {
    if(this.audioData.length) {
      this.audioPlay()
    } else {
      if (!this.audioContext) {
        this.audioInit()
      }
      if (!this.audioContext) {
        alert('该浏览器不支持webAudioApi相关接口')
        return
      }
      this.connectWebSocket()
    }
  }
  stop() {
    this.audioStop()
  }
}

// ======================调用======================
// import TTSRecorder from './js/TTSRecorder.js'
// const ttsRecorder = new TTSRecorder()
// function audioPlay(text='我是默认文本',voiceName='xiaoyan'){
//   ttsRecorder.setParams({ 
//     voiceName, 
//     tte: 'UTF8',
//     text ,
//     // speed : 50 ,
//     // voice : 50
//   })
//   if (['init', 'endPlay', 'errorTTS'].indexOf(ttsRecorder.status) > -1) {
//     ttsRecorder.start()
//   } else {
//     ttsRecorder.stop()
//   }
// }

transcode.worker.js

/*
 * @Autor: lycheng
 * @Date: 2020-01-13 16:12:22
 */
// (function(){
//   let minSampleRate = 22050

self.onmessage = function(e) {
  transcode.transToAudioData(e.data)
}
let transcode = {
  transToAudioData: function(audioDataStr, fromRate = 16000, toRate = 22505) {
    let outputS16 = transcode.base64ToS16(audioDataStr)
    let output = transcode.transS16ToF32(outputS16)
    output = transcode.transSamplingRate(output, fromRate, toRate)
    output = Array.from(output)
    self.postMessage({
      data: output, 
      rawAudioData: Array.from(outputS16)
    })
  },
  transSamplingRate: function(data, fromRate = 44100, toRate = 16000) {
    let fitCount = Math.round(data.length * (toRate / fromRate))
    let newData = new Float32Array(fitCount)
    let springFactor = (data.length - 1) / (fitCount - 1)
    newData[0] = data[0]
    for (let i = 1; i < fitCount - 1; i++) {
      let tmp = i * springFactor
      let before = Math.floor(tmp).toFixed()
      let after = Math.ceil(tmp).toFixed()
      let atPoint = tmp - before
      newData[i] = data[before] + (data[after] - data[before]) * atPoint
    }
    newData[fitCount - 1] = data[data.length - 1]
    return newData
  },
  transS16ToF32: function(input) {
    let tmpData = []
    for (let i = 0; i < input.length; i++) {
      let d = input[i] < 0 ? input[i] / 0x8000 : input[i] / 0x7fff
      tmpData.push(d)
    }
    return new Float32Array(tmpData)
  },
  base64ToS16: function(base64AudioData) {
    base64AudioData = atob(base64AudioData)
    const outputArray = new Uint8Array(base64AudioData.length)
    for (let i = 0; i < base64AudioData.length; ++i) {
      outputArray[i] = base64AudioData.charCodeAt(i)
    }
    return new Int16Array(new DataView(outputArray.buffer).buffer)
  },
}

// })()

warning-notice/index.vue

<!--
 * @Description: 预警通知
 * @Author: wang pingqi
 * @Date: 2023-01-04 16:34:17
 * @LastEditors: wang pingqi
 * @LastEditTime: 2023-01-09 17:35:30
-->
<template>
  <el-dialog
    :visible.sync="dialogTarget.visible"
    :width="dialogTarget.width"
    append-to-body
    :modal="true"
    :close-on-click-modal="false"
    :destroy-on-close="true"
    @open="()=>{
      $emit('dialogShow');
      handleDialogOpen();
    }"
    class="custom-dialog-wrapper"
  >
    <div class="yj-dialog-header">
      <div class="header-bg">
        <div class="radius rd-l" @click="dialogTarget.visible=false"></div>
        <div class="radius rd-r" @click="dialogTarget.visible=false"></div>
      </div>
      <div class="title header-pure"><span class="left-line"></span> <span class="text">{{ dialogTarget.title }}</span></div>
    </div>
    <div class="dialog-content">
      <ul v-loading="dialogTarget.loading">
        <li v-for="(item,index) in dialogTarget.data" :key="index">
          <p class="flex row-between mb9">
            <span class="left">
              <span class="index mr7">{{index+1}}.</span>
              <span class="time">{{item.noticeTime}}</span>
            </span>
            <span class="right">告警类型：<span class="danger">{{item.noticeType}}</span></span>
          </p>
          <p>{{item.noticeMessage}}</p>
        </li>
      </ul>     
    </div>
    <div class="dialog-footer">
      <div slot="footer" class="dialog-footer">
        <ui-pagination
          :total="dialogPager.total"
          @change="handleDialogPagerChange"
          :search-data="dialogPager.searchData"
        >
        </ui-pagination>
      </div>
    </div>
  </el-dialog>
</template>

<script>
  import TTSRecorder from './js/TTSRecorder.js'
  const ttsRecorder = new TTSRecorder()
  export default {
    name: 'warning-notice',
    components: {},
    props: {
      data: {
        type: Object,
        default: () => ({})
      },
    },
    data() {
      return {
        // 预警通知数据
        dialogTarget: {
          title: '预警通知',
          visible: false,
          width: "452px",
          loading : false,
          data : []
        },
        dialogPager: {
          searchData: {
            searchKey: "",
            pageNo: 1,
            pageSize: 3,
          },
          total: 0,
        }
      }
    },
    mounted() {
      // // 预警通知
      // this.getWarningNotice(true)
      this.$on('global:timerTask', () => {
        this.getWarningNotice(true)
      })
    },
    methods: {
      handleDialogOpen(){
        let tt= setTimeout(() => {
          clearTimeout(tt)
          tt = undefined
          this.autoPlay()
        }, 880);
      },
      autoPlay(){
        const records = this.dialogTarget.data
        if(records.length>0){
          const firstTarget = records[0]
          const firstWarnText= `预警通知
          ${firstTarget.noticeMessage}
          `
          this.audioPlay(firstWarnText)
        }
      },
      // 播放音频
      audioPlay(text='我是默认文本'){
        ttsRecorder.setParams({ 
          text
        })
        if (['init', 'endPlay', 'errorTTS'].indexOf(ttsRecorder.status) > -1) {
          ttsRecorder.start()
        } else {
          ttsRecorder.stop()
        } 
      },
      // 预警通知
      async getWarningNotice(openDialog=false){
        this.$set(this.dialogTarget,'loading',true)
        let res = await this.$http.post('/passenger/warning/notice/screen/queryForPage', {
          pageNo : this.dialogPager.searchData.pageNo,
          pageSize : this.dialogPager.searchData.pageSize
        })
        if(res.data.records?.length){
          this.$set(this.dialogPager,'total',res.data.total)
          this.$set(this.dialogTarget,'data',res.data.records)
          openDialog && this.$set(this.dialogTarget,'visible',true)
          this.$set(this.dialogTarget,'loading',false)
        }
      },
      open(){
        this.getWarningNotice(true)
      },
      /**
       * 弹框中分页改变事件
       * @param {Number} pageNo
       * @return Void
       */
      handleDialogPagerChange(pageNo) {
        this.$set(this.dialogPager.searchData,'pageNo',pageNo)
        this.getWarningNotice();
      },
      handleBtnHeadTap(){
        this.$set(this.dialogTarget,'visible',true)
      }
    }
  }
</script>

<style lang="scss" scoped>
.danger{
  color :#f56c6c
}
.warning{
  color :#e6a23c;
}
.primary{
  color :#409eff;
}
.success{
  color :#67c23a;
}
.dialog-content{
  padding: 0 20px;
  transform: translateY(-15px);
  ul{
    li{
      position: relative;
      overflow: hidden;
      padding: 20px 0;
      &::before{
        position: absolute;
        left:0;
        right: 0;
        bottom: 0;
        height: 1px;
        content:"";
        z-index:1;
        border-radius: 15px;
        border-bottom: 0.5px solid;
        border-image: linear-gradient(-154deg, #ffffff 28%, rgba(224,244,253,0.00) 84%) 0.5 0.5;
      }
      &::after{
        position: absolute;
        right: 0;
        bottom: 0;
        z-index:2;
        height: 2px;
        content:"";
        width: 10px;
        background: #fff;
        border-radius: 100px;
      } 
      p{
        &:first-of-type{
          .left{
            .index{
              font: 400 16px/22px PingFang SC, PingFang SC-5;
              color: #bdc2d0;
            }
            .time{
              font: 400 16px/22px PingFang SC, PingFang SC-5;
              color: #ffffff;
            }
          }
          .right{
            font: 400 14px/20px PingFang SC, PingFang SC-5;
            color: #ffffff;
          }
        }
        &:last-of-type{
          font: 400 12px/17px PingFang SC, PingFang SC-5;
          color: #bdc2d0;
        }
      }
    }
  }
}
</style>