如何对webRtc进行流媒体数据监控

海绵泡泡
443 阅读5分钟

背景

在一个使用webRtc进行实时音视频的应用中，经常会遇到因为网络导致的各种各样的问题，如何分别问题类型（是音频问题还是视频问题），如何能实时的了解到当时的各项指标（码率、帧率等）；

举一个例子：有一个用户说听不到对方声音，你如何分辨出具体原因，是他的喇叭坏了还是对方麦克风坏了？ 我们可以看他收到的音频码率，和对方发出的音频码率。没有波动的一方及设备是有问题的，如果都没有问题。可以考虑设备授权问题。

所以要完成上面的内容，需要知道码率是多少吧。所以这就是这项工作的实际落地场景。

实现思路

以1V1的为基础， 首先可以拿到两个流(包含video&audio两个轨道)，然后通过**stream.getTracks()**拿到所有的媒体轨道，以下为一个stream的两个轨道； image.png

  • 通过轨道id，比对rtp信息的trackIdentifier即可拿到当前流的inbound信息；
  • 但是注意,chrome112版本移除了outbound的trackId属性， 所以需要先通过trackIdentifier找出stream对应的mediaSource，然后再找通过mediaSource.id===outbound.mediaSourceId, 找出stream对应的outbound， remote Outbound也是同理

拿到数据后，可以用ant-chart进行展示。

getStats主要类型

codec

发送或接收的RTP 流 当前正在使用的编解码器的统计信息

{
  "id": "CIT01_96",
    "timestamp": 1683355080781.743,
    "type": "codec",
    "transportId": "T01",
    "payloadType": 96,
    "mimeType": "video/VP8",
    "clockRate": 90000
}

inbound-rtp

接收的入站RTP 流 的统计信息

{
    "id": "IT01A3995811649",
    "timestamp": 1683357825557.409,
    "type": "inbound-rtp",
    "ssrc": 3995811649,
    "kind": "audio",
    "transportId": "T01",
    "codecId": "CIT01_111_minptime=10;useinbandfec=1",
    "mediaType": "audio",
    "jitter": 0.002,
    "packetsLost": 0,
    "trackIdentifier": "1d9cbb19-de56-47f5-8229-e164f52d6087",
    "mid": "0",
    "remoteId": "ROA3995811649",
    "packetsReceived": 137255,
    "packetsDiscarded": 0,
    "fecPacketsReceived": 0,
    "fecPacketsDiscarded": 0,
    "bytesReceived": 9591106,
    "headerBytesReceived": 3843140,
    "lastPacketReceivedTimestamp": 1683357825553,
    "jitterBufferDelay": 5964614.4,
    "jitterBufferTargetDelay": 5031878.4,
    "jitterBufferMinimumDelay": 5031724.8,
    "jitterBufferEmittedCount": 131763840,
    "totalSamplesReceived": 131766720,
    "concealedSamples": 51708,
    "silentConcealedSamples": 0,
    "concealmentEvents": 49,
    "insertedSamplesForDeceleration": 17514,
    "removedSamplesForAcceleration": 66296,
    "audioLevel": 0.003357036042359691,
    "totalAudioEnergy": 5.161005023615497,
    "totalSamplesDuration": 2745.1400000134995,
    "playoutId": "AP",
    "estimatedPlayoutTimestamp": 3892346625519
}

outbound-rtp

当前 出站RTP 流RTCPeerConnection的统计信息

{
    "id": "OT01A3190629150",
    "timestamp": 1683357817557.122,
    "type": "outbound-rtp",
    "ssrc": 3190629150,
    "kind": "audio",
    "transportId": "T01",
    "codecId": "COT01_111_minptime=10;useinbandfec=1",
    "mediaType": "audio",
    "mediaSourceId": "SA5",
    "remoteId": "RIA3190629150",
    "mid": "0",
    "packetsSent": 136856,
    "retransmittedPacketsSent": 0,
    "bytesSent": 9595462,
    "headerBytesSent": 3831968,
    "retransmittedBytesSent": 0,
    "targetBitrate": 32000,
    "totalPacketSendDelay": 0,
    "nackCount": 0,
    "active": true
}

{
    "id": "OT01V1674542192",
    "timestamp": 1683357817557.122,
    "type": "outbound-rtp",
    "ssrc": 1674542192,
    "kind": "video",
    "transportId": "T01",
    "codecId": "COT01_96",
    "mediaType": "video",
    "mediaSourceId": "SV6",
    "remoteId": "RIV1674542192",
    "mid": "1",
    "packetsSent": 403325,
    "retransmittedPacketsSent": 0,
    "bytesSent": 414276128,
    "headerBytesSent": 9834468,
    "retransmittedBytesSent": 0,
    "targetBitrate": 1700000,
    "framesEncoded": 58615,
    "keyFramesEncoded": 23,
    "totalEncodeTime": 290.888,
    "totalEncodedBytesTarget": 0,
    "frameWidth": 640,
    "frameHeight": 480,
    "framesPerSecond": 30,
    "framesSent": 58615,
    "hugeFramesSent": 2,
    "totalPacketSendDelay": 1.874777,
    "qualityLimitationReason": "none",
    "qualityLimitationDurations": {
        "other": 0,
        "cpu": 0,
        "bandwidth": 18.133,
        "none": 2718.968
    },
    "qualityLimitationResolutionChanges": 0,
    "encoderImplementation": "libvpx",
    "firCount": 0,
    "pliCount": 1,
    "nackCount": 0,
    "qpSum": 442965,
    "active": true,
    "powerEfficientEncoder": false,
    "scalabilityMode": "L1T1"
}

remote-inbound-rtp

当前与此对象一起发送的出站流相对应的 远程端点入站RTP 流RTCPeerConnection的统计信息。

{
    "id": "RIA3190629150",
    "timestamp": 1683357813812,
    "type": "remote-inbound-rtp",
    "ssrc": 3190629150,
    "kind": "audio",
    "transportId": "T01",
    "codecId": "COT01_111_minptime=10;useinbandfec=1",
    "jitter": 0.000125,
    "packetsLost": 0,
    "localId": "OT01A3190629150",
    "roundTripTime": 0.002,
    "fractionLost": 0,
    "totalRoundTripTime": 0.828,
    "roundTripTimeMeasurements": 544
}

media-source

拿到当前的媒体流信息，比如音视频流

{
    "id": "SA5",
    "timestamp": 1683357825557.409,
    "type": "media-source",
    "trackIdentifier": "116dbc07-9218-45b8-b0a5-2d00a45a4268",
    "kind": "audio",
    "audioLevel": 0.002685628833887753,
    "totalAudioEnergy": 5.348593224408951,
    "totalSamplesDuration": 2745.1400000134995,
    "echoReturnLoss": -30,
    "echoReturnLossEnhancement": 0.2807721793651581
}

{
    "id": "SV6",
    "timestamp": 1683357825557.409,
    "type": "media-source",
    "trackIdentifier": "07a011d2-d980-4272-bfaf-7f7246b0d5d3",
    "kind": "video",
    "width": 640,
    "height": 480,
    "frames": 58859,
    "framesPerSecond": 30
}

网络质量

调用API自带的监控， 将音视频的数据塞在网络监控时进行上传 ，即可同步网络质量和其他数据 eg、声网自带的网络质量监控，两秒钟会触发一次 image.png

本地发送的音频数据

使用**type==='audio'**的outbound数据进行计算

StatsBasicParameter {
  id: string
  kind: 'video' | 'audio'
  timestamp: number
  hasAudio?: boolean
  hasVideo?: boolean
}

interface LocalAudioStats extends StatsBasicParameter {
  bytesSent: number // 发送的字节数
  CodecType: string
  RecordingLevel: number // 记录音量
  SamplingRate: string  // 采样率
  SendBits: number // 发送数据长度
  SendBitrate: number // 发送码率
  SendLevel: number // 发送音量
  packetsLost: number // 丢包数
  PacketLossRate: number // 丢包率
  roundTripTime: number // rtt时间
  packetsSent: number // 发送包数
}

SendBits = outbountRtp.bytesSent * 8 ,（一个字节有8位）

  • 原始数据里面没有码率，需要自己计算：
  • 码率 = 数据大小/时间端 。 ps:因为数据长度是累加的，所以需要取出区间的大小
  • SendBitrate = (outbountRtp.bytesSent * 8 - (prevOutbountRtp.SendBits || 0)) / timeGap
  • packetsLost = remoteInboundRtp.packetsLost
  • roundTripTime = remoteInboundRtp.roundTripTime

本地发送的视频数据

使用**type==='video'**的outbound数据进行计算

interface LocalVideoStats extends StatsBasicParameter {
  CodecType: string
  CaptureFrameRate: number  // 当前设置帧率
  CaptureResolutionHeight: number  // 当前设定分辨率 - 高
  CaptureResolutionWidth: number // 当前设定分辨率 - 宽
  totalEncodeTime: number // 编码时间
  SendBits: number  // 发送数据长度
  SendBitrate: number // 发送码率
  SendFrame: number  // 发送帧数
  SendFrameRate: number // 发送帧率
  SendResolutionHeight: number
  SendResolutionWidth: number
  packetsLost: number
  packetsSent: number
  PacketLossRate: number
  roundTripTime: number
  rid?: string
}
  • 码率计算方式如上；
  • SendFrame = outbountRtp.framesSent
  • 发送帧率和码率的计算方式类似：
SendFrameRate =
      +(
        (outboundRtp.framesSent - (prevStats.SendFrame || 0)) /
        this.timeGap
      ).toFixed(3) * 1000
  • 丢包数要用远端接收流remoteInboundRtp的数据来计算

packetsLost = remoteInboundRtp.packetsLost

  • roundTripTime = remoteInboundRtp.roundTripTime
  • 丢包率：
  const pl = Math.abs(remoteInboundRtp.packetsLost - prevStats.packetsLost)
  const pr = Math.abs(outboundRtp.packetsSent - prevStats.packetsSent)
  PacketLossRate = +(pl / (pl + pr)).toFixed(3)

远端传过来的音频数据

通过inboundRtp来计算，和本地音频流一致

远端传过来的视频数据

  CodecType: string
  PacketLossRate: number
  RecvBits: number
  RecvBitrate: number
  RecvResolutionHeight: number
  RecvResolutionWidth: number
  RenderFrameRate: number //渲染帧率
  framesDecoded: number
  packetsReceived: number
  packetsLost: number
  jitter: number
  • 渲染帧率
  result.RenderFrameRate =
      +(
        (stats.rtp.framesDecoded - (prevStats.framesDecoded || 0)) /
        this.timeGap
      ).toFixed(3) * 1000
avatar
文章
阅读
粉丝