背景
在一个使用webRtc进行实时音视频的应用中,经常会遇到因为网络导致的各种各样的问题,如何分别问题类型(是音频问题还是视频问题),如何能实时的了解到当时的各项指标(码率、帧率等);
举一个例子:有一个用户说听不到对方声音,你如何分辨出具体原因,是他的喇叭坏了还是对方麦克风坏了? 我们可以看他收到的音频码率,和对方发出的音频码率。没有波动的一方及设备是有问题的,如果都没有问题。可以考虑设备授权问题。
所以要完成上面的内容,需要知道码率是多少吧。所以这就是这项工作的实际落地场景。
实现思路
以1V1的为基础, 首先可以拿到两个流(包含video&audio两个轨道),然后通过**stream.getTracks()**拿到所有的媒体轨道,以下为一个stream的两个轨道;
- 通过轨道id,比对rtp信息的trackIdentifier即可拿到当前流的inbound信息;
- 但是注意,chrome112版本移除了outbound的trackId属性, 所以需要先通过trackIdentifier找出stream对应的mediaSource,然后再找通过mediaSource.id===outbound.mediaSourceId, 找出stream对应的outbound, remote Outbound也是同理
拿到数据后,可以用ant-chart进行展示。
getStats主要类型
codec
发送或接收的RTP 流 当前正在使用的编解码器的统计信息
{
"id": "CIT01_96",
"timestamp": 1683355080781.743,
"type": "codec",
"transportId": "T01",
"payloadType": 96,
"mimeType": "video/VP8",
"clockRate": 90000
}
inbound-rtp
接收的入站RTP 流 的统计信息
{
"id": "IT01A3995811649",
"timestamp": 1683357825557.409,
"type": "inbound-rtp",
"ssrc": 3995811649,
"kind": "audio",
"transportId": "T01",
"codecId": "CIT01_111_minptime=10;useinbandfec=1",
"mediaType": "audio",
"jitter": 0.002,
"packetsLost": 0,
"trackIdentifier": "1d9cbb19-de56-47f5-8229-e164f52d6087",
"mid": "0",
"remoteId": "ROA3995811649",
"packetsReceived": 137255,
"packetsDiscarded": 0,
"fecPacketsReceived": 0,
"fecPacketsDiscarded": 0,
"bytesReceived": 9591106,
"headerBytesReceived": 3843140,
"lastPacketReceivedTimestamp": 1683357825553,
"jitterBufferDelay": 5964614.4,
"jitterBufferTargetDelay": 5031878.4,
"jitterBufferMinimumDelay": 5031724.8,
"jitterBufferEmittedCount": 131763840,
"totalSamplesReceived": 131766720,
"concealedSamples": 51708,
"silentConcealedSamples": 0,
"concealmentEvents": 49,
"insertedSamplesForDeceleration": 17514,
"removedSamplesForAcceleration": 66296,
"audioLevel": 0.003357036042359691,
"totalAudioEnergy": 5.161005023615497,
"totalSamplesDuration": 2745.1400000134995,
"playoutId": "AP",
"estimatedPlayoutTimestamp": 3892346625519
}
outbound-rtp
当前 出站RTP 流RTCPeerConnection的统计信息
{
"id": "OT01A3190629150",
"timestamp": 1683357817557.122,
"type": "outbound-rtp",
"ssrc": 3190629150,
"kind": "audio",
"transportId": "T01",
"codecId": "COT01_111_minptime=10;useinbandfec=1",
"mediaType": "audio",
"mediaSourceId": "SA5",
"remoteId": "RIA3190629150",
"mid": "0",
"packetsSent": 136856,
"retransmittedPacketsSent": 0,
"bytesSent": 9595462,
"headerBytesSent": 3831968,
"retransmittedBytesSent": 0,
"targetBitrate": 32000,
"totalPacketSendDelay": 0,
"nackCount": 0,
"active": true
}
{
"id": "OT01V1674542192",
"timestamp": 1683357817557.122,
"type": "outbound-rtp",
"ssrc": 1674542192,
"kind": "video",
"transportId": "T01",
"codecId": "COT01_96",
"mediaType": "video",
"mediaSourceId": "SV6",
"remoteId": "RIV1674542192",
"mid": "1",
"packetsSent": 403325,
"retransmittedPacketsSent": 0,
"bytesSent": 414276128,
"headerBytesSent": 9834468,
"retransmittedBytesSent": 0,
"targetBitrate": 1700000,
"framesEncoded": 58615,
"keyFramesEncoded": 23,
"totalEncodeTime": 290.888,
"totalEncodedBytesTarget": 0,
"frameWidth": 640,
"frameHeight": 480,
"framesPerSecond": 30,
"framesSent": 58615,
"hugeFramesSent": 2,
"totalPacketSendDelay": 1.874777,
"qualityLimitationReason": "none",
"qualityLimitationDurations": {
"other": 0,
"cpu": 0,
"bandwidth": 18.133,
"none": 2718.968
},
"qualityLimitationResolutionChanges": 0,
"encoderImplementation": "libvpx",
"firCount": 0,
"pliCount": 1,
"nackCount": 0,
"qpSum": 442965,
"active": true,
"powerEfficientEncoder": false,
"scalabilityMode": "L1T1"
}
remote-inbound-rtp
当前与此对象一起发送的出站流相对应的 远程端点入站RTP 流RTCPeerConnection的统计信息。
{
"id": "RIA3190629150",
"timestamp": 1683357813812,
"type": "remote-inbound-rtp",
"ssrc": 3190629150,
"kind": "audio",
"transportId": "T01",
"codecId": "COT01_111_minptime=10;useinbandfec=1",
"jitter": 0.000125,
"packetsLost": 0,
"localId": "OT01A3190629150",
"roundTripTime": 0.002,
"fractionLost": 0,
"totalRoundTripTime": 0.828,
"roundTripTimeMeasurements": 544
}
media-source
拿到当前的媒体流信息,比如音视频流
{
"id": "SA5",
"timestamp": 1683357825557.409,
"type": "media-source",
"trackIdentifier": "116dbc07-9218-45b8-b0a5-2d00a45a4268",
"kind": "audio",
"audioLevel": 0.002685628833887753,
"totalAudioEnergy": 5.348593224408951,
"totalSamplesDuration": 2745.1400000134995,
"echoReturnLoss": -30,
"echoReturnLossEnhancement": 0.2807721793651581
}
{
"id": "SV6",
"timestamp": 1683357825557.409,
"type": "media-source",
"trackIdentifier": "07a011d2-d980-4272-bfaf-7f7246b0d5d3",
"kind": "video",
"width": 640,
"height": 480,
"frames": 58859,
"framesPerSecond": 30
}
网络质量
调用API自带的监控, 将音视频的数据塞在网络监控时进行上传 ,即可同步网络质量和其他数据 eg、声网自带的网络质量监控,两秒钟会触发一次
本地发送的音频数据
使用**type==='audio'**
的outbound数据进行计算
StatsBasicParameter {
id: string
kind: 'video' | 'audio'
timestamp: number
hasAudio?: boolean
hasVideo?: boolean
}
interface LocalAudioStats extends StatsBasicParameter {
bytesSent: number // 发送的字节数
CodecType: string
RecordingLevel: number // 记录音量
SamplingRate: string // 采样率
SendBits: number // 发送数据长度
SendBitrate: number // 发送码率
SendLevel: number // 发送音量
packetsLost: number // 丢包数
PacketLossRate: number // 丢包率
roundTripTime: number // rtt时间
packetsSent: number // 发送包数
}
SendBits = outbountRtp.bytesSent * 8 ,(一个字节有8位)
- 原始数据里面没有码率,需要自己计算:
- 码率 = 数据大小/时间端 。 ps:因为数据长度是累加的,所以需要取出区间的大小
SendBitrate = (outbountRtp.bytesSent * 8 - (prevOutbountRtp.SendBits || 0)) / timeGap
packetsLost = remoteInboundRtp.packetsLost
roundTripTime = remoteInboundRtp.roundTripTime
本地发送的视频数据
使用**type==='video'**
的outbound数据进行计算
interface LocalVideoStats extends StatsBasicParameter {
CodecType: string
CaptureFrameRate: number // 当前设置帧率
CaptureResolutionHeight: number // 当前设定分辨率 - 高
CaptureResolutionWidth: number // 当前设定分辨率 - 宽
totalEncodeTime: number // 编码时间
SendBits: number // 发送数据长度
SendBitrate: number // 发送码率
SendFrame: number // 发送帧数
SendFrameRate: number // 发送帧率
SendResolutionHeight: number
SendResolutionWidth: number
packetsLost: number
packetsSent: number
PacketLossRate: number
roundTripTime: number
rid?: string
}
- 码率计算方式如上;
SendFrame = outbountRtp.framesSent
- 发送帧率和码率的计算方式类似:
SendFrameRate =
+(
(outboundRtp.framesSent - (prevStats.SendFrame || 0)) /
this.timeGap
).toFixed(3) * 1000
- 丢包数要用远端接收流remoteInboundRtp的数据来计算
packetsLost = remoteInboundRtp.packetsLost
roundTripTime = remoteInboundRtp.roundTripTime
- 丢包率:
const pl = Math.abs(remoteInboundRtp.packetsLost - prevStats.packetsLost)
const pr = Math.abs(outboundRtp.packetsSent - prevStats.packetsSent)
PacketLossRate = +(pl / (pl + pr)).toFixed(3)
远端传过来的音频数据
通过inboundRtp来计算,和本地音频流一致
远端传过来的视频数据
CodecType: string
PacketLossRate: number
RecvBits: number
RecvBitrate: number
RecvResolutionHeight: number
RecvResolutionWidth: number
RenderFrameRate: number //渲染帧率
framesDecoded: number
packetsReceived: number
packetsLost: number
jitter: number
- 渲染帧率
result.RenderFrameRate =
+(
(stats.rtp.framesDecoded - (prevStats.framesDecoded || 0)) /
this.timeGap
).toFixed(3) * 1000