一个正常的url设置到播放器中。播放器一般会经历如下的流程:
- 请求url
- 解封装
- 分离音频、视频、字幕轨道
- 起一个线程解码音频,同时起一个线程解码视频,然后做好音视频同步
- 音视频使用AudioTrack或者OpenSL ES播放,视频使用TextureView或者SurfaceView渲染
上一篇文章已经分析了ExoPlayer是如何请求url的,我们请求得到了一定的数据,就要对源数据进行解封装。解封装的前提要知道视频是什么封装格式的? 探知视频封装格式的过程就是Extractor。本文主要分析ExoPlayer的Extractor模块。
1. ExoPlayer支持的封装格式
主要看DefaultExtractorsFactory.java类中的createExtractors方法:
public synchronized Extractor[] createExtractors() {
Extractor[] extractors = new Extractor[14];
extractors[0] = new MatroskaExtractor(matroskaFlags);
extractors[1] = new FragmentedMp4Extractor(fragmentedMp4Flags);
extractors[2] = new Mp4Extractor(mp4Flags);
extractors[3] =
new Mp3Extractor(
mp3Flags
| (constantBitrateSeekingEnabled
? Mp3Extractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING
: 0));
extractors[4] =
new AdtsExtractor(
adtsFlags
| (constantBitrateSeekingEnabled
? AdtsExtractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING
: 0));
extractors[5] = new Ac3Extractor();
extractors[6] = new TsExtractor(tsMode, tsFlags);
extractors[7] = new FlvExtractor();
extractors[8] = new OggExtractor();
extractors[9] = new PsExtractor();
extractors[10] = new WavExtractor();
extractors[11] =
new AmrExtractor(
amrFlags
| (constantBitrateSeekingEnabled
? AmrExtractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING
: 0));
extractors[12] = new Ac4Extractor();
if (FLAC_EXTENSION_EXTRACTOR_CONSTRUCTOR != null) {
try {
extractors[13] = FLAC_EXTENSION_EXTRACTOR_CONSTRUCTOR.newInstance();
} catch (Exception e) {
// Should never happen.
throw new IllegalStateException("Unexpected error creating FLAC extractor", e);
}
} else {
extractors[13] = new FlacExtractor();
}
return extractors;
}
- MatroskaExtractor:Extracts data from the Matroska and WebM container formats
- FragmentedMp4Extractor:Extracts data from the FMP4 container format
- Mp4Extractor:Extracts data from the MP4 container format
- Mp3Extractor:Extracts data from the MP3 container format
- AdtsExtractor:Extracts data from AAC bit streams with ADTS framing
- Ac3Extractor:Extracts data from (E-)AC-3 bitstreams
- TsExtractor:Extracts data from the MPEG-2 TS container format
- FlvExtractor:Extracts data from the FLV container format
- OggExtractor:Extracts data from the Ogg container format
- PsExtractor:Extracts data from the MPEG-2 PS container format
- WavExtractor:Extracts data from WAV byte streams
- AmrExtractor:Extracts data from the AMR containers format (either AMR or AMR-WB)
- Ac4Extractor:Extracts data from AC-4 bitstreams
- FlacExtractor:Extracts data from FLAC container format
2.Extractor过程
Extractor的接口有如下方法:
public interface Extractor {
int RESULT_CONTINUE = 0;
int RESULT_SEEK = 1;
int RESULT_END_OF_INPUT = C.RESULT_END_OF_INPUT;
boolean sniff(ExtractorInput input) throws IOException, InterruptedException;
void init(ExtractorOutput output);
int read(ExtractorInput input, PositionHolder seekPosition)
throws IOException, InterruptedException;
void seek(long position, long timeUs);
void release();
}
- sniff函数:嗅探视频封装格式的函数,这个函数主要读取视频文件中的二进制数据,和封装格式的协议比较。
- init函数:已经知道了封装格式,开始初始化具体的封装Extractor,因为接下来要准备读取视频中的各个轨道数据了。
- read函数:读取具体的视频数据。
探测视频封装格式的地方在ProgressiveMediaPeriod.java中的ExtractorHolder内部类。
public Extractor selectExtractor(ExtractorInput input, ExtractorOutput output, Uri uri)
throws IOException, InterruptedException {
if (extractor != null) {
return extractor;
}
if (extractors.length == 1) {
this.extractor = extractors[0];
} else {
for (Extractor extractor : extractors) {
try {
if (extractor.sniff(input)) {
this.extractor = extractor;
break;
}
} catch (EOFException e) {
// Do nothing.
} finally {
input.resetPeekPosition();
}
}
if (extractor == null) {
throw new UnrecognizedInputFormatException(
"None of the available extractors ("
+ Util.getCommaDelimitedSimpleClassNames(extractors)
+ ") could read the stream.",
uri);
}
}
extractor.init(output);
return extractor;
}
遍历extractors数组,发现符合特定格式的规则之后,就认为它是这种封装格式。
以Mp4Extractor为例分析一下ExoPlayer内部如何识别特定的封装格式。
- extractor.sniff 调用到 Mp4Extractor.sniff中
- 然后调用到Sniffer.sniffUnfragmented中,Sniffer.sniffInternal
在Sniffer.sniffInternal中有探测是否是MP4视频的代码。
这儿有一个问题:MP4视频是有排列顺序的,主要是moov和mdat,moov表示的视频相关的属性集合,mdat是具体的音视频数据。必须要先读到moov数据才能正确解析mdat数据。否则无法正常播放MP4视频
正常情况下moov在mdat之前。加入moov就在mdat之后,播放器很容易看到解析的过程。
在Sniffer.sniffInternal函数中:
这就是双IO检测机制。对解决MP4视频的moov位置很有帮组。
获取了正确的Extractor,可以开始读取视频数据了,现在我们回到ProgressiveMediaPeriod.load方法:
while (result == Extractor.RESULT_CONTINUE && !loadCanceled) {
loadCondition.block();
result = extractor.read(input, positionHolder);
if (input.getPosition() > position + continueLoadingCheckIntervalBytes) {
position = input.getPosition();
loadCondition.close();
handler.post(onContinueLoadingRequestedRunnable);
}
}
onContinueLoadingRequestedRunnable =
() -> {
if (!released) {
Assertions.checkNotNull(callback)
.onContinueLoadingRequested(ProgressiveMediaPeriod.this);
}
};
加载数据都放在子线程中完成。
这个extractor.read操作,对应Mp4Extractor.read
public int read(ExtractorInput input, PositionHolder seekPosition)
throws IOException, InterruptedException {
while (true) {
switch (parserState) {
case STATE_READING_ATOM_HEADER:
if (!readAtomHeader(input)) {
return RESULT_END_OF_INPUT;
}
break;
case STATE_READING_ATOM_PAYLOAD:
if (readAtomPayload(input, seekPosition)) {
return RESULT_SEEK;
}
break;
case STATE_READING_SAMPLE:
return readSample(input, seekPosition);
default:
throw new IllegalStateException();
}
}
}
- STATE_READING_ATOM_HEADER 开始读atom header信息
- STATE_READING_ATOM_PAYLOAD 如果atom没有读完全,播放器会将parserState设置为STATE_READING_ATOM_PAYLOAD
- STATE_READING_SAMPLE atom读完,开始读媒体文件原始数据
readAtomHeader : 读取MP4文件的头部
readAtomPayload:atom没有读完,继续请求匹配atom header
readSample:开始读取moov的解析出的track信息