基于Android P版本分析
音视频数据收集
AudioRecord 概述
我们常见的录音使用到的Audio API有两种：MediaRecorder和AudioRecord，这两种方式都可以用于采集音频数据，但是有一定的区别：
| 优点 | 缺点 | |
|---|---|---|
| MediaRecorder（基于文件录音） | 封装了AudioRecord和MediaCodec，操作简单 | 无法实现实处理音频 |
| AudioRecord（基于字节流录音） | 实现了音频的实时处理，进行边录边播 | 输出的数据格式为PCM音频原始数据，保存生成的音频文件不能使用常见的播放器播放，需要使用配对的AudioTrack进度读取播放 |
我们分别对者两种数据采集方式做分析，因为MediaRecorder中集成了AudioRecord了，所以先描述AudioRecord；
Audio 参数概念
声音其实是一种能量波，声波的三要素是频率、振幅和波形，频率代表音阶的高低，振幅代表响度，波形代表音色；
采样率（sampleRate）
每秒钟内取得声音样本的次数；
声波是无限光滑的，波形可以看做无数点组成的，很显然，点数越多，波形就会越光滑，相对的获取的频率信息越丰富，即采样频率越高，声音的质量就越好，声音还原越真实，但相应的空间占用率也大；
人耳对于声音分别率其实也有一定的范围，我们常见的几种声音分别率：
- 8000 Hz：电话所用采样率，对于人的说话已经足够
- 11025 Hz：电话所用采样率
- 22050 Hz：无线电广播所用采样率
- 32000 Hz：miniDV 数码视频 camcorder、DAT (LP mode)所用采样率
- 44100 Hz：音频 CD, 也常用于 MPEG-1 音频（VCD，SVCD，MP3）所用采样率
- 47250 Hz：商用 PCM 录音机所用采样率
- 48000 Hz：miniDV、数字电视、DVD、DAT、电影和专业音频所用的数字声音所用采样率
- 50000 Hz：商用数字录音机所用采样率
- 96000 Hz或者 192000 Hz：DVD-Audio、一些 LPCM DVD 音轨、BD-ROM（蓝光盘）音轨、和 HD-DVD （高清晰度 DVD）音轨所用所用采样率
采样深度（encoding）
采样深度也是一个重要的属性，代表了每个样本的细节水平或质量，简单的理解为样本精度和样本大小，每一帧数据的大小；
按照声波的要素，采样深度对应振幅，在数字录音中，有越多不同的振幅，数字录音听起来就越接近原生；
我们常见的采样深度范围：
- 4-bit：16个数值，24dB的动态范围。有时也会用于极低保真的“bitcrushed”效果器上
- 8-bit：256个数值，48dB的动态范围。经常用于早期的经典的视频游戏系统
- 16-bit：65536个数值，96dB的动态范围，CD音频的标准比特深度
- 24-bit：16777216个数值，145dB的动态范围，最常用的比特深度
- 32或者 64-bit：“浮点”，目前可以做到提供最佳信噪比的数值，但是尚未被广泛采用
比特率（bps）
比特率一般也称为码率，指每秒播放的数据量，单位用bit表示，其实和采样率、采样深度联系在一起，比特率 = 采样率 * 采样深度，采样率是以每秒的样本数来衡量的，而采样深度是以每个样本的比特数来衡量的；
通道数（channel）
我们常见的耳机分为了左右声道，其实这个就是指的通道数，左右声道为双声道（立体声）。
由于音频的采集和播放是可以同时进行的，因此可以同时从多个音频源采集数据，并分别输出到不同的扬声器，故声道数一般表示声音录制时的音源数据或回放时相应的扬声器数据量；
我们常见的通道数：
- 单声道：单声道是比较原始的声音复制形式，早期的声卡采用的比较普遍。单声道的声音只能使用一个扬声器发声，有的也处理成两个扬声器输出同一个声道的声音，当通过两个扬声器回放单声道信息的时候，我们可以明显感觉到声音是从两个音箱中间传递到我们耳朵里的，无法判断声源的具体位置
- 双声道：双声道就是有两个声音通道，其原理是人们听到声音时可以根据左耳和右耳对声音相位差来判断声源的具体位置。声音在录制过程中被分配到两个独立的声道，从而达到了很好的声音定位效果
AudioRecord API
Java
| Method Name | Description |
|---|---|
| release | 释放AudioRecord资源，release之后AudioRecord置为null |
| getSampleRate | 返回采样率Hz |
| getAudioSource | 返回录音源信息 |
| getAudioFormat | 获取音频数据格式 |
| getChannelConfiguration | 获取声道配置 |
| getFormat | 获取流的音频格式 |
| getChannelCount | 获取声道数 |
| getState | 获取实例初始化状态 |
| getRecordingState | 获取AudioRecord实例的录制状态 |
| getBufferSizeInFrames | 获取流的缓冲区大小 |
| getNotificationMarkerPosition | 获取通知标志位 |
| getPositionNotificationPeriod | 标志位更新时间 |
| getTimestamp | 获取时间戳标志 |
| getMinBufferSize | 获取创建AudioRecord对象所需要的最小缓冲区大小 |
| getAudioSessionId | 当前AudioRecord的属于的任务id |
| startRecording | 开启录音 |
| stop | 停止录音 |
| read | 读取录音数据到指定的内存中 |
| setRecordPositionUpdateListener | 设置回调监听 |
| setNotificationMarkerPosition | 设置通知标志 |
| getRoutedDevice | 获取当前AudioTrack的路由id信息 |
| addOnRoutingChangedListener | 添加音频路由状态监听 |
| removeOnRoutingChangedListener | 移除音频路由状态监听 |
| setPreferredDevice | 为输出流音频路由设置audio device |
| getPreferredDevice | 获取输出流音频路由指定的audio device |
| getActiveMicrophones | 返回设备中可用的麦克风的数组 |
我们首先先看一个AudioRecord的使用实例，根据实例中的步骤，逐一分析AudioRecord的执行逻辑；
AudioRecord使用实例
// 音频源：音频输入-麦克风
private final static int AUDIO_INPUT = MediaRecorder.AudioSource.MIC;
// 采样率
// 44100是目前的标准，但是某些设备仍然支持22050，16000，11025
// 采样频率一般共分为22.05KHz、44.1KHz、48KHz三个等级
private final static int AUDIO_SAMPLE_RATE = 16000;
// 音频通道 单声道
private final static int AUDIO_CHANNEL = AudioFormat.CHANNEL_IN_MONO;
// 音频格式：PCM编码
private final static int AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT;
private AudioRecord audioRecord = null; // 声明 AudioRecord 对象
private int recordBufSize = 0; // 声明recoordBufffer的大小字段
private byte[] buffer;
private boolean isRecording;
private void stop(){
isRecording = false;
if (null != audioRecord) {
audioRecord.stop();
audioRecord.release();
audioRecord = null;
}
}
private void start(){
//audioRecord能接受的最小的buffer大小
recordBufSize = AudioRecord.getMinBufferSize(AUDIO_SAMPLE_RATE, AUDIO_CHANNEL, AUDIO_ENCODING);
audioRecord = new AudioRecord(AUDIO_INPUT, AUDIO_SAMPLE_RATE, AUDIO_CHANNEL, AUDIO_ENCODING, recordBufSize);
buffer = new byte[recordBufSize];
audioRecord.startRecording();
isRecording = true;
new Thread(() -> {
FileOutputStream os = null;
try {
if(!new File(pcmFileName).exists()){
new File(pcmFileName).createNewFile();
}
os = new FileOutputStream(pcmFileName);
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
if (null != os) {
while (isRecording) {
int read = audioRecord.read(buffer, 0, recordBufSize);
// 如果读取音频数据没有出现错误，就将数据写入到文件
if (AudioRecord.ERROR_INVALID_OPERATION != read) {
try {
os.write(buffer);
} catch (IOException e) {
e.printStackTrace();
}
}
}
try {
os.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
AudioRecord
AudioRecord构建
AudioRecord的构建，其实也会从Java层一直调用到native service，我们分别对每一个层的构建流程进行分析；
Java - JNI AudioRecord
public AudioRecord(int audioSource, int sampleRateInHz, int channelConfig, int audioFormat,
int bufferSizeInBytes)
throws IllegalArgumentException {
this((new AudioAttributes.Builder())
.setInternalCapturePreset(audioSource)
.build(),
(new AudioFormat.Builder())
.setChannelMask(getChannelMaskFromLegacyConfig(channelConfig,
true/*allow legacy configurations*/))
.setEncoding(audioFormat)
.setSampleRate(sampleRateInHz)
.build(),
bufferSizeInBytes,
AudioManager.AUDIO_SESSION_ID_GENERATE);
}
@SystemApi
public AudioRecord(AudioAttributes attributes, AudioFormat format, int bufferSizeInBytes,
int sessionId) throws IllegalArgumentException {
mRecordingState = RECORDSTATE_STOPPED;
if (attributes == null) {
throw new IllegalArgumentException("Illegal null AudioAttributes");
}
if (format == null) {
throw new IllegalArgumentException("Illegal null AudioFormat");
}
// remember which looper is associated with the AudioRecord instanciation
if ((mInitializationLooper = Looper.myLooper()) == null) {
mInitializationLooper = Looper.getMainLooper();
}
// is this AudioRecord using REMOTE_SUBMIX at full volume?
if (attributes.getCapturePreset() == MediaRecorder.AudioSource.REMOTE_SUBMIX) {
final AudioAttributes.Builder filteredAttr = new AudioAttributes.Builder();
final Iterator<String> tagsIter = attributes.getTags().iterator();
while (tagsIter.hasNext()) {
final String tag = tagsIter.next();
if (tag.equalsIgnoreCase(SUBMIX_FIXED_VOLUME)) {
mIsSubmixFullVolume = true;
Log.v(TAG, "Will record from REMOTE_SUBMIX at full fixed volume");
} else { // SUBMIX_FIXED_VOLUME: is not to be propagated to the native layers
filteredAttr.addTag(tag);
}
}
filteredAttr.setInternalCapturePreset(attributes.getCapturePreset());
mAudioAttributes = filteredAttr.build();
} else {
mAudioAttributes = attributes;
}
int rate = format.getSampleRate();
if (rate == AudioFormat.SAMPLE_RATE_UNSPECIFIED) {
rate = 0;
}
int encoding = AudioFormat.ENCODING_DEFAULT;
if ((format.getPropertySetMask() & AudioFormat.AUDIO_FORMAT_HAS_PROPERTY_ENCODING) != 0)
{
encoding = format.getEncoding();
}
audioParamCheck(attributes.getCapturePreset(), rate, encoding);
if ((format.getPropertySetMask()
& AudioFormat.AUDIO_FORMAT_HAS_PROPERTY_CHANNEL_INDEX_MASK) != 0) {
mChannelIndexMask = format.getChannelIndexMask();
mChannelCount = format.getChannelCount();
}
if ((format.getPropertySetMask()
& AudioFormat.AUDIO_FORMAT_HAS_PROPERTY_CHANNEL_MASK) != 0) {
mChannelMask = getChannelMaskFromLegacyConfig(format.getChannelMask(), false);
mChannelCount = format.getChannelCount();
} else if (mChannelIndexMask == 0) {
mChannelMask = getChannelMaskFromLegacyConfig(AudioFormat.CHANNEL_IN_DEFAULT, false);
mChannelCount = AudioFormat.channelCountFromInChannelMask(mChannelMask);
}
audioBuffSizeCheck(bufferSizeInBytes);
int[] sampleRate = new int[] {mSampleRate};
int[] session = new int[1];
session[0] = sessionId;
//TODO: update native initialization when information about hardware init failure
// due to capture device already open is available.
int initResult = native_setup( new WeakReference<AudioRecord>(this),
mAudioAttributes, sampleRate, mChannelMask, mChannelIndexMask,
mAudioFormat, mNativeBufferSizeInBytes,
session, ActivityThread.currentOpPackageName(), 0 /*nativeRecordInJavaObj*/);
if (initResult != SUCCESS) {
loge("Error code "+initResult+" when initializing native AudioRecord object.");
return; // with mState == STATE_UNINITIALIZED
}
mSampleRate = sampleRate[0];
mSessionId = session[0];
mState = STATE_INITIALIZED;
}
和AudioTrack的构造方法类似，都是在AudioRecord的构造方法中，通过传入的参数创建了对应的AudioAttribute对象和AudioFormat对象；
然后在该构造方法最后，调用了JNI层的native_setup()方法，其中传入了mAudioAttribute、sampleRate、mChannelMask、mAudioFormat等一些配置信息，其中mAudioAttribute和mAudioFormat都是通过传入的参数进行了一系列的判断，最终确定的设备或者服务支持的最符合用于要求的条件组合；
static jint
android_media_AudioRecord_setup(JNIEnv *env, jobject thiz, jobject weak_this,
jobject jaa, jintArray jSampleRate, jint channelMask, jint channelIndexMask,
jint audioFormat, jint buffSizeInBytes, jintArray jSession, jstring opPackageName,
jlong nativeRecordInJavaObj)
{
//ALOGV(">> Entering android_media_AudioRecord_setup");
//ALOGV("sampleRate=%d, audioFormat=%d, channel mask=%x, buffSizeInBytes=%d "
// "nativeRecordInJavaObj=0x%llX",
// sampleRateInHertz, audioFormat, channelMask, buffSizeInBytes, nativeRecordInJavaObj);
audio_channel_mask_t localChanMask = inChannelMaskToNative(channelMask);
if (jSession == NULL) {
ALOGE("Error creating AudioRecord: invalid session ID pointer");
return (jint) AUDIO_JAVA_ERROR;
}
jint* nSession = (jint *) env->GetPrimitiveArrayCritical(jSession, NULL);
if (nSession == NULL) {
ALOGE("Error creating AudioRecord: Error retrieving session id pointer");
return (jint) AUDIO_JAVA_ERROR;
}
audio_session_t sessionId = (audio_session_t) nSession[0];
env->ReleasePrimitiveArrayCritical(jSession, nSession, 0);
nSession = NULL;
audio_attributes_t *paa = NULL;
sp<AudioRecord> lpRecorder = 0;
audiorecord_callback_cookie *lpCallbackData = NULL;
jclass clazz = env->GetObjectClass(thiz);
if (clazz == NULL) {
ALOGE("Can't find %s when setting up callback.", kClassPathName);
return (jint) AUDIORECORD_ERROR_SETUP_NATIVEINITFAILED;
}
// if we pass in an existing *Native* AudioRecord, we don't need to create/initialize one.
if (nativeRecordInJavaObj == 0) {
if (jaa == 0) {
ALOGE("Error creating AudioRecord: invalid audio attributes");
return (jint) AUDIO_JAVA_ERROR;
}
if (jSampleRate == 0) {
ALOGE("Error creating AudioRecord: invalid sample rates");
return (jint) AUDIO_JAVA_ERROR;
}
jint elements[1];
env->GetIntArrayRegion(jSampleRate, 0, 1, elements);
int sampleRateInHertz = elements[0];
// channel index mask takes priority over channel position masks.
if (channelIndexMask) {
// Java channel index masks need the representation bits set.
localChanMask = audio_channel_mask_from_representation_and_bits(
AUDIO_CHANNEL_REPRESENTATION_INDEX,
channelIndexMask);
}
// Java channel position masks map directly to the native definition
if (!audio_is_input_channel(localChanMask)) {
ALOGE("Error creating AudioRecord: channel mask %#x is not valid.", localChanMask);
return (jint) AUDIORECORD_ERROR_SETUP_INVALIDCHANNELMASK;
}
uint32_t channelCount = audio_channel_count_from_in_mask(localChanMask);
// compare the format against the Java constants
audio_format_t format = audioFormatToNative(audioFormat);
if (format == AUDIO_FORMAT_INVALID) {
ALOGE("Error creating AudioRecord: unsupported audio format %d.", audioFormat);
return (jint) AUDIORECORD_ERROR_SETUP_INVALIDFORMAT;
}
size_t bytesPerSample = audio_bytes_per_sample(format);
if (buffSizeInBytes == 0) {
ALOGE("Error creating AudioRecord: frameCount is 0.");
return (jint) AUDIORECORD_ERROR_SETUP_ZEROFRAMECOUNT;
}
size_t frameSize = channelCount * bytesPerSample;
size_t frameCount = buffSizeInBytes / frameSize;
ScopedUtfChars opPackageNameStr(env, opPackageName);
// create an uninitialized AudioRecord object
lpRecorder = new AudioRecord(String16(opPackageNameStr.c_str()));
// read the AudioAttributes values
paa = (audio_attributes_t *) calloc(1, sizeof(audio_attributes_t));
const jstring jtags =
(jstring) env->GetObjectField(jaa, javaAudioAttrFields.fieldFormattedTags);
const char* tags = env->GetStringUTFChars(jtags, NULL);
// copying array size -1, char array for tags was calloc'd, no need to NULL-terminate it
strncpy(paa->tags, tags, AUDIO_ATTRIBUTES_TAGS_MAX_SIZE - 1);
env->ReleaseStringUTFChars(jtags, tags);
paa->source = (audio_source_t) env->GetIntField(jaa, javaAudioAttrFields.fieldRecSource);
paa->flags = (audio_flags_mask_t)env->GetIntField(jaa, javaAudioAttrFields.fieldFlags);
ALOGV("AudioRecord_setup for source=%d tags=%s flags=%08x", paa->source, paa->tags, paa->flags);
// 初始化input flag，和output flag有相似之处
audio_input_flags_t flags = AUDIO_INPUT_FLAG_NONE;
if (paa->flags & AUDIO_FLAG_HW_HOTWORD) {
flags = AUDIO_INPUT_FLAG_HW_HOTWORD;
}
// create the callback information:
// this data will be passed with every AudioRecord callback
lpCallbackData = new audiorecord_callback_cookie;
lpCallbackData->audioRecord_class = (jclass)env->NewGlobalRef(clazz);
// we use a weak reference so the AudioRecord object can be garbage collected.
lpCallbackData->audioRecord_ref = env->NewGlobalRef(weak_this);
lpCallbackData->busy = false;
const status_t status = lpRecorder->set(paa->source,
sampleRateInHertz,
format, // word length, PCM
localChanMask,
frameCount,
recorderCallback,// callback_t
lpCallbackData,// void* user
0, // notificationFrames,
true, // threadCanCallJava
sessionId,
AudioRecord::TRANSFER_DEFAULT,
flags,
-1, -1, // default uid, pid
paa);
if (status != NO_ERROR) {
ALOGE("Error creating AudioRecord instance: initialization check failed with status %d.",
status);
goto native_init_failure;
}
} else { // end if nativeRecordInJavaObj == 0)
……………………
}
……………………
}
和AudioTrack的创建流程非常类似，都是先创建Native层的AudioRecord对象，然后通过调用AudioRecord.cpp中的set()函数，传入在该JNI函数中配置好的参数，例如AudioAttribute对象、sampleRateInHertz、frameCount等一些参数，进行Native层的AudioRecord对象的初始化；
AudioRecord初始化成功之后，保存到JNI中；
Native AudioRecord
AudioRecord::AudioRecord(const String16 &opPackageName)
: mActive(false), mStatus(NO_INIT), mOpPackageName(opPackageName),
mSessionId(AUDIO_SESSION_ALLOCATE),
mPreviousPriority(ANDROID_PRIORITY_NORMAL), mPreviousSchedulingGroup(SP_DEFAULT),
mSelectedDeviceId(AUDIO_PORT_HANDLE_NONE), mRoutedDeviceId(AUDIO_PORT_HANDLE_NONE)
{
}
对AudioRecord中的一些变量进行初始化，初始化的值为默认值；
Native AudioRecord::set
status_t AudioRecord::set(
audio_source_t inputSource, // 音频输入源，在应用层，一般采用的是MIC作为音频源输入
uint32_t sampleRate,
audio_format_t format,
audio_channel_mask_t channelMask,
size_t frameCount,
callback_t cbf,
void* user,
uint32_t notificationFrames,
bool threadCanCallJava,
audio_session_t sessionId,
transfer_type transferType, // AudioRecord::TRANSFER_DEFAULT
audio_input_flags_t flags,
uid_t uid,
pid_t pid,
const audio_attributes_t* pAttributes, // paa，android_media_AudioRecord_setup()函数中重新定义的AudioAttribute对象，包含了更新后的AudioAttribute信息，不是应用层完全传入的配置
audio_port_handle_t selectedDeviceId)
{
status_t status = NO_ERROR;
uint32_t channelCount;
pid_t callingPid;
pid_t myPid;
ALOGV("set(): inputSource %d, sampleRate %u, format %#x, channelMask %#x, frameCount %zu, "
"notificationFrames %u, sessionId %d, transferType %d, flags %#x, opPackageName %s "
"uid %d, pid %d",
inputSource, sampleRate, format, channelMask, frameCount, notificationFrames,
sessionId, transferType, flags, String8(mOpPackageName).string(), uid, pid);
mSelectedDeviceId = selectedDeviceId;
switch (transferType) {
case TRANSFER_DEFAULT:
if (cbf == NULL || threadCanCallJava) {
transferType = TRANSFER_SYNC;
} else {
transferType = TRANSFER_CALLBACK;
}
break;
case TRANSFER_CALLBACK:
if (cbf == NULL) {
ALOGE("Transfer type TRANSFER_CALLBACK but cbf == NULL");
status = BAD_VALUE;
goto exit;
}
break;
case TRANSFER_OBTAIN:
case TRANSFER_SYNC:
break;
default:
ALOGE("Invalid transfer type %d", transferType);
status = BAD_VALUE;
goto exit;
}
mTransfer = transferType;
// invariant that mAudioRecord != 0 is true only after set() returns successfully
if (mAudioRecord != 0) {
ALOGE("Track already in use");
status = INVALID_OPERATION;
goto exit;
}
if (pAttributes == NULL) {
memset(&mAttributes, 0, sizeof(audio_attributes_t));
mAttributes.source = inputSource;
} else {
// stream type shouldn't be looked at, this track has audio attributes
// 将JNI层传入的AudioAttribute赋值给AudioRecord中定义的mAttribute对象
memcpy(&mAttributes, pAttributes, sizeof(audio_attributes_t));
ALOGV("Building AudioRecord with attributes: source=%d flags=0x%x tags=[%s]",
mAttributes.source, mAttributes.flags, mAttributes.tags);
}
mSampleRate = sampleRate;
// these below should probably come from the audioFlinger too...
if (format == AUDIO_FORMAT_DEFAULT) {
format = AUDIO_FORMAT_PCM_16_BIT;
}
// validate parameters
// AudioFlinger capture only supports linear PCM
if (!audio_is_valid_format(format) || !audio_is_linear_pcm(format)) {
ALOGE("Format %#x is not linear pcm", format);
status = BAD_VALUE;
goto exit;
}
mFormat = format;
// 声道检查
if (!audio_is_input_channel(channelMask)) {
ALOGE("Invalid channel mask %#x", channelMask);
status = BAD_VALUE;
goto exit;
}
mChannelMask = channelMask;
channelCount = audio_channel_count_from_in_mask(channelMask);
mChannelCount = channelCount;
// 音频帧长度
if (audio_is_linear_pcm(format)) {
mFrameSize = channelCount * audio_bytes_per_sample(format);
} else {
mFrameSize = sizeof(uint8_t);
}
// mFrameCount is initialized in createRecord_l
mReqFrameCount = frameCount;
mNotificationFramesReq = notificationFrames;
// mNotificationFramesAct is initialized in createRecord_l
mSessionId = sessionId;
ALOGV("set(): mSessionId %d", mSessionId);
callingPid = IPCThreadState::self()->getCallingPid();
myPid = getpid();
if (uid == AUDIO_UID_INVALID || (callingPid != myPid)) {
mClientUid = IPCThreadState::self()->getCallingUid();
} else {
mClientUid = uid;
}
if (pid == -1 || (callingPid != myPid)) {
mClientPid = callingPid;
} else {
mClientPid = pid;
}
mOrigFlags = mFlags = flags;
mCbf = cbf;
if (cbf != NULL) {
// 如果cbf不为空，则根据threadCanCallJava创建对应的AudioRecordThread线程
mAudioRecordThread = new AudioRecordThread(*this, threadCanCallJava);
mAudioRecordThread->run("AudioRecord", ANDROID_PRIORITY_AUDIO);
// thread begins in paused state, and will not reference us until start()
}
// create the IAudioRecord
status = createRecord_l(0 /*epoch*/, mOpPackageName);
if (status != NO_ERROR) {
if (mAudioRecordThread != 0) {
mAudioRecordThread->requestExit(); // see comment in AudioRecord.h
mAudioRecordThread->requestExitAndWait();
mAudioRecordThread.clear();
}
goto exit;
}
mUserData = user;
// TODO: add audio hardware input latency here
mLatency = (1000LL * mFrameCount) / mSampleRate;
mMarkerPosition = 0;
mMarkerReached = false;
mNewPosition = 0;
mUpdatePeriod = 0;
AudioSystem::acquireAudioSessionId(mSessionId, -1);
mSequence = 1;
mObservedSequence = mSequence;
mInOverrun = false;
mFramesRead = 0;
mFramesReadServerOffset = 0;
exit:
mStatus = status;
if (status != NO_ERROR) {
mMediaMetrics.markError(status, __FUNCTION__);
}
return status;
}
和AudioTrack的set()函数类似，其中同样也执行了几个逻辑：
- set()函数刚进入，就对transferType进行判断，选择数据传输方式，transferType就是我们在AudioTrack中提及到的TRANSFER_SYNC、TRANSFER_SHARED、TRANSFER_CALLBACK和TRANSFER_OBTAIN这四种Transfer Type；
- 对传入的AudioAttribute对象进行判空，如果不为空，则将AudioRecord创建时传入的AudioAttribute对象赋值给AudioRecord.cpp中的mAttribute对象；
- 对一些音频Format参数的初始化，例如mSampleRate、format（该参数代表采样深度）、mChannelMask等一些配置；
- 根据cbf（类型为callback_t，在android_media_AudioRecord.cpp中定义）创建AudioRecordThread对象，用于处理recorderCallback中的回调函数；
- 调用createRecord_l()函数；
其中核心逻辑为调用了createRecord_l()函数：
// must be called with mLock held
status_t AudioRecord::createRecord_l(const Modulo<uint32_t> &epoch, const String16& opPackageName)
{
const sp<IAudioFlinger>& audioFlinger = AudioSystem::get_audio_flinger();
IAudioFlinger::CreateRecordInput input;
IAudioFlinger::CreateRecordOutput output;
audio_session_t originalSessionId;
sp<media::IAudioRecord> record;
void *iMemPointer;
audio_track_cblk_t* cblk;
status_t status;
if (audioFlinger == 0) {
ALOGE("Could not get audioflinger");
status = NO_INIT;
goto exit;
}
// mFlags (not mOrigFlags) is modified depending on whether fast request is accepted.
// After fast request is denied, we will request again if IAudioRecord is re-created.
// Now that we have a reference to an I/O handle and have not yet handed it off to AudioFlinger,
// we must release it ourselves if anything goes wrong.
// Client can only express a preference for FAST. Server will perform additional tests.
if (mFlags & AUDIO_INPUT_FLAG_FAST) {
bool useCaseAllowed =
// any of these use cases:
// use case 1: callback transfer mode
(mTransfer == TRANSFER_CALLBACK) ||
// use case 2: blocking read mode
// The default buffer capacity at 48 kHz is 2048 frames, or ~42.6 ms.
// That's enough for double-buffering with our standard 20 ms rule of thumb for
// the minimum period of a non-SCHED_FIFO thread.
// This is needed so that AAudio apps can do a low latency non-blocking read from a
// callback running with SCHED_FIFO.
(mTransfer == TRANSFER_SYNC) ||
// use case 3: obtain/release mode
(mTransfer == TRANSFER_OBTAIN);
if (!useCaseAllowed) {
ALOGW("AUDIO_INPUT_FLAG_FAST denied, incompatible transfer = %s",
convertTransferToText(mTransfer));
mFlags = (audio_input_flags_t) (mFlags & ~(AUDIO_INPUT_FLAG_FAST |
AUDIO_INPUT_FLAG_RAW));
}
}
input.attr = mAttributes;
input.config.sample_rate = mSampleRate;
input.config.channel_mask = mChannelMask;
input.config.format = mFormat;
input.clientInfo.clientUid = mClientUid;
input.clientInfo.clientPid = mClientPid;
input.clientInfo.clientTid = -1;
if (mFlags & AUDIO_INPUT_FLAG_FAST) {
if (mAudioRecordThread != 0) {
input.clientInfo.clientTid = mAudioRecordThread->getTid();
}
}
input.opPackageName = opPackageName;
input.flags = mFlags;
// The notification frame count is the period between callbacks, as suggested by the client
// but moderated by the server. For record, the calculations are done entirely on server side.
input.frameCount = mReqFrameCount;
input.notificationFrameCount = mNotificationFramesReq;
input.selectedDeviceId = mSelectedDeviceId;
input.sessionId = mSessionId;
originalSessionId = mSessionId;
record = audioFlinger->createRecord(input,
output,
&status);
……………………
mFlags = output.flags;
mRoutedDeviceId = output.selectedDeviceId;
mSessionId = output.sessionId;
mSampleRate = output.sampleRate;
if (output.cblk == 0) {
ALOGE("Could not get control block");
status = NO_INIT;
goto exit;
}
iMemPointer = output.cblk ->pointer();
if (iMemPointer == NULL) {
ALOGE("Could not get control block pointer");
status = NO_INIT;
goto exit;
}
cblk = static_cast<audio_track_cblk_t*>(iMemPointer);
// Starting address of buffers in shared memory.
// The buffers are either immediately after the control block,
// or in a separate area at discretion of server.
void *buffers;
if (output.buffers == 0) {
buffers = cblk + 1;
} else {
buffers = output.buffers->pointer();
if (buffers == NULL) {
ALOGE("Could not get buffer pointer");
status = NO_INIT;
goto exit;
}
}
// invariant that mAudioRecord != 0 is true only after set() returns successfully
if (mAudioRecord != 0) {
IInterface::asBinder(mAudioRecord)->unlinkToDeath(mDeathNotifier, this);
mDeathNotifier.clear();
}
mAudioRecord = record;
mCblkMemory = output.cblk;
mBufferMemory = output.buffers;
IPCThreadState::self()->flushCommands();
mCblk = cblk;
// note that output.frameCount is the (possibly revised) value of mReqFrameCount
if (output.frameCount < mReqFrameCount || (mReqFrameCount == 0 && output.frameCount == 0)) {
ALOGW("Requested frameCount %zu but received frameCount %zu",
mReqFrameCount, output.frameCount);
}
// Make sure that application is notified with sufficient margin before overrun.
// The computation is done on server side.
if (mNotificationFramesReq > 0 && output.notificationFrameCount != mNotificationFramesReq) {
ALOGW("Server adjusted notificationFrames from %u to %zu for frameCount %zu",
mNotificationFramesReq, output.notificationFrameCount, output.frameCount);
}
mNotificationFramesAct = (uint32_t)output.notificationFrameCount;
//mInput != input includes the case where mInput == AUDIO_IO_HANDLE_NONE for first creation
if (mDeviceCallback != 0 && mInput != output.inputId) {
if (mInput != AUDIO_IO_HANDLE_NONE) {
AudioSystem::removeAudioDeviceCallback(this, mInput);
}
AudioSystem::addAudioDeviceCallback(this, output.inputId);
}
// We retain a copy of the I/O handle, but don't own the reference
mInput = output.inputId;
mRefreshRemaining = true;
mFrameCount = output.frameCount;
// If IAudioRecord is re-created, don't let the requested frameCount
// decrease. This can confuse clients that cache frameCount().
if (mFrameCount > mReqFrameCount) {
mReqFrameCount = mFrameCount;
}
// update proxy
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
mProxy->setEpoch(epoch);
mProxy->setMinimum(mNotificationFramesAct);
mDeathNotifier = new DeathNotifier(this);
IInterface::asBinder(mAudioRecord)->linkToDeath(mDeathNotifier, this);
exit:
mStatus = status;
// sp<IAudioTrack> track destructor will cause releaseOutput() to be called by AudioFlinger
return status;
}
在这其中涉及到了两个类：CreateRecordInput和CreateRecordOutput，都继承自Parcelable，用于跨进程，input中定义的就是audio_attributes、audio_config_t、AudioClient等一些相关信息；output用于接收Record创建成功后的一些信息值；
在createRecord_l()函数中，其实本质上和AudioTrack类似，也是执行了如下几件事：
- 调用AudioFlinger::createRecord()函数，创建对应的RecordTrack音频管理对象；
- 获取Record对象中的FIFO环形缓冲区并进行配置；
- 根据FIFO创建对应控制的AudioRecordClientProxy对象；
AudioFlinger::createRecord
sp<media::IAudioRecord> AudioFlinger::createRecord(const CreateRecordInput& input,
CreateRecordOutput& output,
status_t *status)
{
sp<RecordThread::RecordTrack> recordTrack;
sp<RecordHandle> recordHandle;
sp<Client> client;
status_t lStatus;
audio_session_t sessionId = input.sessionId;
audio_port_handle_t portId = AUDIO_PORT_HANDLE_NONE;
output.cblk.clear();
output.buffers.clear();
output.inputId = AUDIO_IO_HANDLE_NONE;
bool updatePid = (input.clientInfo.clientPid == -1);
const uid_t callingUid = IPCThreadState::self()->getCallingUid();
uid_t clientUid = input.clientInfo.clientUid;
if (!isTrustedCallingUid(callingUid)) {
ALOGW_IF(clientUid != callingUid,
"%s uid %d tried to pass itself off as %d",
__FUNCTION__, callingUid, clientUid);
clientUid = callingUid;
updatePid = true;
}
pid_t clientPid = input.clientInfo.clientPid;
if (updatePid) {
const pid_t callingPid = IPCThreadState::self()->getCallingPid();
ALOGW_IF(clientPid != -1 && clientPid != callingPid,
"%s uid %d pid %d tried to pass itself off as pid %d",
__func__, callingUid, callingPid, clientPid);
clientPid = callingPid;
}
// we don't yet support anything other than linear PCM
if (!audio_is_valid_format(input.config.format) || !audio_is_linear_pcm(input.config.format)) {
ALOGE("createRecord() invalid format %#x", input.config.format);
lStatus = BAD_VALUE;
goto Exit;
}
// further channel mask checks are performed by createRecordTrack_l()
if (!audio_is_input_channel(input.config.channel_mask)) {
ALOGE("createRecord() invalid channel mask %#x", input.config.channel_mask);
lStatus = BAD_VALUE;
goto Exit;
}
if (sessionId == AUDIO_SESSION_ALLOCATE) {
sessionId = (audio_session_t) newAudioUniqueId(AUDIO_UNIQUE_ID_USE_SESSION);
} else if (audio_unique_id_get_use(sessionId) != AUDIO_UNIQUE_ID_USE_SESSION) {
lStatus = BAD_VALUE;
goto Exit;
}
output.sessionId = sessionId;
output.selectedDeviceId = input.selectedDeviceId;
output.flags = input.flags;
client = registerPid(clientPid);
// Not a conventional loop, but a retry loop for at most two iterations total.
// Try first maybe with FAST flag then try again without FAST flag if that fails.
// Exits loop via break on no error of got exit on error
// The sp<> references will be dropped when re-entering scope.
// The lack of indentation is deliberate, to reduce code churn and ease merges.
for (;;) {
// release previously opened input if retrying.
if (output.inputId != AUDIO_IO_HANDLE_NONE) {
recordTrack.clear();
AudioSystem::releaseInput(portId);
output.inputId = AUDIO_IO_HANDLE_NONE;
output.selectedDeviceId = input.selectedDeviceId;
portId = AUDIO_PORT_HANDLE_NONE;
}
lStatus = AudioSystem::getInputForAttr(&input.attr, &output.inputId,
sessionId,
// FIXME compare to AudioTrack
clientPid,
clientUid,
input.opPackageName,
&input.config,
output.flags, &output.selectedDeviceId, &portId);
{
Mutex::Autolock _l(mLock);
RecordThread *thread = checkRecordThread_l(output.inputId);
if (thread == NULL) {
ALOGE("createRecord() checkRecordThread_l failed");
lStatus = BAD_VALUE;
goto Exit;
}
ALOGV("createRecord() lSessionId: %d input %d", sessionId, output.inputId);
output.sampleRate = input.config.sample_rate;
output.frameCount = input.frameCount;
output.notificationFrameCount = input.notificationFrameCount;
recordTrack = thread->createRecordTrack_l(client, input.attr, &output.sampleRate,
input.config.format, input.config.channel_mask,
&output.frameCount, sessionId,
&output.notificationFrameCount,
clientUid, &output.flags,
input.clientInfo.clientTid,
&lStatus, portId);
LOG_ALWAYS_FATAL_IF((lStatus == NO_ERROR) && (recordTrack == 0));
// lStatus == BAD_TYPE means FAST flag was rejected: request a new input from
// audio policy manager without FAST constraint
if (lStatus == BAD_TYPE) {
continue;
}
if (lStatus != NO_ERROR) {
goto Exit;
}
// Check if one effect chain was awaiting for an AudioRecord to be created on this
// session and move it to this thread.
sp<EffectChain> chain = getOrphanEffectChain_l(sessionId);
if (chain != 0) {
Mutex::Autolock _l(thread->mLock);
thread->addEffectChain_l(chain);
}
break;
}
// End of retry loop.
// The lack of indentation is deliberate, to reduce code churn and ease merges.
}
output.cblk = recordTrack->getCblk();
output.buffers = recordTrack->getBuffers();
// return handle to client
recordHandle = new RecordHandle(recordTrack);
Exit:
if (lStatus != NO_ERROR) {
// remove local strong reference to Client before deleting the RecordTrack so that the
// Client destructor is called by the TrackBase destructor with mClientLock held
// Don't hold mClientLock when releasing the reference on the track as the
// destructor will acquire it.
{
Mutex::Autolock _cl(mClientLock);
client.clear();
}
recordTrack.clear();
if (output.inputId != AUDIO_IO_HANDLE_NONE) {
AudioSystem::releaseInput(portId);
}
}
*status = lStatus;
return recordHandle;
}
- 同理，首先先定义了一下变量，包括recordTrack、recordHandle等一下变量，和Track一样，recordTrack和Track类似，recordHandle和TrackHandle类似；
- 根据CreateRecordInput对象input获取一下基本的状态信息；
- 根据input信息调用了AudioSystem::getInputForAttr()函数，通过input信息获取到对应的音频输入流设备，一般情况下该音频输入流设备的标识为AUDIO_DEVICE_IN_BUILTIN_MIC；
- 调用了checkRecordThread_l()函数，根据在getInputForAttr()函数中获取到的audio_io_handle_t类型的outputId并通过mpClientInterface开启输入流设备，从mRecordThreads集合中查找对应的RecordThread对象thread
- RecordThread对象查找成功之后，紧接着就是根据audio config信息，包括attr、sampleRate、format、flag等一些信息通过thread->createRecordTrack_l()函数创建对应的RecordTrack对象；
- RecordTrack对象创建成功之后，将recordTrack中的Cblk和Buffers信息保存到CreateRecordOutput类型的output对象中；
- 创建RecordHandle对象，其中封装了RecordTrack对象；
FIFO控制块
if (output.cblk == 0) {
ALOGE("Could not get control block");
status = NO_INIT;
goto exit;
}
iMemPointer = output.cblk ->pointer();
if (iMemPointer == NULL) {
ALOGE("Could not get control block pointer");
status = NO_INIT;
goto exit;
}
cblk = static_cast<audio_track_cblk_t*>(iMemPointer);
// Starting address of buffers in shared memory.
// The buffers are either immediately after the control block,
// or in a separate area at discretion of server.
void *buffers;
if (output.buffers == 0) {
buffers = cblk + 1;
} else {
buffers = output.buffers->pointer();
if (buffers == NULL) {
ALOGE("Could not get buffer pointer");
status = NO_INIT;
goto exit;
}
}
// invariant that mAudioRecord != 0 is true only after set() returns successfully
if (mAudioRecord != 0) {
IInterface::asBinder(mAudioRecord)->unlinkToDeath(mDeathNotifier, this);
mDeathNotifier.clear();
}
mAudioRecord = record;
mCblkMemory = output.cblk;
mBufferMemory = output.buffers;
IPCThreadState::self()->flushCommands();
mCblk = cblk;
这样AudioRecord和AudioFlinger管理着同一个audio_track_cblk_t，通过它实现了环形FIFO，AudioRecord从FIFO中读取音频数据，MIC采集音频数据添加到FIFO缓冲区中；
audio_track_cblk_t的主要数据成员：
- user：MIC采集数据，AudioFlinger写入当前的写位置的偏移
- userBase：AudioFlinger写偏移的基准位置，结合user的值方可确定真实FIFO地址指针
- server：AudioRecord当前的读位置的偏移
- serverBase：AudioRecord读偏移的基准位置，结合server的值方可确定真实的FIFO地址指针
- frameCount：FIFO的大小，以音频数据的帧为单位，16bit的音频每帧的大小是2字节
- buffers：指向FIFO的起始地址
- out：音频流的方向，对于AudioTrack，out=1，对于AudioRecord，out=0
audio_track_cblk_t的主要成员函数：
- framesAvailable_l()和framesAvailable()：用于获取FIFO中可写的空闲空间的大小，只是加锁和不加锁的区别
- framesReady()：用于获取FIFO中可读取的空间大小
AudioRecordClientProxy
// update proxy
mProxy = new AudioRecordClientProxy(cblk, buffers, mFrameCount, mFrameSize);
mProxy->setEpoch(epoch);
mProxy->setMinimum(mNotificationFramesAct);
根据获取到的Cblk，创建对应的AudioRecordClientProxy对象，用于管理录音数据；
AudioRecord音频路由建立
Java - JNI startRecording
在应用层创建了AudioRecord对象之后，就可以开启录制功能，其执行操作：
audioRecord.startRecording();
public void startRecording()
throws IllegalStateException {
if (mState != STATE_INITIALIZED) {
throw new IllegalStateException("startRecording() called on an "
+ "uninitialized AudioRecord.");
}
// start recording
synchronized(mRecordingStateLock) {
if (native_start(MediaSyncEvent.SYNC_EVENT_NONE, 0) == SUCCESS) {
handleFullVolumeRec(true);
mRecordingState = RECORDSTATE_RECORDING;
}
}
}
static jint
android_media_AudioRecord_start(JNIEnv *env, jobject thiz, jint event, jint triggerSession)
{
sp<AudioRecord> lpRecorder = getAudioRecord(env, thiz);
if (lpRecorder == NULL ) {
jniThrowException(env, "java/lang/IllegalStateException", NULL);
return (jint) AUDIO_JAVA_ERROR;
}
return nativeToJavaStatus(
lpRecorder->start((AudioSystem::sync_event_t)event, (audio_session_t) triggerSession));
}
在JNI层，其中通过AudioRecord对象调用了start()函数；
Native AudioRecord::start
status_t AudioRecord::start(AudioSystem::sync_event_t event, audio_session_t triggerSession)
{
ALOGV("start, sync event %d trigger session %d", event, triggerSession);
AutoMutex lock(mLock);
if (mActive) {
return NO_ERROR;
}
// discard data in buffer
// 重置当前录音Buffer中的录音数据写入的起始位置
const uint32_t framesFlushed = mProxy->flush();
mFramesReadServerOffset -= mFramesRead + framesFlushed;
mFramesRead = 0;
mProxy->clearTimestamp(); // timestamp is invalid until next server push
// reset current position as seen by client to 0
mProxy->setEpoch(mProxy->getEpoch() - mProxy->getPosition());
// force refresh of remaining frames by processAudioBuffer() as last
// read before stop could be partial.
// 标记mRefreshRemaining为true，从注释中可以看到，他应该是用来强制刷新剩余的frames
mRefreshRemaining = true;
mNewPosition = mProxy->getPosition() + mUpdatePeriod;
// 从mCblk->mFlags中获取flag，这里是0x0
int32_t flags = android_atomic_acquire_load(&mCblk->mFlags);
// we reactivate markers (mMarkerPosition != 0) as the position is reset to 0.
// This is legacy behavior. This is not done in stop() to avoid a race condition
// where the last marker event is issued twice.
mMarkerReached = false;
mActive = true;
status_t status = NO_ERROR;
if (!(flags & CBLK_INVALID)) {
// 如果是首次执行该函数，正常情况下，都会执行start()函数
status = mAudioRecord->start(event, triggerSession).transactionError();
if (status == DEAD_OBJECT) {
flags |= CBLK_INVALID;
}
}
if (flags & CBLK_INVALID) {
// 如果start()函数执行不正确，会调用restoreRecord_l()函数，再次建立输入流通道
status = restoreRecord_l("start");
}
if (status != NO_ERROR) {
mActive = false;
ALOGE("start() status %d", status);
} else {
sp<AudioRecordThread> t = mAudioRecordThread;
if (t != 0) {
// 调用AudioRecordThread的resume()函数
t->resume();
} else {
mPreviousPriority = getpriority(PRIO_PROCESS, 0);
get_sched_policy(0, &mPreviousSchedulingGroup);
androidSetThreadPriority(0, ANDROID_PRIORITY_AUDIO);
}
// we've successfully started, log that time
mMediaMetrics.logStart(systemTime());
}
if (status != NO_ERROR) {
mMediaMetrics.markError(status, __FUNCTION__);
}
return status;
}
上述函数中，最核心的两个逻辑：
- mAudioRecord->start()
- mAudioRecordThread->resume()
我们对这两个函数进行分析；
mAudioRecord->start
首先，我们需要看一下，mAudioRecord代表是什么；
sp<media::IAudioRecord> mAudioRecord;
mAudioRecord是sp类型的，也就是说他是Binder中的Bp端，我们需要找到BnAudioRecord，可以在AudioFlinger.h中找到Bn端的定义
// server side of the client's IAudioRecord
class RecordHandle : public android::BnAudioRecord {
public:
RecordHandle(const sp<RecordThread::RecordTrack>& recordTrack);
virtual ~RecordHandle();
virtual status_t start(int /*AudioSystem::sync_event_t*/ event, int triggerSession);
virtual void stop();
virtual status_t onTransact(
uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags);
private:
const sp<RecordThread::RecordTrack> mRecordTrack;
// for use from destructor
void stop_nonvirtual();
};
我们在上面提及过，RecordHandle其实就是RecordTrack的封装，我们继续找RecordHandle类是在哪里实现的；
RecordHandle::start()函数的实现在Tracks.cpp中实现；
binder::Status AudioFlinger::RecordHandle::start(int /*AudioSystem::sync_event_t*/ event,
int /*audio_session_t*/ triggerSession) {
ALOGV("RecordHandle::start()");
return binder::Status::fromStatusT(
mRecordTrack->start((AudioSystem::sync_event_t)event, (audio_session_t) triggerSession));
}
在其中返回了mRecordTrack->start()函数调用；
status_t AudioFlinger::RecordThread::start(RecordThread::RecordTrack* recordTrack,
AudioSystem::sync_event_t event,
audio_session_t triggerSession)
{
ALOGV("RecordThread::start event %d, triggerSession %d", event, triggerSession);
sp<ThreadBase> strongMe = this;
status_t status = NO_ERROR;
// 判断传过来的event的值，从AudioRecord.java可以看到他一直是SYNC_EVENT_NONE，所以这里就清除SyncStartEvent
if (event == AudioSystem::SYNC_EVENT_NONE) {
recordTrack->clearSyncStartEvent();
} else if (event != AudioSystem::SYNC_EVENT_SAME) {
recordTrack->mSyncStartEvent = mAudioFlinger->createSyncEvent(event,
triggerSession,
recordTrack->sessionId(),
syncStartEventCallback,
recordTrack);
// Sync event can be cancelled by the trigger session if the track is not in a
// compatible state in which case we start record immediately
if (recordTrack->mSyncStartEvent->isCancelled()) {
recordTrack->clearSyncStartEvent();
} else {
// do not wait for the event for more than AudioSystem::kSyncRecordStartTimeOutMs
recordTrack->mFramesToDrop = -(ssize_t)
((AudioSystem::kSyncRecordStartTimeOutMs * recordTrack->mSampleRate) / 1000);
}
}
{
// This section is a rendezvous between binder thread executing start() and RecordThread
AutoMutex lock(mLock);
// 判断在mActiveTracks集合中传过来的recordTrack是否是第一个，而我们这是第一次来，肯定会是第一个，而如果不是第一个，也就是说之前因为某种状态已经开始了录音，所以再判断是否是PAUSING状态，更新状态到ACTIVE，然后直接return；
if (mActiveTracks.indexOf(recordTrack) >= 0) {
if (recordTrack->mState == TrackBase::PAUSING) {
ALOGV("active record track PAUSING -> ACTIVE");
recordTrack->mState = TrackBase::ACTIVE;
} else {
ALOGV("active record track state %d", recordTrack->mState);
}
return status;
}
// TODO consider other ways of handling this, such as changing the state to :STARTING and
// adding the track to mActiveTracks after returning from AudioSystem::startInput(),
// or using a separate command thread
// 设置recordTrack的状态为STARTING_1，然后加到mActiveTracks集合中
recordTrack->mState = TrackBase::STARTING_1;
mActiveTracks.add(recordTrack);
status_t status = NO_ERROR;
// 判断recordTrack是否是外部的Track，而isExternalTrack的定义如下：
/**
bool isTimedTrack() const { return (mType == TYPE_TIMED); }
bool isOutputTrack() const { return (mType == TYPE_OUTPUT); }
bool isPatchTrack() const { return (mType == TYPE_PATCH); }
bool isExternalTrack() const { return !isOutputTrack() && !isPatchTrack(); }
*/
// 我们在new RecordTrack的时候传入的mType是TrackBase::TYPE_DEFAULT，所以这个recordTrack是外部的Track；
if (recordTrack->isExternalTrack()) {
mLock.unlock();
bool silenced;
// 确定是ExternalTrack，那么就会调用AudioSystem::startInput方法开始采集数据，这个sessionId就是上一篇文章中出现的那个了，而对于这个mId，在AudioSystem::startInput中他的类型是audio_io_handle_t，在上一篇文章中，这个io_handle是通过AudioSystem::getInputForAttr获取到的，获取到之后通过checkRecordThread_l(input)获取到了一个RecordThread对象，我们看下RecordThread类：class RecordThread : public ThreadBase，再看下ThreadBase父类，父类的构造函数实现在Threads.cpp文件中，在这里我们发现把input赋值给了mId，也就是说，调用AudioSystem::startInput函数的参数，就是之前建立的输入流input以及生成的sessionId了；
status = AudioSystem::startInput(recordTrack->portId(), &silenced);
mLock.lock();
// FIXME should verify that recordTrack is still in mActiveTracks
if (status != NO_ERROR) {
mActiveTracks.remove(recordTrack);
recordTrack->clearSyncStartEvent();
ALOGV("RecordThread::start error %d", status);
return status;
}
recordTrack->setSilenced(silenced);
}
// Catch up with current buffer indices if thread is already running.
// This is what makes a new client discard all buffered data. If the track's mRsmpInFront
// was initialized to some value closer to the thread's mRsmpInFront, then the track could
// see previously buffered data before it called start(), but with greater risk of overrun.
recordTrack->mResamplerBufferProvider->reset();
// clear any converter state as new data will be discontinuous
recordTrack->mRecordBufferConverter->reset();
// 设置recordTrack的状态为STARTING_2，然后调用mWaitWorkCV.broadcast()广播通知所有的线程开始工作。
// 注意：这里提前说明一下，在AudioSystem::startInput中，AudioFlinger::RecordThread已经开始跑起来了，所以其实broadcast对RecordThread是没有作用的，并且，需要特别注意的是，这里更新了recordTrack->mState为STARTING_2，之前在加入mActiveTracks时的状态是STARTING_1，这个地方比较有意思，这里先标记下，到时候在分析RecordThread的时候揭晓答案；
recordTrack->mState = TrackBase::STARTING_2;
// signal thread to start
mWaitWorkCV.broadcast();
// 判断下recordTrack是否已经加到mActiveTracks集合中了，如果没有的话，就说明start失败了，需要stopInput等；
if (mActiveTracks.indexOf(recordTrack) < 0) {
ALOGV("Record failed to start");
status = BAD_VALUE;
goto startError;
}
return status;
}
startError:
if (recordTrack->isExternalTrack()) {
AudioSystem::stopInput(recordTrack->portId());
}
recordTrack->clearSyncStartEvent();
// FIXME I wonder why we do not reset the state here?
return status;
}
该函数中，主要是对recordTrack进行有效性判断并更新recordTrack状态为ACTIVE，然后将开启的recordTrack添加到mActiveTracks中，然后调用AudioSystem::startInput()函数，将recordTrack->portId()传入开启数据采集功能；
AudioSystem::startInput
status_t AudioSystem::startInput(audio_port_handle_t portId, bool *silenced)
{
const sp<IAudioPolicyService>& aps = AudioSystem::get_audio_policy_service();
if (aps == 0) return PERMISSION_DENIED;
return aps->startInput(portId, silenced);
}
status_t AudioPolicyService::startInput(audio_port_handle_t portId, bool *silenced)
{
…………………………
status_t status;
{
AutoCallerClear acc;
status = mAudioPolicyManager->startInput(
client->input, client->session, *silenced, &concurrency);
}
…………………………
}
status_t AudioPolicyManager::startInput(audio_io_handle_t input,
audio_session_t session,
bool silenced,
concurrency_type__mask_t *concurrency)
{
ALOGV("AudioPolicyManager::startInput(input:%d, session:%d, silenced:%d, concurrency:%d)",
input, session, silenced, *concurrency);
*concurrency = API_INPUT_CONCURRENCY_NONE;
// 通过input找到mInputs集合中的位置，并获取他的inputDesc
ssize_t index = mInputs.indexOfKey(input);
if (index < 0) {
ALOGW("startInput() unknown input %d", input);
return BAD_VALUE;
}
sp<AudioInputDescriptor> inputDesc = mInputs.valueAt(index);
sp<AudioSession> audioSession = inputDesc->getAudioSession(session);
if (audioSession == 0) {
ALOGW("startInput() unknown session %d on input %d", session, input);
return BAD_VALUE;
}
// FIXME: disable concurrent capture until UI is ready
#if 0
if (!isConcurentCaptureAllowed(inputDesc, audioSession)) {
ALOGW("startInput(%d) failed: other input already started", input);
return INVALID_OPERATION;
}
if (isInCall()) {
*concurrency |= API_INPUT_CONCURRENCY_CALL;
}
if (mInputs.activeInputsCountOnDevices() != 0) {
*concurrency |= API_INPUT_CONCURRENCY_CAPTURE;
}
#else
// 判断input设备是否是虚拟设备，若不是则再判断是否存在active的设备
if (!is_virtual_input_device(inputDesc->mDevice)) {
if (mCallTxPatch != 0 &&
inputDesc->getModuleHandle() == mCallTxPatch->mPatch.sources[0].ext.device.hw_module) {
ALOGW("startInput(%d) failed: call in progress", input);
*concurrency |= API_INPUT_CONCURRENCY_CALL;
return INVALID_OPERATION;
}
Vector<sp<AudioInputDescriptor>> activeInputs = mInputs.getActiveInputs();
// If a UID is idle and records silence and another not silenced recording starts
// from another UID (idle or active) we stop the current idle UID recording in
// favor of the new one - "There can be only one" TM
if (!silenced) {
for (const auto& activeDesc : activeInputs) {
if ((audioSession->flags() & AUDIO_INPUT_FLAG_MMAP_NOIRQ) != 0 &&
activeDesc->getId() == inputDesc->getId()) {
continue;
}
AudioSessionCollection activeSessions = activeDesc->getAudioSessions(
true /*activeOnly*/);
sp<AudioSession> activeSession = activeSessions.valueAt(0);
if (activeSession->isSilenced()) {
audio_io_handle_t activeInput = activeDesc->mIoHandle;
audio_session_t activeSessionId = activeSession->session();
stopInput(activeInput, activeSessionId);
releaseInput(activeInput, activeSessionId);
ALOGV("startInput(%d) stopping silenced input %d", input, activeInput);
activeInputs = mInputs.getActiveInputs();
}
}
}
for (const auto& activeDesc : activeInputs) {
if ((audioSession->flags() & AUDIO_INPUT_FLAG_MMAP_NOIRQ) != 0 &&
activeDesc->getId() == inputDesc->getId()) {
continue;
}
audio_source_t activeSource = activeDesc->inputSource(true);
if (audioSession->inputSource() == AUDIO_SOURCE_HOTWORD) {
if (activeSource == AUDIO_SOURCE_HOTWORD) {
if (activeDesc->hasPreemptedSession(session)) {
ALOGW("startInput(%d) failed for HOTWORD: "
"other input %d already started for HOTWORD",
input, activeDesc->mIoHandle);
*concurrency |= API_INPUT_CONCURRENCY_HOTWORD;
return INVALID_OPERATION;
}
} else {
ALOGV("startInput(%d) failed for HOTWORD: other input %d already started",
input, activeDesc->mIoHandle);
*concurrency |= API_INPUT_CONCURRENCY_CAPTURE;
return INVALID_OPERATION;
}
} else {
if (activeSource != AUDIO_SOURCE_HOTWORD) {
ALOGW("startInput(%d) failed: other input %d already started",
input, activeDesc->mIoHandle);
*concurrency |= API_INPUT_CONCURRENCY_CAPTURE;
return INVALID_OPERATION;
}
}
}
// We only need to check if the sound trigger session supports concurrent capture if the
// input is also a sound trigger input. Otherwise, we should preempt any hotword stream
// that's running.
const bool allowConcurrentWithSoundTrigger =
inputDesc->isSoundTrigger() ? soundTriggerSupportsConcurrentCapture() : false;
// if capture is allowed, preempt currently active HOTWORD captures
for (const auto& activeDesc : activeInputs) {
if (allowConcurrentWithSoundTrigger && activeDesc->isSoundTrigger()) {
continue;
}
audio_source_t activeSource = activeDesc->inputSource(true);
if (activeSource == AUDIO_SOURCE_HOTWORD) {
AudioSessionCollection activeSessions =
activeDesc->getAudioSessions(true /*activeOnly*/);
audio_session_t activeSession = activeSessions.keyAt(0);
audio_io_handle_t activeHandle = activeDesc->mIoHandle;
SortedVector<audio_session_t> sessions = activeDesc->getPreemptedSessions();
*concurrency |= API_INPUT_CONCURRENCY_PREEMPT;
sessions.add(activeSession);
inputDesc->setPreemptedSessions(sessions);
stopInput(activeHandle, activeSession);
releaseInput(activeHandle, activeSession);
ALOGV("startInput(%d) for HOTWORD preempting HOTWORD input %d",
input, activeDesc->mIoHandle);
}
}
}
#endif
// Make sure we start with the correct silence state
audioSession->setSilenced(silenced);
// increment activity count before calling getNewInputDevice() below as only active sessions
// are considered for device selection
audioSession->changeActiveCount(1);
// Routing?
mInputRoutes.incRouteActivity(session);
if (audioSession->activeCount() == 1 || mInputRoutes.getAndClearRouteChanged(session)) {
// indicate active capture to sound trigger service if starting capture from a mic on
// primary HW module
audio_devices_t device = getNewInputDevice(inputDesc);
// 继续调用setInputDevice()函数，其中getNewInputDevice函数的作用是根据input获取audio_devices_t设备，同样，这个设备在上述的AudioPolicyManager::getInputForAttr()方法中通过getDeviceAndMixForInputSource()获取到的，即AUDIO_DEVICE_IN_BUILTIN_MIC内置MIC设备，同时在该函数最后更新了inputDesc->mDevice；
setInputDevice(input, device, true /* force */);
status_t status = inputDesc->start();
if (status != NO_ERROR) {
mInputRoutes.decRouteActivity(session);
audioSession->changeActiveCount(-1);
return status;
}
if (inputDesc->getAudioSessionCount(true/*activeOnly*/) == 1) {
sp<AudioPolicyMix> policyMix = inputDesc->mPolicyMix.promote();
// if input maps to a dynamic policy with an activity listener, notify of state change
if ((policyMix != NULL)
&& ((policyMix->mCbFlags & AudioMix::kCbFlagNotifyActivity) != 0)) {
mpClientInterface->onDynamicPolicyMixStateUpdate(policyMix->mDeviceAddress,
MIX_STATE_MIXING);
}
audio_devices_t primaryInputDevices = availablePrimaryInputDevices();
if (((device & primaryInputDevices & ~AUDIO_DEVICE_BIT_IN) != 0) &&
mInputs.activeInputsCountOnDevices(primaryInputDevices) == 1) {
SoundTrigger::setCaptureState(true);
}
// automatically enable the remote submix output when input is started if not
// used by a policy mix of type MIX_TYPE_RECORDERS
// For remote submix (a virtual device), we open only one input per capture request.
// 判断是否是remote_submix设备，然后做相应处理
if (audio_is_remote_submix_device(inputDesc->mDevice)) {
String8 address = String8("");
if (policyMix == NULL) {
address = String8("0");
} else if (policyMix->mMixType == MIX_TYPE_PLAYERS) {
address = policyMix->mDeviceAddress;
}
if (address != "") {
setDeviceConnectionStateInt(AUDIO_DEVICE_OUT_REMOTE_SUBMIX,
AUDIO_POLICY_DEVICE_STATE_AVAILABLE,
address, "remote-submix");
}
}
}
}
ALOGV("AudioPolicyManager::startInput() input source = %d", audioSession->inputSource());
return NO_ERROR;
}
在该方法中主要执行了如下几个事件：
- 通过input找到mInputs集合中的位置，并获取他的inputDesc；
- 判断input设备是否是虚拟设备，若不是则再判断是否存在active的设备；
- 调用setInputDevice()函数；
- 调用inputDesc->start()；
- 判断是否是remote_submix设备，然后做相应处理；
- inputDesc的mRefCount计数+1；
其中在上述的逻辑中，有两个主要的逻辑：setInputDevice()函数和inputDesc->start()函数，我们着重对这两个步骤进行分析；
setInputDevice函数
status_t AudioPolicyManager::setInputDevice(audio_io_handle_t input,
audio_devices_t device,
bool force,
audio_patch_handle_t *patchHandle)
{
status_t status = NO_ERROR;
// input代表的音频输入流设备的标识为AUDIO_DEVICE_IN_BUILTIN_MIC
sp<AudioInputDescriptor> inputDesc = mInputs.valueFor(input);
if ((device != AUDIO_DEVICE_NONE) && ((device != inputDesc->mDevice) || force)) {
inputDesc->mDevice = device;
// 通过device获取mAvailableInputDevices集合中所有的设备
DeviceVector deviceList = mAvailableInputDevices.getDevicesFromType(device);
if (!deviceList.isEmpty()) {
// audio_patch，这个变量定义在audio.h中，该逻辑中对audiopatch的source和sinks进行了赋值
struct audio_patch patch;
inputDesc->toAudioPortConfig(&patch.sinks[0]);
// AUDIO_SOURCE_HOTWORD is for internal use only:
// handled as AUDIO_SOURCE_VOICE_RECOGNITION by the audio HAL
if (patch.sinks[0].ext.mix.usecase.source == AUDIO_SOURCE_HOTWORD &&
!inputDesc->isSoundTrigger()) {
patch.sinks[0].ext.mix.usecase.source = AUDIO_SOURCE_VOICE_RECOGNITION;
}
patch.num_sinks = 1;
//only one input device for now
deviceList.itemAt(0)->toAudioPortConfig(&patch.sources[0]);
patch.num_sources = 1;
ssize_t index;
if (patchHandle && *patchHandle != AUDIO_PATCH_HANDLE_NONE) {
index = mAudioPatches.indexOfKey(*patchHandle);
} else {
index = mAudioPatches.indexOfKey(inputDesc->getPatchHandle());
}
sp< AudioPatch> patchDesc;
audio_patch_handle_t afPatchHandle = AUDIO_PATCH_HANDLE_NONE;
if (index >= 0) {
patchDesc = mAudioPatches.valueAt(index);
afPatchHandle = patchDesc->mAfPatchHandle;
}
// 创建Audio通路
status_t status = mpClientInterface->createAudioPatch(&patch,
&afPatchHandle,
0);
ALOGV("setInputDevice() createAudioPatch returned %d patchHandle %d",
status, afPatchHandle);
if (status == NO_ERROR) {
// 更新patchDesc的属性
if (index < 0) {
patchDesc = new AudioPatch(&patch, mUidCached);
addAudioPatch(patchDesc->mHandle, patchDesc);
} else {
patchDesc->mPatch = patch;
}
patchDesc->mAfPatchHandle = afPatchHandle;
if (patchHandle) {
*patchHandle = patchDesc->mHandle;
}
inputDesc->setPatchHandle(patchDesc->mHandle);
nextAudioPortGeneration();
// 更新AudioPatch列表
mpClientInterface->onAudioPatchListUpdate();
}
}
}
return status;
}
mpClientInterface->createAudioPatch()
mpClientInterface对象的类型为AudioPolicyClientInterface，AudioPolicyClient继承自AudioPolicyClientInterface，AudioPolicyClient的实现类为AudioPolicyClientImpl.cpp；
status_t AudioPolicyService::AudioPolicyClient::createAudioPatch(const struct audio_patch *patch,
audio_patch_handle_t *handle,
int delayMs)
{
return mAudioPolicyService->clientCreateAudioPatch(patch, handle, delayMs);
}
status_t AudioPolicyService::clientCreateAudioPatch(const struct audio_patch *patch,
audio_patch_handle_t *handle,
int delayMs)
{
return mAudioCommandThread->createAudioPatchCommand(patch, handle, delayMs);
}
status_t AudioPolicyService::AudioCommandThread::createAudioPatchCommand(
const struct audio_patch *patch,
audio_patch_handle_t *handle,
int delayMs)
{
status_t status = NO_ERROR;
sp<AudioCommand> command = new AudioCommand();
command->mCommand = CREATE_AUDIO_PATCH;
CreateAudioPatchData *data = new CreateAudioPatchData();
data->mPatch = *patch;
data->mHandle = *handle;
command->mParam = data;
command->mWaitStatus = true;
ALOGV("AudioCommandThread() adding create patch delay %d", delayMs);
status = sendCommand(command, delayMs);
if (status == NO_ERROR) {
*handle = data->mHandle;
}
return status;
}
在该函数中，创建了AudioPatch对应的command对象，其中command->mCommand为CREATE_AUDIO_PATCH，然后通过调用sendCommand()函数将command发送出去；
status_t AudioPolicyService::AudioCommandThread::sendCommand(sp<AudioCommand>& command, int delayMs)
{
{
Mutex::Autolock _l(mLock);
insertCommand_l(command, delayMs);
mWaitWorkCV.signal();
}
Mutex::Autolock _l(command->mLock);
while (command->mWaitStatus) {
nsecs_t timeOutNs = kAudioCommandTimeoutNs + milliseconds(delayMs);
if (command->mCond.waitRelative(command->mLock, timeOutNs) != NO_ERROR) {
command->mStatus = TIMED_OUT;
command->mWaitStatus = false;
}
}
return command->mStatus;
}
在sendComman()函数中，调用insertCommand_l()函数将command以及对应的执行时间通过insertAt的方式添加到mAudioCommands队列中；
而我们之前在描述AudioPolicyService启动的时候，在AudioPolicyService的构造函数中，创建了AudioCommandThread线程对象；
void AudioPolicyService::onFirstRef()
{
{
Mutex::Autolock _l(mLock);
// start tone playback thread
mTonePlaybackThread = new AudioCommandThread(String8("ApmTone"), this);
// start audio commands thread
mAudioCommandThread = new AudioCommandThread(String8("ApmAudio"), this);
// start output activity command thread
mOutputCommandThread = new AudioCommandThread(String8("ApmOutput"), this);
mAudioPolicyClient = new AudioPolicyClient(this);
mAudioPolicyManager = createAudioPolicyManager(mAudioPolicyClient);
}
上述创建了3种AudioCommandThread对象，我们现在涉及到的为MAudioCommandThread对象；
// 在AudioPolicyService对象构造函数中，创建了AudioCommandThread对象，在第一次强引用AudioCommandThread线程对象时，onFirstRef函数会被调用
void AudioPolicyService::AudioCommandThread::onFirstRef()
{
run(mName.string(), ANDROID_PRIORITY_AUDIO);
}
该函数中执行了run()函数，传入了CommandThread对应的name，该线程主要是用于其中包括了音量控制、声音设备的选择和切换等；
我们看一下AudioCommandThread的threadLoop()线程体：
bool AudioPolicyService::AudioCommandThread::threadLoop()
{
nsecs_t waitTime = -1;
mLock.lock();
while (!exitPending())
{
sp<AudioPolicyService> svc;
while (!mAudioCommands.isEmpty() && !exitPending()) {
nsecs_t curTime = systemTime();
// commands are sorted by increasing time stamp: execute them from index 0 and up
if (mAudioCommands[0]->mTime <= curTime) {
sp<AudioCommand> command = mAudioCommands[0];
mAudioCommands.removeAt(0);
mLastCommand = command;
switch (command->mCommand) {
…………………………
case CREATE_AUDIO_PATCH: {
CreateAudioPatchData *data = (CreateAudioPatchData *)command->mParam.get();
ALOGV("AudioCommandThread() processing create audio patch");
sp<IAudioFlinger> af = AudioSystem::get_audio_flinger();
if (af == 0) {
command->mStatus = PERMISSION_DENIED;
} else {
command->mStatus = af->createAudioPatch(&data->mPatch, &data->mHandle);
}
} break;
…………………………
}
{
Mutex::Autolock _l(command->mLock);
if (command->mWaitStatus) {
command->mWaitStatus = false;
command->mCond.signal();
}
}
waitTime = -1;
// release mLock before releasing strong reference on the service as
// AudioPolicyService destructor calls AudioCommandThread::exit() which
// acquires mLock.
mLock.unlock();
svc.clear();
mLock.lock();
} else {
waitTime = mAudioCommands[0]->mTime - curTime;
break;
}
}
// release delayed commands wake lock if the queue is empty
if (mAudioCommands.isEmpty()) {
release_wake_lock(mName.string());
}
// At this stage we have either an empty command queue or the first command in the queue
// has a finite delay. So unless we are exiting it is safe to wait.
if (!exitPending()) {
ALOGV("AudioCommandThread() going to sleep");
if (waitTime == -1) {
mWaitWorkCV.wait(mLock);
} else {
mWaitWorkCV.waitRelative(mLock, waitTime);
}
}
}
// release delayed commands wake lock before quitting
if (!mAudioCommands.isEmpty()) {
release_wake_lock(mName.string());
}
mLock.unlock();
return false;
}
我们在createAudioPatch过程中涉及到的command为CREATE_AUDIO_PATCH，在该分支中，我们主要是获取到IAudioFlinger对象，通过调用AF的createAudioPatch()函数，创建对应的Audio通路；
AudioFlinger::createAudioPatch
/* Connect a patch between several source and sink ports */
status_t AudioFlinger::createAudioPatch(const struct audio_patch *patch,
audio_patch_handle_t *handle)
{
Mutex::Autolock _l(mLock);
if (mPatchPanel != 0) {
return mPatchPanel->createAudioPatch(patch, handle);
}
return NO_INIT;
}
/* Connect a patch between several source and sink ports */
status_t AudioFlinger::PatchPanel::createAudioPatch(const struct audio_patch *patch,
audio_patch_handle_t *handle)
{
status_t status = NO_ERROR;
audio_patch_handle_t halHandle = AUDIO_PATCH_HANDLE_NONE;
sp<AudioFlinger> audioflinger = mAudioFlinger.promote();
if (handle == NULL || patch == NULL) {
return BAD_VALUE;
}
// 在AudioPolicyManager::setInputDevice()函数中，num_sources与num_sinks都为1，我们基于第一次启动分析，所以在这种情况下，num_sources和num_sinks都为1；
ALOGV("createAudioPatch() num_sources %d num_sinks %d handle %d",
patch->num_sources, patch->num_sinks, *handle);
if (audioflinger == 0) {
return NO_INIT;
}
if (patch->num_sources == 0 || patch->num_sources > AUDIO_PATCH_PORTS_MAX ||
(patch->num_sinks == 0 && patch->num_sources != 2) ||
patch->num_sinks > AUDIO_PATCH_PORTS_MAX) {
return BAD_VALUE;
}
// limit number of sources to 1 for now or 2 sources for special cross hw module case.
// only the audio policy manager can request a patch creation with 2 sources.
if (patch->num_sources > 2) {
return INVALID_OPERATION;
}
if (*handle != AUDIO_PATCH_HANDLE_NONE) {
// 这里主要是用于判断handle状态信息
}
Patch *newPatch = new Patch(patch);
// 判断patch->source的type
switch (patch->sources[0].type) {
case AUDIO_PORT_TYPE_DEVICE: {
audio_module_handle_t srcModule = patch->sources[0].ext.device.hw_module;
ssize_t index = audioflinger->mAudioHwDevs.indexOfKey(srcModule);
if (index < 0) {
ALOGW("createAudioPatch() bad src hw module %d", srcModule);
status = BAD_VALUE;
goto exit;
}
AudioHwDevice *audioHwDevice = audioflinger->mAudioHwDevs.valueAt(index);
for (unsigned int i = 0; i < patch->num_sinks; i++) {
// support only one sink if connection to a mix or across HW modules
if ((patch->sinks[i].type == AUDIO_PORT_TYPE_MIX ||
patch->sinks[i].ext.mix.hw_module != srcModule) &&
patch->num_sinks > 1) {
status = INVALID_OPERATION;
goto exit;
}
// reject connection to different sink types
if (patch->sinks[i].type != patch->sinks[0].type) {
ALOGW("createAudioPatch() different sink types in same patch not supported");
status = BAD_VALUE;
goto exit;
}
}
// manage patches requiring a software bridge
// - special patch request with 2 sources (reuse one existing output mix) OR
// - Device to device AND
// - source HW module != destination HW module OR
// - audio HAL does not support audio patches creation
if ((patch->num_sources == 2) ||
((patch->sinks[0].type == AUDIO_PORT_TYPE_DEVICE) &&
((patch->sinks[0].ext.device.hw_module != srcModule) ||
!audioHwDevice->supportsAudioPatches()))) {
if (patch->num_sources == 2) {
if (patch->sources[1].type != AUDIO_PORT_TYPE_MIX ||
(patch->num_sinks != 0 && patch->sinks[0].ext.device.hw_module !=
patch->sources[1].ext.mix.hw_module)) {
ALOGW("createAudioPatch() invalid source combination");
status = INVALID_OPERATION;
goto exit;
}
sp<ThreadBase> thread =
audioflinger->checkPlaybackThread_l(patch->sources[1].ext.mix.handle);
newPatch->mPlaybackThread = (MixerThread *)thread.get();
if (thread == 0) {
ALOGW("createAudioPatch() cannot get playback thread");
status = INVALID_OPERATION;
goto exit;
}
} else {
audio_config_t config = AUDIO_CONFIG_INITIALIZER;
audio_devices_t device = patch->sinks[0].ext.device.type;
String8 address = String8(patch->sinks[0].ext.device.address);
audio_io_handle_t output = AUDIO_IO_HANDLE_NONE;
sp<ThreadBase> thread = audioflinger->openOutput_l(
patch->sinks[0].ext.device.hw_module,
&output,
&config,
device,
address,
AUDIO_OUTPUT_FLAG_NONE);
newPatch->mPlaybackThread = (PlaybackThread *)thread.get();
ALOGV("audioflinger->openOutput_l() returned %p",
newPatch->mPlaybackThread.get());
if (newPatch->mPlaybackThread == 0) {
status = NO_MEMORY;
goto exit;
}
}
audio_devices_t device = patch->sources[0].ext.device.type;
String8 address = String8(patch->sources[0].ext.device.address);
audio_config_t config = AUDIO_CONFIG_INITIALIZER;
// open input stream with source device audio properties if provided or
// default to peer output stream properties otherwise.
if (patch->sources[0].config_mask & AUDIO_PORT_CONFIG_SAMPLE_RATE) {
config.sample_rate = patch->sources[0].sample_rate;
} else {
config.sample_rate = newPatch->mPlaybackThread->sampleRate();
}
if (patch->sources[0].config_mask & AUDIO_PORT_CONFIG_CHANNEL_MASK) {
config.channel_mask = patch->sources[0].channel_mask;
} else {
config.channel_mask =
audio_channel_in_mask_from_count(newPatch->mPlaybackThread->channelCount());
}
if (patch->sources[0].config_mask & AUDIO_PORT_CONFIG_FORMAT) {
config.format = patch->sources[0].format;
} else {
config.format = newPatch->mPlaybackThread->format();
}
audio_io_handle_t input = AUDIO_IO_HANDLE_NONE;
// 打开出入设备
sp<ThreadBase> thread = audioflinger->openInput_l(srcModule,
&input,
&config,
device,
address,
AUDIO_SOURCE_MIC,
AUDIO_INPUT_FLAG_NONE);
newPatch->mRecordThread = (RecordThread *)thread.get();
ALOGV("audioflinger->openInput_l() returned %p inChannelMask %08x",
newPatch->mRecordThread.get(), config.channel_mask);
if (newPatch->mRecordThread == 0) {
status = NO_MEMORY;
goto exit;
}
// 切换声音通道
status = createPatchConnections(newPatch, patch);
if (status != NO_ERROR) {
goto exit;
}
} else {
……………………
}
} break;
case AUDIO_PORT_TYPE_MIX: {
……………………
} break;
default:
status = BAD_VALUE;
goto exit;
}
exit:
ALOGV("createAudioPatch() status %d", status);
if (status == NO_ERROR) {
*handle = (audio_patch_handle_t) audioflinger->nextUniqueId(AUDIO_UNIQUE_ID_USE_PATCH);
newPatch->mHandle = *handle;
newPatch->mHalHandle = halHandle;
mPatches.add(newPatch);
ALOGV("createAudioPatch() added new patch handle %d halHandle %d", *handle, halHandle);
} else {
clearPatchConnections(newPatch);
delete newPatch;
}
return status;
}
在该函数中，调用了openInput_l()，开启了指定了音频输入设备；
AudioFlinger::openInput_l
sp<AudioFlinger::ThreadBase> AudioFlinger::openInput_l(audio_module_handle_t module,
audio_io_handle_t *input,
audio_config_t *config,
audio_devices_t devices,
const String8& address,
audio_source_t source,
audio_input_flags_t flags)
{
AudioHwDevice *inHwDev = findSuitableHwDev_l(module, devices);
if (inHwDev == NULL) {
*input = AUDIO_IO_HANDLE_NONE;
return 0;
}
// Audio Policy can request a specific handle for hardware hotword.
// The goal here is not to re-open an already opened input.
// It is to use a pre-assigned I/O handle.
if (*input == AUDIO_IO_HANDLE_NONE) {
*input = nextUniqueId(AUDIO_UNIQUE_ID_USE_INPUT);
} else if (audio_unique_id_get_use(*input) != AUDIO_UNIQUE_ID_USE_INPUT) {
ALOGE("openInput_l() requested input handle %d is invalid", *input);
return 0;
} else if (mRecordThreads.indexOfKey(*input) >= 0) {
// This should not happen in a transient state with current design.
ALOGE("openInput_l() requested input handle %d is already assigned", *input);
return 0;
}
audio_config_t halconfig = *config;
sp<DeviceHalInterface> inHwHal = inHwDev->hwDevice();
sp<StreamInHalInterface> inStream;
status_t status = inHwHal->openInputStream(
*input, devices, &halconfig, flags, address.string(), source, &inStream);
ALOGV("openInput_l() openInputStream returned input %p, devices %#x, SamplingRate %d"
", Format %#x, Channels %#x, flags %#x, status %d addr %s",
inStream.get(),
devices,
halconfig.sample_rate,
halconfig.format,
halconfig.channel_mask,
flags,
status, address.string());
// If the input could not be opened with the requested parameters and we can handle the
// conversion internally, try to open again with the proposed parameters.
if (status == BAD_VALUE &&
audio_is_linear_pcm(config->format) &&
audio_is_linear_pcm(halconfig.format) &&
(halconfig.sample_rate <= AUDIO_RESAMPLER_DOWN_RATIO_MAX * config->sample_rate) &&
(audio_channel_count_from_in_mask(halconfig.channel_mask) <= FCC_8) &&
(audio_channel_count_from_in_mask(config->channel_mask) <= FCC_8)) {
// FIXME describe the change proposed by HAL (save old values so we can log them here)
ALOGV("openInput_l() reopening with proposed sampling rate and channel mask");
inStream.clear();
status = inHwHal->openInputStream(
*input, devices, &halconfig, flags, address.string(), source, &inStream);
// FIXME log this new status; HAL should not propose any further changes
}
if (status == NO_ERROR && inStream != 0) {
AudioStreamIn *inputStream = new AudioStreamIn(inHwDev, inStream, flags);
if ((flags & AUDIO_INPUT_FLAG_MMAP_NOIRQ) != 0) {
sp<MmapCaptureThread> thread =
new MmapCaptureThread(this, *input,
inHwDev, inputStream,
primaryOutputDevice_l(), devices, mSystemReady);
mMmapThreads.add(*input, thread);
ALOGV("openInput_l() created mmap capture thread: ID %d thread %p", *input,
thread.get());
return thread;
} else {
#ifdef TEE_SINK
// Try to re-use most recently used Pipe to archive a copy of input for dumpsys,
// or (re-)create if current Pipe is idle and does not match the new format
sp<NBAIO_Sink> teeSink;
enum {
TEE_SINK_NO, // don't copy input
TEE_SINK_NEW, // copy input using a new pipe
TEE_SINK_OLD, // copy input using an existing pipe
} kind;
NBAIO_Format format = Format_from_SR_C(halconfig.sample_rate,
audio_channel_count_from_in_mask(halconfig.channel_mask), halconfig.format);
if (!mTeeSinkInputEnabled) {
kind = TEE_SINK_NO;
} else if (!Format_isValid(format)) {
kind = TEE_SINK_NO;
} else if (mRecordTeeSink == 0) {
kind = TEE_SINK_NEW;
} else if (mRecordTeeSink->getStrongCount() != 1) {
kind = TEE_SINK_NO;
} else if (Format_isEqual(format, mRecordTeeSink->format())) {
kind = TEE_SINK_OLD;
} else {
kind = TEE_SINK_NEW;
}
switch (kind) {
case TEE_SINK_NEW: {
Pipe *pipe = new Pipe(mTeeSinkInputFrames, format);
size_t numCounterOffers = 0;
const NBAIO_Format offers[1] = {format};
ssize_t index = pipe->negotiate(offers, 1, NULL, numCounterOffers);
ALOG_ASSERT(index == 0);
PipeReader *pipeReader = new PipeReader(*pipe);
numCounterOffers = 0;
index = pipeReader->negotiate(offers, 1, NULL, numCounterOffers);
ALOG_ASSERT(index == 0);
mRecordTeeSink = pipe;
mRecordTeeSource = pipeReader;
teeSink = pipe;
}
break;
case TEE_SINK_OLD:
teeSink = mRecordTeeSink;
break;
case TEE_SINK_NO:
default:
break;
}
#endif
// Start record thread
// RecordThread requires both input and output device indication to forward to audio
// pre processing modules
sp<RecordThread> thread = new RecordThread(this,
inputStream,
*input,
primaryOutputDevice_l(),
devices,
mSystemReady
#ifdef TEE_SINK
, teeSink
#endif
);
mRecordThreads.add(*input, thread);
ALOGV("openInput_l() created record thread: ID %d thread %p", *input, thread.get());
return thread;
}
}
*input = AUDIO_IO_HANDLE_NONE;
return 0;
}
在该函数中，通过调用inHwHal->openInputStream()函数，进入HAL层，开启输入设备，这里暂不叙述；
AudioRecord读取数据
startRecording()方法开启输入设备之后，紧接着执行了int read = audioRecord.read(buffer, 0, recordBufSize);逻辑，用于读入输入设备采集的音频数据；
Java - JNI read
public int read(@NonNull byte[] audioData, int offsetInBytes, int sizeInBytes) {
return read(audioData, offsetInBytes, sizeInBytes, READ_BLOCKING);
}
public int read(@NonNull byte[] audioData, int offsetInBytes, int sizeInBytes,
@ReadMode int readMode) {
if (mState != STATE_INITIALIZED || mAudioFormat == AudioFormat.ENCODING_PCM_FLOAT) {
return ERROR_INVALID_OPERATION;
}
if ((readMode != READ_BLOCKING) && (readMode != READ_NON_BLOCKING)) {
Log.e(TAG, "AudioRecord.read() called with invalid blocking mode");
return ERROR_BAD_VALUE;
}
if ( (audioData == null) || (offsetInBytes < 0 ) || (sizeInBytes < 0)
|| (offsetInBytes + sizeInBytes < 0) // detect integer overflow
|| (offsetInBytes + sizeInBytes > audioData.length)) {
return ERROR_BAD_VALUE;
}
return native_read_in_byte_array(audioData, offsetInBytes, sizeInBytes,
readMode == READ_BLOCKING);
}
template <typename T>
static jint android_media_AudioRecord_readInArray(JNIEnv *env, jobject thiz,
T javaAudioData,
jint offsetInSamples, jint sizeInSamples,
jboolean isReadBlocking) {
// get the audio recorder from which we'll read new audio samples
sp<AudioRecord> lpRecorder = getAudioRecord(env, thiz);
if (lpRecorder == NULL) {
ALOGE("Unable to retrieve AudioRecord object");
return (jint)AUDIO_JAVA_INVALID_OPERATION;
}
if (javaAudioData == NULL) {
ALOGE("Invalid Java array to store recorded audio");
return (jint)AUDIO_JAVA_BAD_VALUE;
}
// NOTE: We may use GetPrimitiveArrayCritical() when the JNI implementation changes in such
// a way that it becomes much more efficient. When doing so, we will have to prevent the
// AudioSystem callback to be called while in critical section (in case of media server
// process crash for instance)
// get the pointer to where we'll record the audio
auto *recordBuff = envGetArrayElements(env, javaAudioData, NULL);
if (recordBuff == NULL) {
ALOGE("Error retrieving destination for recorded audio data");
return (jint)AUDIO_JAVA_BAD_VALUE;
}
// read the new audio data from the native AudioRecord object
const size_t sizeInBytes = sizeInSamples * sizeof(*recordBuff);
ssize_t readSize = lpRecorder->read(
recordBuff + offsetInSamples, sizeInBytes, isReadBlocking == JNI_TRUE /* blocking */);
envReleaseArrayElements(env, javaAudioData, recordBuff, 0);
if (readSize < 0) {
return interpretReadSizeError(readSize);
}
return (jint)(readSize / sizeof(*recordBuff));
}
……………………
{"native_read_in_byte_array",
"([BIIZ)I",
(void *)android_media_AudioRecord_readInArray<jbyteArray>},
{"native_read_in_short_array",
"([SIIZ)I",
(void *)android_media_AudioRecord_readInArray<jshortArray>},
{"native_read_in_float_array",
"([FIIZ)I",
(void *)android_media_AudioRecord_readInArray<jfloatArray>},
……………………
javaAudioData为java层传入的byte数组，用于存放获取到的audio数据，通过native层的AudioRecord对象的read()函数，并根据传入的一些参数进行read指定位置上的audio数据；
Native read
ssize_t AudioRecord::read(void* buffer, size_t userSize, bool blocking)
{
if (mTransfer != TRANSFER_SYNC) {
return INVALID_OPERATION;
}
if (ssize_t(userSize) < 0 || (buffer == NULL && userSize != 0)) {
// sanity-check. user is most-likely passing an error code, and it would
// make the return value ambiguous (actualSize vs error).
ALOGE("AudioRecord::read(buffer=%p, size=%zu (%zu)", buffer, userSize, userSize);
return BAD_VALUE;
}
ssize_t read = 0;
Buffer audioBuffer;
while (userSize >= mFrameSize) {
audioBuffer.frameCount = userSize / mFrameSize;
status_t err = obtainBuffer(&audioBuffer,
blocking ? &ClientProxy::kForever : &ClientProxy::kNonBlocking);
if (err < 0) {
if (read > 0) {
break;
}
if (err == TIMED_OUT || err == -EINTR) {
err = WOULD_BLOCK;
}
return ssize_t(err);
}
size_t bytesRead = audioBuffer.size;
memcpy(buffer, audioBuffer.i8, bytesRead);
buffer = ((char *) buffer) + bytesRead;
userSize -= bytesRead;
read += bytesRead;
releaseBuffer(&audioBuffer);
}
if (read > 0) {
mFramesRead += read / mFrameSize;
// mFramesReadTime = systemTime(SYSTEM_TIME_MONOTONIC); // not provided at this time.
}
return read;
}
在该函数中，通过调用obtainBuffer()函数，将audio数据填充到了audioBuffer Buffer对象中，然后通过memcpy()函数将audioBuffer中的数据拷贝到JNI层传入的buffer对象中，然后调用releaseBuffer()函数释放native层创建的Buffer对象；
obtainBuffer
status_t AudioRecord::obtainBuffer(Buffer* audioBuffer, int32_t waitCount, size_t *nonContig)
{
if (audioBuffer == NULL) {
if (nonContig != NULL) {
*nonContig = 0;
}
return BAD_VALUE;
}
if (mTransfer != TRANSFER_OBTAIN) {
audioBuffer->frameCount = 0;
audioBuffer->size = 0;
audioBuffer->raw = NULL;
if (nonContig != NULL) {
*nonContig = 0;
}
return INVALID_OPERATION;
}
const struct timespec *requested;
struct timespec timeout;
if (waitCount == -1) {
requested = &ClientProxy::kForever;
} else if (waitCount == 0) {
requested = &ClientProxy::kNonBlocking;
} else if (waitCount > 0) {
long long ms = WAIT_PERIOD_MS * (long long) waitCount;
timeout.tv_sec = ms / 1000;
timeout.tv_nsec = (int) (ms % 1000) * 1000000;
requested = &timeout;
} else {
ALOGE("%s invalid waitCount %d", __func__, waitCount);
requested = NULL;
}
return obtainBuffer(audioBuffer, requested, NULL /*elapsed*/, nonContig);
}
status_t AudioRecord::obtainBuffer(Buffer* audioBuffer, const struct timespec *requested,
struct timespec *elapsed, size_t *nonContig)
{
// previous and new IAudioRecord sequence numbers are used to detect track re-creation
uint32_t oldSequence = 0;
uint32_t newSequence;
Proxy::Buffer buffer;
status_t status = NO_ERROR;
static const int32_t kMaxTries = 5;
int32_t tryCounter = kMaxTries;
do {
// obtainBuffer() is called with mutex unlocked, so keep extra references to these fields to
// keep them from going away if another thread re-creates the track during obtainBuffer()
sp<AudioRecordClientProxy> proxy;
sp<IMemory> iMem;
sp<IMemory> bufferMem;
{
// start of lock scope
AutoMutex lock(mLock);
newSequence = mSequence;
// did previous obtainBuffer() fail due to media server death or voluntary invalidation?
if (status == DEAD_OBJECT) {
// re-create track, unless someone else has already done so
if (newSequence == oldSequence) {
status = restoreRecord_l("obtainBuffer");
if (status != NO_ERROR) {
buffer.mFrameCount = 0;
buffer.mRaw = NULL;
buffer.mNonContig = 0;
break;
}
}
}
oldSequence = newSequence;
// Keep the extra references
proxy = mProxy;
iMem = mCblkMemory;
bufferMem = mBufferMemory;
// Non-blocking if track is stopped
if (!mActive) {
requested = &ClientProxy::kNonBlocking;
}
} // end of lock scope
buffer.mFrameCount = audioBuffer->frameCount;
// FIXME starts the requested timeout and elapsed over from scratch
status = proxy->obtainBuffer(&buffer, requested, elapsed);
} while ((status == DEAD_OBJECT) && (tryCounter-- > 0));
audioBuffer->frameCount = buffer.mFrameCount;
audioBuffer->size = buffer.mFrameCount * mFrameSize;
audioBuffer->raw = buffer.mRaw;
if (nonContig != NULL) {
*nonContig = buffer.mNonContig;
}
return status;
}
计算出可读的buffer大小并设置相应的标识，首先先创建了一个Proxy::Buffer对象，用该对象对FIFO空间进行操作获取audio数据，获取完成之后，将audioBuffer->frameCount赋值给buffer.mFrameCount，告知需要获取的frame count，然后通过frame count来获取指定大小的audio data；
AudioRecord 写入文件
这个逻辑其实就是常见的通过IOStream将读取到的byte数据写入到文件中，暂不做分析；
