Android-LowmemoryKiller机制
1、为什么引入LowmemoryKiller? 进程的启动分冷启动和热启动,当用户退出某一个进程的时候,并不会真正的将进程退出,而是将这个进程放到后台,以便下次启动的时候可以马上启动起来,这个过程名为热启动,这也是Android的设计理念之一。这个机制会带来一个问题,每个进程都有自己独立的内存地址空间,随着应用打开数量的增多,系统已使用的内存越来越大,就很有可能导致系统内存不足。为了解决这个问题,系统引入LowmemoryKiller(简称lmk)管理所有进程,根据一定策略来kill某个进程并释放占用的内存,保证系统的正常运行。
2、 LMK基本原理? 所有应用进程都是从zygote孵化出来的,记录在AMS中mLruProcesses列表中,由AMS进行统一管理,AMS中会根据进程的状态更新进程对应的oom_adj值,这个值会通过文件传递到kernel中去,kernel有个低内存回收机制,在内存达到一定阀值时会触发清理oom_adj值高的进程腾出更多的内存空间,这就是Lowmemorykiller工作原理。
3、LMK基本实现方案 所以根据不同的配置,就有对应的杀进程标准,这个标准用minfree和adj两个文件来定义: /sys/module/lowmemorykiller/parameters/minfree:里面是以","分割的一组数,每个数字代表一个内存级别。 /sys/module/lowmemorykiller/parameters/adj:对应上面的一组数,每个数组代表一个进程优先级级别
wangjing@wangjing-OptiPlex-7050:~$ adb root
restarting adbd as root
wangjing@wangjing-OptiPlex-7050:~$ adb shell
jason:/ # cat /sys/module/lowmemorykiller/parameters/minfree
18432,23040,27648,32256,55296,80640
jason:/ #
jason:/ # cat /sys/module/lowmemorykiller/parameters/adj
0,100,200,300,900,906
jason:/ #
minfree中数值的单位是内存中的页面数量,一般情况下一个页面是4KB,当内存低于80640的时候,系统会杀死adjj>=906级别的进程,当内存低于55296的时候,系统会杀死adj>=900级别的进程。不同配置的机器这两个文件会有区别,我把minfree文件中的值理解成五个水位线,而adj这个文件中的值与minfree文件中的数值一一对应,意味着到达什么样的水位线,杀死对应数值的进程。 对于应用进程来说,也需要有自身的adj,由AMS负责更新。定义在oom_adj和oom_score_adj文件中: /proc/pid/oom_adj:代表当前进程的优先级,这个优先级是kernel中的优先级。 /proc/pid/oom_score_adj:这个是AMS上层的优先级,与ProcessList中的优先级对应 比如查看一下头条进程的adj值,如下:
jason:/ # ps -ef |grep news
u0_a159 7113 1119 8 15:21:12 ? 00:00:11 com.ss.android.article.news
u0_a159 7188 1119 0 15:21:12 ? 00:00:00 com.ss.android.article.news:ad
u0_a159 7299 1119 1 15:21:16 ? 00:00:02 com.ss.android.article.news:push
u0_a159 7384 1119 1 15:21:17 ? 00:00:00 com.ss.android.article.news:pushservice
root 7838 6429 3 15:23:35 pts/0 00:00:00 grep news
jason:/ # cat proc/7113/oom_adj
0
jason:/ # cat proc/7113/oom_score_adj
0
jason:/ # cat proc/7113/oom_adj
12
jason:/ # cat proc/7113/oom_score_adj
700
jason:/ #
当头条位于前台进程的时候oom_adj值为0,oom_score_adj值也是0,当退出成为后台进程的时候,oom_adj值为12,oom_score_adj值是700。 其实oom_adj与oom_score_adj这两个值是有换算关系的。
kernel/drivers/staging/android/lowmemorykiller.c
271static short lowmem_oom_adj_to_oom_score_adj(short oom_adj)
272{
273 if (oom_adj == OOM_ADJUST_MAX)
274 return OOM_SCORE_ADJ_MAX;
275 else
276 return (oom_adj * OOM_SCORE_ADJ_MAX) / -OOM_DISABLE;
277}
其中OOM_ADJUST_MAX=-15,OOM_SCORE_ADJ_MAX=1000,OOM_DISABLE=-17,那么换算就是:oom_score_adj=12*1000/17=700。高版本的内核都不在使用oom_adj,而是用oom_score_adj,oom_score_adj是一个向后兼容。 综上总结一下LMK的基本原理,如下
用户在启动一个进程之后,通常伴随着启动一个Activity游览页面或者一个Service播放音乐等等,这个时候此进程的adj被AMS提高,LMK就不会杀死这个进程,当这个进程要做的事情做完了,退出后台了,此进程的adj很快又被AMS降低。当需要杀死一个进程释放内存时,一般先根据当前手机剩余内存的状态,在minfree节点中找到当前等级,再根据这个等级去adj节点中找到这个等级应该杀掉的进程的优先级, 之后遍历所有进程并比较进程优先级adj与优先级阈值,并杀死优先级低于阈值的进程,达到释放内存的目的。
总的来说,Framework层通过调整adj的值和阈值数组,输送给kernel中的lmk,为lmk提供杀进程的原材料,因为用户空间和内核空间相互隔离,就采用了文件节点进行通讯,用socket将adj的值与阈值数组传给lmkd(5.0之后不在由AMS直接与lmk通信,引入lmkd守护进程),lmkd将这些值写到内核节点中。lmk通过读取这些节点,实现进程的kill,所以整个lmk机制大概可分成三层。
1.Framework层
AMS中与adj调整的有三个核心的方法,如下 • AMS.updateOomLevelsForDisplay:更新窗口配置,这个过程中,分别向/sys/module/lowmemorykiller/parameters目录下的minfree和adj节点写入相应数值; • OomAdjuster.applyOomAdjLSP:应用adj,当需要杀掉目标进程则返回false;否则返回true,这个过程中,调用setOomAdj(),向/proc/pid/oom_score_adj写入oom_adj 后直接返回; • AMS.cleanUpApplicationRecordLocked & AMS.handleAppDiedLocked:进程死亡后,调用remove(),直接返回;
AMS.updateOomLevelsForDisplay
W\android14\frameworks\base\services\core\java\com\android\server\am\ActivityManagerService.java
public void updateOomLevelsForDisplay(int displayId) {
synchronized(ActivityManagerService.this) {
if (mWindowManager != null) {
mProcessList.applyDisplaySize(mWindowManager);
}
}
}
mProcessList是ProcessList对象,调用applyDisplaySize方法,基于屏幕尺寸,更新LMK的水位线
\android14\frameworks\base\services\core\java\com\android\server\am\ProcessList.java
void applyDisplaySize(WindowManagerService wm) {
if (!mHaveDisplaySize) {
Point p = new Point();
// TODO(multi-display): Compute based on sum of all connected displays' resolutions.
wm.getBaseDisplaySize(Display.DEFAULT_DISPLAY, p);
if (p.x != 0 && p.y != 0) {
updateOomLevels(p.x, p.y, true);
mHaveDisplaySize = true;
}
}
}
传入屏幕的尺寸更新水位线,逻辑很简单
private void updateOomLevels(int displayWidth, int displayHeight, boolean write) {
// Scale buckets from avail memory: at 300MB we use the lowest values to
// 700MB or more for the top values.
float scaleMem = ((float) (mTotalMemMb - 350)) / (700 - 350);
// Scale buckets from screen size.
int minSize = 480 * 800; // 384000
int maxSize = 1280 * 800; // 1024000 230400 870400 .264
float scaleDisp = ((float)(displayWidth * displayHeight) - minSize) / (maxSize - minSize);
if (false) {
Slog.i("XXXXXX", "scaleMem=" + scaleMem);
Slog.i("XXXXXX", "scaleDisp=" + scaleDisp + " dw=" + displayWidth
+ " dh=" + displayHeight);
}
float scale = scaleMem > scaleDisp ? scaleMem : scaleDisp;
if (scale < 0) scale = 0;
else if (scale > 1) scale = 1;
int minfree_adj = Resources.getSystem().getInteger(
com.android.internal.R.integer.config_lowMemoryKillerMinFreeKbytesAdjust);
int minfree_abs = Resources.getSystem().getInteger(
com.android.internal.R.integer.config_lowMemoryKillerMinFreeKbytesAbsolute);
if (false) {
Slog.i("XXXXXX", "minfree_adj=" + minfree_adj + " minfree_abs=" + minfree_abs);
}
final boolean is64bit = Build.SUPPORTED_64_BIT_ABIS.length > 0;
for (int i = 0; i < mOomAdj.length; i++) {
int low = mOomMinFreeLow[i];
int high = mOomMinFreeHigh[i];
if (is64bit) {
// Increase the high min-free levels for cached processes for 64-bit
if (i == 4) high = (high * 3) / 2;
else if (i == 5) high = (high * 7) / 4;
}
mOomMinFree[i] = (int)(low + ((high - low) * scale));
}
if (minfree_abs >= 0) {
for (int i = 0; i < mOomAdj.length; i++) {
mOomMinFree[i] = (int)((float)minfree_abs * mOomMinFree[i]
/ mOomMinFree[mOomAdj.length - 1]);
}
}
if (minfree_adj != 0) {
for (int i = 0; i < mOomAdj.length; i++) {
mOomMinFree[i] += (int)((float) minfree_adj * mOomMinFree[i]
/ mOomMinFree[mOomAdj.length - 1]);
if (mOomMinFree[i] < 0) {
mOomMinFree[i] = 0;
}
}
}
// The maximum size we will restore a process from cached to background, when under
// memory duress, is 1/3 the size we have reserved for kernel caches and other overhead
// before killing background processes.
mCachedRestoreLevel = (getMemLevel(ProcessList.CACHED_APP_MAX_ADJ) / 1024) / 3;
// Ask the kernel to try to keep enough memory free to allocate 3 full
// screen 32bpp buffers without entering direct reclaim.
int reserve = displayWidth * displayHeight * 4 * 3 / 1024;
int reserve_adj = Resources.getSystem().getInteger(
com.android.internal.R.integer.config_extraFreeKbytesAdjust);
int reserve_abs = Resources.getSystem().getInteger(
com.android.internal.R.integer.config_extraFreeKbytesAbsolute);
if (reserve_abs >= 0) {
reserve = reserve_abs;
}
if (reserve_adj != 0) {
reserve += reserve_adj;
if (reserve < 0) {
reserve = 0;
}
}
if (write) {
ByteBuffer buf = ByteBuffer.allocate(4 * (2 * mOomAdj.length + 1));
buf.putInt(LMK_TARGET);
for (int i = 0; i < mOomAdj.length; i++) {
buf.putInt((mOomMinFree[i] * 1024)/PAGE_SIZE);
buf.putInt(mOomAdj[i]);
}
writeLmkd(buf, null);
SystemProperties.set("sys.sysctl.extra_free_kbytes", Integer.toString(reserve));
mOomLevelsSet = true;
}
// GB: 2048,3072,4096,6144,7168,8192
// HC: 8192,10240,12288,14336,16384,20480
}
这里携带的命令协议是LMK_TARGET,它对应到kernel里面执行的函数是cmd_target,要求kernel干的事情就是更新两面两个文件
/sys/module/lowmemorykiller/parameters/minfree
/sys/module/lowmemorykiller/parameters/adj
这两个文件的作用我已经在开头说过了,我把minfree文件中的值理解成五个水位线,而adj这个文件中的值与minfree文件中的数值一一对应,意味着到达什么样的水位线,杀死对应数值的进程。而AMS里面就是通过调用applyDisplaySize方法,基于屏幕尺寸以及机器的CPU位数,更新LMK的水位线的。
AMS.applyOomAdjLocked
在看applyOomAdjLocked方法,这个方法的作用是应用adj,这个过程中,调用setOomAdj(),向/proc/pid/oom_score_adj写入oom_adj 后直接返回;系统中更新adj的操作很频繁,四大组件的生命周期都会影响着adj的值。而更新adj一般由applyOomAdjLocked完成
查看adj
1、cat proc/<pid>/oom_score_adj
2、adb shell dumpsys activity o/p
Android14新增了OomAdjuster.java,经过AMS的一系列调用,最终调用到这个OomAdjuster,再掉ProcessList.setOomAdj
\android14\frameworks\base\services\core\java\com\android\server\am\OomAdjuster.java
private boolean applyOomAdjLSP(ProcessRecord app, boolean doingAll, long now,
long nowElapsed, @OomAdjReason int oomAdjReson) {
boolean success = true;
final ProcessStateRecord state = app.mState;
final UidRecord uidRec = app.getUidRecord();
if (state.getCurRawAdj() != state.getSetRawAdj()) {
state.setSetRawAdj(state.getCurRawAdj());
}
int changes = 0;
if (state.getCurAdj() != state.getSetAdj()) {
mCachedAppOptimizer.onOomAdjustChanged(state.getSetAdj(), state.getCurAdj(), app);
}
if (state.getCurAdj() != state.getSetAdj()) {
if("com.antutu.ABenchMark".equals(app.processName)) {
if(state.getCurAdj() == 0) {
SystemProperties.set("sys.performance", "boost");
} else if (state.getCurAdj() > 0 && state.getSetAdj() == 0) {
SystemProperties.set("sys.performance", "calm");
}
}
ProcessList.setOomAdj(app.getPid(), app.uid, state.getCurAdj());
if (DEBUG_SWITCH || DEBUG_OOM_ADJ || mService.mCurOomAdjUid == app.info.uid) {
String msg = "Set " + app.getPid() + " " + app.processName + " adj "
+ state.getCurAdj() + ": " + state.getAdjType();
reportOomAdjMessageLocked(TAG_OOM_ADJ, msg);
}
state.setSetAdj(state.getCurAdj());
if (uidRec != null) {
uidRec.noteProcAdjChanged();
}
state.setVerifiedAdj(INVALID_ADJ);
}
final int curSchedGroup = state.getCurrentSchedulingGroup();
if (state.getSetSchedGroup() != curSchedGroup) {
int oldSchedGroup = state.getSetSchedGroup();
state.setSetSchedGroup(curSchedGroup);
if (DEBUG_SWITCH || DEBUG_OOM_ADJ || mService.mCurOomAdjUid == app.uid) {
String msg = "Setting sched group of " + app.processName
+ " to " + curSchedGroup + ": " + state.getAdjType();
reportOomAdjMessageLocked(TAG_OOM_ADJ, msg);
}
if (app.getWaitingToKill() != null && app.mReceivers.numberOfCurReceivers() == 0
&& ActivityManager.isProcStateBackground(state.getSetProcState())) {
app.killLocked(app.getWaitingToKill(), ApplicationExitInfo.REASON_USER_REQUESTED,
ApplicationExitInfo.SUBREASON_REMOVE_TASK, true);
success = false;
} else {
int processGroup;
switch (curSchedGroup) {
case SCHED_GROUP_BACKGROUND:
processGroup = THREAD_GROUP_BACKGROUND;
break;
case SCHED_GROUP_TOP_APP:
case SCHED_GROUP_TOP_APP_BOUND:
processGroup = THREAD_GROUP_TOP_APP;
break;
case SCHED_GROUP_RESTRICTED:
processGroup = THREAD_GROUP_RESTRICTED;
break;
default:
processGroup = THREAD_GROUP_DEFAULT;
break;
}
mProcessGroupHandler.sendMessage(mProcessGroupHandler.obtainMessage(
0 /* unused */, app.getPid(), processGroup, app.processName));
try {
final int renderThreadTid = app.getRenderThreadTid();
if (curSchedGroup == SCHED_GROUP_TOP_APP) {
// do nothing if we already switched to RT
if (oldSchedGroup != SCHED_GROUP_TOP_APP) {
app.getWindowProcessController().onTopProcChanged();
if (mService.mUseFifoUiScheduling) {
// Switch UI pipeline for app to SCHED_FIFO
state.setSavedPriority(Process.getThreadPriority(app.getPid()));
mService.scheduleAsFifoPriority(app.getPid(), true);
if (renderThreadTid != 0) {
mService.scheduleAsFifoPriority(renderThreadTid,
/* suppressLogs */true);
if (DEBUG_OOM_ADJ) {
Slog.d("UI_FIFO", "Set RenderThread (TID " +
renderThreadTid + ") to FIFO");
}
} else {
if (DEBUG_OOM_ADJ) {
Slog.d("UI_FIFO", "Not setting RenderThread TID");
}
}
} else {
// Boost priority for top app UI and render threads
setThreadPriority(app.getPid(), THREAD_PRIORITY_TOP_APP_BOOST);
if (renderThreadTid != 0) {
try {
setThreadPriority(renderThreadTid,
THREAD_PRIORITY_TOP_APP_BOOST);
} catch (IllegalArgumentException e) {
// thread died, ignore
}
}
}
}
} else if (oldSchedGroup == SCHED_GROUP_TOP_APP
&& curSchedGroup != SCHED_GROUP_TOP_APP) {
app.getWindowProcessController().onTopProcChanged();
if (mService.mUseFifoUiScheduling) {
try {
// Reset UI pipeline to SCHED_OTHER
setThreadScheduler(app.getPid(), SCHED_OTHER, 0);
setThreadPriority(app.getPid(), state.getSavedPriority());
if (renderThreadTid != 0) {
setThreadScheduler(renderThreadTid,
SCHED_OTHER, 0);
}
} catch (IllegalArgumentException e) {
Slog.w(TAG,
"Failed to set scheduling policy, thread does not exist:\n"
+ e);
} catch (SecurityException e) {
Slog.w(TAG, "Failed to set scheduling policy, not allowed:\n" + e);
}
} else {
// Reset priority for top app UI and render threads
setThreadPriority(app.getPid(), 0);
}
if (renderThreadTid != 0) {
setThreadPriority(renderThreadTid, THREAD_PRIORITY_DISPLAY);
}
}
} catch (Exception e) {
if (DEBUG_ALL) {
Slog.w(TAG, "Failed setting thread priority of " + app.getPid(), e);
}
}
}
}
if (state.hasRepForegroundActivities() != state.hasForegroundActivities()) {
state.setRepForegroundActivities(state.hasForegroundActivities());
changes |= ActivityManagerService.ProcessChangeItem.CHANGE_ACTIVITIES;
}
updateAppFreezeStateLSP(app, oomAdjReson);
if (state.getReportedProcState() != state.getCurProcState()) {
state.setReportedProcState(state.getCurProcState());
if (app.getThread() != null) {
try {
if (false) {
//RuntimeException h = new RuntimeException("here");
Slog.i(TAG, "Sending new process state " + state.getReportedProcState()
+ " to " + app /*, h*/);
}
app.getThread().setProcessState(state.getReportedProcState());
} catch (RemoteException e) {
}
}
}
boolean forceUpdatePssTime = false;
if (state.getSetProcState() == PROCESS_STATE_NONEXISTENT
|| ProcessList.procStatesDifferForMem(
state.getCurProcState(), state.getSetProcState())) {
state.setLastStateTime(now);
forceUpdatePssTime = true;
if (DEBUG_PSS) {
Slog.d(TAG_PSS, "Process state change from "
+ ProcessList.makeProcStateString(state.getSetProcState()) + " to "
+ ProcessList.makeProcStateString(state.getCurProcState()) + " next pss in "
+ (app.mProfile.getNextPssTime() - now) + ": " + app);
}
}
synchronized (mService.mAppProfiler.mProfilerLock) {
app.mProfile.updateProcState(app.mState);
mService.mAppProfiler.updateNextPssTimeLPf(
state.getCurProcState(), app.mProfile, now, forceUpdatePssTime);
}
if (state.getSetProcState() != state.getCurProcState()) {
if (DEBUG_SWITCH || DEBUG_OOM_ADJ || mService.mCurOomAdjUid == app.uid) {
String msg = "Proc state change of " + app.processName
+ " to " + ProcessList.makeProcStateString(state.getCurProcState())
+ " (" + state.getCurProcState() + ")" + ": " + state.getAdjType();
reportOomAdjMessageLocked(TAG_OOM_ADJ, msg);
}
boolean setImportant = state.getSetProcState() < PROCESS_STATE_SERVICE;
boolean curImportant = state.getCurProcState() < PROCESS_STATE_SERVICE;
if (setImportant && !curImportant) {
// This app is no longer something we consider important enough to allow to use
// arbitrary amounts of battery power. Note its current CPU time to later know to
// kill it if it is not behaving well.
state.setWhenUnimportant(now);
app.mProfile.mLastCpuTime.set(0);
}
// Inform UsageStats of important process state change
// Must be called before updating setProcState
maybeUpdateUsageStatsLSP(app, nowElapsed);
maybeUpdateLastTopTime(state, now);
state.setSetProcState(state.getCurProcState());
if (state.getSetProcState() >= ActivityManager.PROCESS_STATE_HOME) {
state.setNotCachedSinceIdle(false);
}
if (!doingAll) {
synchronized (mService.mProcessStats.mLock) {
mService.setProcessTrackerStateLOSP(app,
mService.mProcessStats.getMemFactorLocked());
}
} else {
state.setProcStateChanged(true);
}
} else if (state.hasReportedInteraction()) {
final boolean fgsInteractionChangeEnabled = state.getCachedCompatChange(
CACHED_COMPAT_CHANGE_USE_SHORT_FGS_USAGE_INTERACTION_TIME);
final long interactionThreshold = fgsInteractionChangeEnabled
? mConstants.USAGE_STATS_INTERACTION_INTERVAL_POST_S
: mConstants.USAGE_STATS_INTERACTION_INTERVAL_PRE_S;
// For apps that sit around for a long time in the interactive state, we need
// to report this at least once a day so they don't go idle.
if ((nowElapsed - state.getInteractionEventTime()) > interactionThreshold) {
maybeUpdateUsageStatsLSP(app, nowElapsed);
}
} else {
final boolean fgsInteractionChangeEnabled = state.getCachedCompatChange(
CACHED_COMPAT_CHANGE_USE_SHORT_FGS_USAGE_INTERACTION_TIME);
final long interactionThreshold = fgsInteractionChangeEnabled
? mConstants.SERVICE_USAGE_INTERACTION_TIME_POST_S
: mConstants.SERVICE_USAGE_INTERACTION_TIME_PRE_S;
// For foreground services that sit around for a long time but are not interacted with.
if ((nowElapsed - state.getFgInteractionTime()) > interactionThreshold) {
maybeUpdateUsageStatsLSP(app, nowElapsed);
}
}
if (state.getCurCapability() != state.getSetCapability()) {
state.setSetCapability(state.getCurCapability());
}
final boolean curBoundByNonBgRestrictedApp = state.isCurBoundByNonBgRestrictedApp();
if (curBoundByNonBgRestrictedApp != state.isSetBoundByNonBgRestrictedApp()) {
state.setSetBoundByNonBgRestrictedApp(curBoundByNonBgRestrictedApp);
if (!curBoundByNonBgRestrictedApp && state.isBackgroundRestricted()) {
mService.mHandler.post(() -> {
synchronized (mService) {
mService.mServices.stopAllForegroundServicesLocked(
app.uid, app.info.packageName);
}
});
}
}
if (changes != 0) {
if (DEBUG_PROCESS_OBSERVERS) Slog.i(TAG_PROCESS_OBSERVERS,
"Changes in " + app + ": " + changes);
ActivityManagerService.ProcessChangeItem item =
mProcessList.enqueueProcessChangeItemLocked(app.getPid(), app.info.uid);
item.changes |= changes;
item.foregroundActivities = state.hasRepForegroundActivities();
if (DEBUG_PROCESS_OBSERVERS) Slog.i(TAG_PROCESS_OBSERVERS,
"Item " + Integer.toHexString(System.identityHashCode(item))
+ " " + app.toShortString() + ": changes=" + item.changes
+ " foreground=" + item.foregroundActivities
+ " type=" + state.getAdjType() + " source=" + state.getAdjSource()
+ " target=" + state.getAdjTarget());
}
if (state.isCached() && !state.shouldNotKillOnBgRestrictedAndIdle()) {
// It's eligible to get killed when in UID idle and bg restricted mode,
// check if these states are just flipped.
if (!state.isSetCached() || state.isSetNoKillOnBgRestrictedAndIdle()) {
// Take the timestamp, we'd hold the killing for the background settle time
// (for states debouncing to avoid from thrashing).
state.setLastCanKillOnBgRestrictedAndIdleTime(nowElapsed);
// Kick off the delayed checkup message if needed.
if (mService.mDeterministicUidIdle
|| !mService.mHandler.hasMessages(IDLE_UIDS_MSG)) {
mService.mHandler.sendEmptyMessageDelayed(IDLE_UIDS_MSG,
mConstants.mKillBgRestrictedAndCachedIdleSettleTimeMs);
}
}
}
state.setSetCached(state.isCached());
state.setSetNoKillOnBgRestrictedAndIdle(state.shouldNotKillOnBgRestrictedAndIdle());
return success;
}
这里携带的命令协议是LMK_PROCPRIO,对应kernel里面cmd_procprio函数,要求kernel干的事情是---把AMS发送过来的adj值更新到下面的文件中去。这样内存紧张的时候,LMK就会遍历内核中进程列表,杀死相应adj的进程了
\android14\frameworks\base\services\core\java\com\android\server\am\ProcessList.java
/**
* Set the out-of-memory badness adjustment for a process.
* If {@code pid <= 0}, this method will be a no-op.
*
* @param pid The process identifier to set.
* @param uid The uid of the app
* @param amt Adjustment value -- lmkd allows -1000 to +1000
*
* {@hide}
*/
public static void setOomAdj(int pid, int uid, int amt) {
// This indicates that the process is not started yet and so no need to proceed further.
if (pid <= 0) {
return;
}
if (amt == UNKNOWN_ADJ)
return;
long start = SystemClock.elapsedRealtime();
ByteBuffer buf = ByteBuffer.allocate(4 * 4);
buf.putInt(LMK_PROCPRIO);
buf.putInt(pid);
buf.putInt(uid);
buf.putInt(amt);
//将AMS已经计算好的adj值通过socket发送到lmkd
writeLmkd(buf, null);
long now = SystemClock.elapsedRealtime();
if ((now-start) > 250) {
Slog.w("ActivityManager", "SLOW OOM ADJ: " + (now-start) + "ms for pid " + pid
+ " = " + amt);
}
}
进程死掉后,AMS会调用该进程的ProcessList.remove方法,也会通过Socket通知lmkd更新adj。
AMS.cleanUpApplicationRecordLocked
\android14\frameworks\base\services\core\java\com\android\server\am\ActivityManagerService.java
/**
* Main code for cleaning up a process when it has gone away. This is
* called both as a result of the process dying, or directly when stopping
* a process when running in single process mode.
*
* @return Returns true if the given process has been restarted, so the
* app that was passed in must remain on the process lists.
*/
@GuardedBy("this")
final boolean cleanUpApplicationRecordLocked(ProcessRecord app, int pid,
boolean restarting, boolean allowRestart, int index, boolean replacingPid,
boolean fromBinderDied) {
boolean restart;
synchronized (mProcLock) {
if (index >= 0) {
removeLruProcessLocked(app);
ProcessList.remove(pid);
}
// We don't want to unlinkDeathRecipient immediately, if it's not called from binder
// and it's not isolated, as we'd need the signal to bookkeeping the dying process list.
restart = app.onCleanupApplicationRecordLSP(mProcessStats, allowRestart,
fromBinderDied || app.isolated /* unlinkDeath */);
// Cancel pending frozen task and clean up frozen record if there is any.
mOomAdjuster.mCachedAppOptimizer.onCleanupApplicationRecordLocked(app);
}
mAppProfiler.onCleanupApplicationRecordLocked(app);
for (BroadcastQueue queue : mBroadcastQueues) {
queue.onApplicationCleanupLocked(app);
}
clearProcessForegroundLocked(app);
mServices.killServicesLocked(app, allowRestart);
mPhantomProcessList.onAppDied(pid);
// If the app is undergoing backup, tell the backup manager about it
final BackupRecord backupTarget = mBackupTargets.get(app.userId);
if (backupTarget != null && pid == backupTarget.app.getPid()) {
if (DEBUG_BACKUP || DEBUG_CLEANUP) Slog.d(TAG_CLEANUP, "App "
+ backupTarget.appInfo + " died during backup");
mHandler.post(new Runnable() {
@Override
public void run() {
try {
IBackupManager bm = IBackupManager.Stub.asInterface(
ServiceManager.getService(Context.BACKUP_SERVICE));
bm.agentDisconnectedForUser(app.userId, app.info.packageName);
} catch (RemoteException e) {
// can't happen; backup manager is local
}
}
});
}
mProcessList.scheduleDispatchProcessDiedLocked(pid, app.info.uid);
// If this is a preceding instance of another process instance
allowRestart = mProcessList.handlePrecedingAppDiedLocked(app);
// If somehow this process was still waiting for the death of its predecessor,
// (probably it's "killed" before starting for real), reset the bookkeeping.
final ProcessRecord predecessor = app.mPredecessor;
if (predecessor != null) {
predecessor.mSuccessor = null;
predecessor.mSuccessorStartRunnable = null;
app.mPredecessor = null;
}
// If the caller is restarting this app, then leave it in its
// current lists and let the caller take care of it.
if (restarting) {
return false;
}
if (!app.isPersistent() || app.isolated) {
if (DEBUG_PROCESSES || DEBUG_CLEANUP) Slog.v(TAG_CLEANUP,
"Removing non-persistent process during cleanup: " + app);
if (!replacingPid) {
mProcessList.removeProcessNameLocked(app.processName, app.uid, app);
}
mAtmInternal.clearHeavyWeightProcessIfEquals(app.getWindowProcessController());
} else if (!app.isRemoved()) {
// This app is persistent, so we need to keep its record around.
// If it is not already on the pending app list, add it there
// and start a new process for it.
if (mPersistentStartingProcesses.indexOf(app) < 0) {
mPersistentStartingProcesses.add(app);
restart = true;
}
}
if ((DEBUG_PROCESSES || DEBUG_CLEANUP) && mProcessesOnHold.contains(app)) Slog.v(
TAG_CLEANUP, "Clean-up removing on hold: " + app);
mProcessesOnHold.remove(app);
mAtmInternal.onCleanUpApplicationRecord(app.getWindowProcessController());
mProcessList.noteProcessDiedLocked(app);
if (restart && allowRestart && !app.isolated) {
// We have components that still need to be running in the
// process, so re-launch it.
if (index < 0) {
ProcessList.remove(pid);
}
// Remove provider publish timeout because we will start a new timeout when the
// restarted process is attaching (if the process contains launching providers).
mHandler.removeMessages(CONTENT_PROVIDER_PUBLISH_TIMEOUT_MSG, app);
mProcessList.addProcessNameLocked(app);
app.setPendingStart(false);
mProcessList.startProcessLocked(app, new HostingRecord(
HostingRecord.HOSTING_TYPE_RESTART, app.processName),
ZYGOTE_POLICY_FLAG_EMPTY);
return true;
} else if (pid > 0 && pid != MY_PID) {
// Goodbye!
removePidLocked(pid, app);
mHandler.removeMessages(PROC_START_TIMEOUT_MSG, app);
mBatteryStatsService.noteProcessFinish(app.processName, app.info.uid);
if (app.isolated) {
mBatteryStatsService.removeIsolatedUid(app.uid, app.info.uid);
}
app.setPid(0);
}
return false;
}
AMS.handleAppDiedLocked
\android14\frameworks\base\services\core\java\com\android\server\am\ActivityManagerService.java
/**
* Main function for removing an existing process from the activity manager
* as a result of that process going away. Clears out all connections
* to the process.
*/
@GuardedBy("this")
final void handleAppDiedLocked(ProcessRecord app, int pid,
boolean restarting, boolean allowRestart, boolean fromBinderDied) {
boolean kept = cleanUpApplicationRecordLocked(app, pid, restarting, allowRestart, -1,
false /*replacingPid*/, fromBinderDied);
if (!kept && !restarting) {
removeLruProcessLocked(app);
if (pid > 0) {
ProcessList.remove(pid);
}
}
mAppProfiler.onAppDiedLocked(app);
mAtmInternal.handleAppDied(app.getWindowProcessController(), restarting, () -> {
Slog.w(TAG, "Crash of app " + app.processName
+ " running instrumentation " + app.getActiveInstrumentation().mClass);
Bundle info = new Bundle();
info.putString("shortMsg", "Process crashed.");
finishInstrumentationLocked(app, Activity.RESULT_CANCELED, info);
});
}
@GuardedBy(anyOf = {"this", "mProcLock"})
ProcessRecord getRecordForAppLOSP(IApplicationThread thread) {
if (thread == null) {
return null;
}
return getRecordForAppLOSP(thread.asBinder());
}
@GuardedBy(anyOf = {"this", "mProcLock"})
ProcessRecord getRecordForAppLOSP(IBinder threadBinder) {
if (threadBinder == null) {
return null;
}
ProcessRecord record = mProcessList.getLRURecordForAppLOSP(threadBinder);
if (record != null) return record;
// Validation: if it isn't in the LRU list, it shouldn't exist, but let's double-check that.
final ArrayMap<String, SparseArray<ProcessRecord>> pmap =
mProcessList.getProcessNamesLOSP().getMap();
for (int i = pmap.size()-1; i >= 0; i--) {
final SparseArray<ProcessRecord> procs = pmap.valueAt(i);
for (int j = procs.size()-1; j >= 0; j--) {
final ProcessRecord proc = procs.valueAt(j);
final IApplicationThread procThread = proc.getThread();
if (procThread != null && procThread.asBinder() == threadBinder) {
if (!proc.isPendingFinishAttach()) {
Slog.wtf(TAG, "getRecordForApp: exists in name list but not in LRU list: "
+ proc);
}
return proc;
}
}
}
return null;
}
@GuardedBy("this")
final void appDiedLocked(ProcessRecord app, String reason) {
appDiedLocked(app, app.getPid(), app.getThread(), false, reason);
}
@GuardedBy("this")
final void appDiedLocked(ProcessRecord app, int pid, IApplicationThread thread,
boolean fromBinderDied, String reason) {
// First check if this ProcessRecord is actually active for the pid.
final ProcessRecord curProc;
synchronized (mPidsSelfLocked) {
curProc = mPidsSelfLocked.get(pid);
}
if (curProc != app) {
if (!fromBinderDied || !mProcessList.handleDyingAppDeathLocked(app, pid)) {
Slog.w(TAG, "Spurious death for " + app + ", curProc for " + pid + ": " + curProc);
}
return;
}
mBatteryStatsService.noteProcessDied(app.info.uid, pid);
if (!app.isKilled()) {
if (!fromBinderDied) {
killProcessQuiet(pid);
mProcessList.noteAppKill(app, ApplicationExitInfo.REASON_OTHER,
ApplicationExitInfo.SUBREASON_UNKNOWN, reason);
}
app.killProcessGroupIfNecessaryLocked(true);
synchronized (mProcLock) {
app.setKilled(true);
}
}
// Clean up already done if the process has been re-started.
IApplicationThread appThread;
final int setAdj = app.mState.getSetAdj();
final int setProcState = app.mState.getSetProcState();
if (app.getPid() == pid && (appThread = app.getThread()) != null
&& appThread.asBinder() == thread.asBinder()) {
boolean doLowMem = app.getActiveInstrumentation() == null;
boolean doOomAdj = doLowMem;
if (!app.isKilledByAm()) {
reportUidInfoMessageLocked(TAG,
"Process " + app.processName + " (pid " + pid + ") has died: "
+ ProcessList.makeOomAdjString(setAdj, true) + " "
+ ProcessList.makeProcStateString(setProcState), app.info.uid);
mAppProfiler.setAllowLowerMemLevelLocked(true);
} else {
// Note that we always want to do oom adj to update our state with the
// new number of procs.
mAppProfiler.setAllowLowerMemLevelLocked(false);
doLowMem = false;
}
EventLogTags.writeAmProcDied(app.userId, pid, app.processName, setAdj, setProcState);
if (DEBUG_CLEANUP) Slog.v(TAG_CLEANUP,
"Dying app: " + app + ", pid: " + pid + ", thread: " + thread.asBinder());
handleAppDiedLocked(app, pid, false, true, fromBinderDied);
if (doOomAdj) {
updateOomAdjLocked(OOM_ADJ_REASON_PROCESS_END);
}
if (doLowMem) {
mAppProfiler.doLowMemReportIfNeededLocked(app);
}
} else if (app.getPid() != pid) {
// A new process has already been started.
reportUidInfoMessageLocked(TAG,
"Process " + app.processName + " (pid " + pid
+ ") has died and restarted (pid " + app.getPid() + ").", app.info.uid);
EventLogTags.writeAmProcDied(app.userId, app.getPid(), app.processName,
setAdj, setProcState);
} else if (DEBUG_PROCESSES) {
Slog.d(TAG_PROCESSES, "Received spurious death notification for thread "
+ thread.asBinder());
}
// On the device which doesn't have Cgroup, log LmkStateChanged which is used as a signal
// for pulling memory stats of other running processes when this process died.
if (!hasMemcg()) {
FrameworkStatsLog.write(FrameworkStatsLog.APP_DIED, SystemClock.elapsedRealtime());
}
}
ProcessList.remove
\android14\frameworks\base\services\core\java\com\android\server\am\ProcessList.java
/*
* {@hide}
*/
public static final void remove(int pid) {
// This indicates that the process is not started yet and so no need to proceed further.
if (pid <= 0) {
return;
}
ByteBuffer buf = ByteBuffer.allocate(4 * 2);
buf.putInt(LMK_PROCREMOVE);
buf.putInt(pid);
writeLmkd(buf, null);
}
这里携带的命令协议是LMK_PROCREMOVE,对应kernel里面的cmd_procremove函数,要求kernel干的事情是,当进程死亡了,删除/proc/下面的文件。
上面三大方法最后都是通过writeLmkd与lmkd通信,现在看看writeLmkd中怎么和lmkd通信的,首先需要打开与lmkd通信的socket,lmkd创建名称为lmkd的socket,节点位于/dev/socket/lmkd
\android14\frameworks\base\services\core\java\com\android\server\am\ProcessList.java
private static boolean writeLmkd(ByteBuffer buf, ByteBuffer repl) {
if (!sLmkdConnection.isConnected()) {
// try to connect immediately and then keep retrying
sKillHandler.sendMessage(
sKillHandler.obtainMessage(KillHandler.LMKD_RECONNECT_MSG));
// wait for connection retrying 3 times (up to 3 seconds)
if (!sLmkdConnection.waitForConnection(3 * LMKD_RECONNECT_DELAY_MS)) {
return false;
}
}
return sLmkdConnection.exchange(buf, repl);
}
void init(ActivityManagerService service, ActiveUids activeUids,
PlatformCompat platformCompat) {
sKillThread = new ServiceThread(TAG + ":kill",
THREAD_PRIORITY_BACKGROUND, true /* allowIo */);
sKillThread.start();
sKillHandler = new KillHandler(sKillThread.getLooper());
sLmkdConnection = new LmkdConnection(sKillThread.getLooper().getQueue(),
new LmkdConnection.LmkdConnectionListener() {
@Override
public boolean onConnect(OutputStream ostream) {
Slog.i(TAG, "Connection with lmkd established");
return onLmkdConnect(ostream);
}
@Override
public void onDisconnect() {
Slog.w(TAG, "Lost connection to lmkd");
// start reconnection after delay to let lmkd restart
sKillHandler.sendMessageDelayed(sKillHandler.obtainMessage(
KillHandler.LMKD_RECONNECT_MSG), LMKD_RECONNECT_DELAY_MS);
}
@Override
public boolean isReplyExpected(ByteBuffer replyBuf,
ByteBuffer dataReceived, int receivedLen) {
// compare the preambule (currently one integer) to check if
// this is the reply packet we are waiting for
return (receivedLen == replyBuf.array().length &&
dataReceived.getInt(0) == replyBuf.getInt(0));
}
@Override
public boolean handleUnsolicitedMessage(DataInputStream inputData,
int receivedLen) {
if (receivedLen < 4) {
return false;
}
try {
switch (inputData.readInt()) {
case LMK_PROCKILL:
if (receivedLen != 12) {
return false;
}
final int pid = inputData.readInt();
final int uid = inputData.readInt();
mAppExitInfoTracker.scheduleNoteLmkdProcKilled(pid, uid);
return true;
case LMK_KILL_OCCURRED:
if (receivedLen
< LmkdStatsReporter.KILL_OCCURRED_MSG_SIZE) {
return false;
}
// Note: directly access
// ActiveServices.sNumForegroundServices, do not try to
// hold AMS lock here, otherwise it is a potential deadlock.
Pair<Integer, Integer> foregroundServices =
ActiveServices.sNumForegroundServices.get();
LmkdStatsReporter.logKillOccurred(inputData,
foregroundServices.first,
foregroundServices.second);
return true;
case LMK_STATE_CHANGED:
if (receivedLen
!= LmkdStatsReporter.STATE_CHANGED_MSG_SIZE) {
return false;
}
final int state = inputData.readInt();
LmkdStatsReporter.logStateChanged(state);
return true;
default:
return false;
}
} catch (IOException e) {
Slog.e(TAG, "Invalid buffer data. Failed to log LMK_KILL_OCCURRED");
}
return false;
}
}
);
}
\android14\frameworks\base\services\core\java\com\android\server\am\LmkdConnection.java
public boolean connect() {
synchronized (mLmkdSocketLock) {
if (mLmkdSocket != null) {
return true;
}
// temporary sockets and I/O streams
final LocalSocket socket = openSocket();
if (socket == null) {
Slog.w(TAG, "Failed to connect to lowmemorykiller, retry later");
return false;
}
final OutputStream ostream;
final InputStream istream;
try {
ostream = socket.getOutputStream();
istream = socket.getInputStream();
} catch (IOException ex) {
IoUtils.closeQuietly(socket);
return false;
}
// execute onConnect callback
if (mListener != null && !mListener.onConnect(ostream)) {
Slog.w(TAG, "Failed to communicate with lowmemorykiller, retry later");
IoUtils.closeQuietly(socket);
return false;
}
// connection established
mLmkdSocket = socket;
mLmkdOutputStream = ostream;
mLmkdInputStream = istream;
mMsgQueue.addOnFileDescriptorEventListener(mLmkdSocket.getFileDescriptor(),
EVENT_INPUT | EVENT_ERROR,
new MessageQueue.OnFileDescriptorEventListener() {
public int onFileDescriptorEvents(FileDescriptor fd, int events) {
return fileDescriptorEventHandler(fd, events);
}
}
);
mLmkdSocketLock.notifyAll();
}
return true;
}
public boolean isConnected() {
synchronized (mLmkdSocketLock) {
return (mLmkdSocket != null);
}
}
public boolean waitForConnection(long timeoutMs) {
synchronized (mLmkdSocketLock) {
if (mLmkdSocket != null) {
return true;
}
try {
mLmkdSocketLock.wait(timeoutMs);
return (mLmkdSocket != null);
} catch (InterruptedException e) {
return false;
}
}
}
private LocalSocket openSocket() {
final LocalSocket socket;
try {
socket = new LocalSocket(LocalSocket.SOCKET_SEQPACKET);
socket.connect(
new LocalSocketAddress("lmkd",
LocalSocketAddress.Namespace.RESERVED));
} catch (IOException ex) {
Slog.e(TAG, "Connection failed: " + ex.toString());
return null;
}
return socket;
}
private boolean write(ByteBuffer buf) {
synchronized (mLmkdSocketLock) {
try {
mLmkdOutputStream.write(buf.array(), 0, buf.position());
} catch (IOException ex) {
return false;
}
return true;
}
}
private int read(ByteBuffer buf) {
synchronized (mLmkdSocketLock) {
try {
return mLmkdInputStream.read(buf.array(), 0, buf.array().length);
} catch (IOException ex) {
}
return -1;
}
}
/**
* Exchange a request/reply packets with lmkd
*
* @param req The buffer holding the request data to be sent
* @param repl The buffer to receive the reply
*/
public boolean exchange(ByteBuffer req, ByteBuffer repl) {
if (repl == null) {
return write(req);
}
boolean result = false;
// set reply buffer to user-defined one to fill it
synchronized (mReplyBufLock) {
mReplyBuf = repl;
if (write(req)) {
try {
// wait for the reply
mReplyBufLock.wait();
result = (mReplyBuf != null);
} catch (InterruptedException ie) {
result = false;
}
}
// reset reply buffer
mReplyBuf = null;
}
return result;
}
LMKD
ProcessList中定义有进程的优先级,越重要的进程的优先级越低,前台APP的优先级为0,系统APP的优先级一般都是负值,所以一般进程管理以及杀进程都是针对与上层的APP来说的,而这些进程的优先级调整都在AMS里面,AMS根据进程中的组件的状态去不断的计算每个进程的优先级,计算之后,会及时更新到对应进程的文件节点中,而这个对文件节点的更新并不是它完成的,而是lmkd,他们之间通过socket通信。 lmkd在Android中是一个常驻进程,用来处理上层ActivityManager在进行updateOomAdj之后,通过socket与lmkd进行通信,更新进程的优先级,如果必要则杀掉进程释放内存。lmkd是在init进程启动的时候启动的,在lmkd中有定义lmkd.rc:
\android14\system\memory\lmkd\lmkd.rc
service lmkd /system/bin/lmkd
class core
user lmkd
group lmkd system readproc
capabilities DAC_OVERRIDE KILL IPC_LOCK SYS_NICE SYS_RESOURCE
critical
socket lmkd seqpacket+passcred 0660 system system
task_profiles ServiceCapacityLow
上层AMS跟lmkd通信主要分为三种command,每种command代表一种数据控制方式,在ProcessList以及lmkd中都有定义:
LMK_TARGET:更新/sys/module/lowmemorykiller/parameters/中的minfree以及adj
LMK_PROCPRIO:更新指定进程的优先级,也就是oom_score_adj
LMK_PROCREMOVE:移除进程
在开始介绍lmkd的处理逻辑之前,lmkd.c中有几个重要的变量与数据结构提前说明一下:
// 内存级别限额
#define INKERNEL_MINFREE_PATH "/sys/module/lowmemorykiller/parameters/minfree"
// 不同级别内存对应要杀的的优先级
#define INKERNEL_ADJ_PATH "/sys/module/lowmemorykiller/parameters/adj"
// 装载上面两组数字的数组
static int lowmem_adj[MAX_TARGETS];
static int lowmem_minfree[MAX_TARGETS];
// 三种command
enum lmk_cmd {
LMK_TARGET,
LMK_PROCPRIO,
LMK_PROCREMOVE,
};
// 优先级的最小值
#define OOM_SCORE_ADJ_MIN (-1000)
// 优先级最大值
#define OOM_SCORE_ADJ_MAX 1000
// 双向链表结构体
struct adjslot_list {
struct adjslot_list *next;
struct adjslot_list *prev;
};
// 进程在lmkd中的数据结构体
struct proc {
struct adjslot_list asl;
int pid;
uid_t uid;
int oomadj;
struct proc *pidhash_next;
};
// 存放进程proc的hashtable,index是通过pid的计算得出
static struct proc *pidhash[PIDHASH_SZ];
// 根据pid计算index的hash算法
#define pid_hashfn(x) ((((x) >> 8) ^ (x)) & (PIDHASH_SZ - 1))
// 进程优先级到数组的index之间的转换
// 因为进程的优先级可以是负值,但是数组的index不能为负值
// 不过因为这个转换只是简单加了1000,为了方便,后面的描述中就认为是优先级直接做了index
#define ADJTOSLOT(adj) (adj + -OOM_SCORE_ADJ_MIN)
// table,类似hashtable,不过计算index的方式不是hash,而是oom_score_adj经过转换后直接作为index
// 数组的每个元素都是双向循环链表
// 进程的优先级作为数组的index
// 即以进程的优先级为index,从-1000到+1000 + 1大小的数组,根据优先级,同优先级的进程index相同
// 每个元素是一个双向链表,这个链表上的所有proc的优先级都相同
// 这样根据优先级杀进程的时候就会非常方便,要杀指定优先级的进程可以根据优先级获取到一个进程链表,逐个去杀。
static struct adjslot_list procadjslot_list[ADJTOSLOT(OOM_SCORE_ADJ_MAX) + 1];
lmkd进程启动入口
前面已经提到,这个进程存在的主要作用是跟AMS进行通信,更新oomAdj,在必要的时候杀掉进程。所以在main函数中主要就是创建了epoll以及初始化socket并连接ActivityManager,然后阻塞等待上层传递cmd以及数据过来。
int main(int argc, char **argv) {
if ((argc > 1) && argv[1] && !strcmp(argv[1], "--reinit")) {
if (property_set(LMKD_REINIT_PROP, "")) {
ALOGE("Failed to reset " LMKD_REINIT_PROP " property");
}
return issue_reinit();
}
if (!update_props()) {
ALOGE("Failed to initialize props, exiting.");
return -1;
}
ctx = create_android_logger(KILLINFO_LOG_TAG);
if (!init()) {
if (!use_inkernel_interface) {
/*
* MCL_ONFAULT pins pages as they fault instead of loading
* everything immediately all at once. (Which would be bad,
* because as of this writing, we have a lot of mapped pages we
* never use.) Old kernels will see MCL_ONFAULT and fail with
* EINVAL; we ignore this failure.
*
* N.B. read the man page for mlockall. MCL_CURRENT | MCL_ONFAULT
* pins ⊆ MCL_CURRENT, converging to just MCL_CURRENT as we fault
* in pages.
*/
/* CAP_IPC_LOCK required */
// 将此进程未来使用到的所有内存都锁在物理内存中,防止内存被交换
if (mlockall(MCL_CURRENT | MCL_FUTURE | MCL_ONFAULT) && (errno != EINVAL)) {
ALOGW("mlockall failed %s", strerror(errno));
}
/* CAP_NICE required */
struct sched_param param = {
.sched_priority = 1,
};
// 设置此线程的调度策略为SCHED_FIFO,first-in-first-out,param中主要设置sched_priority
// 由于SCHED_FIFO是一种实时调度策略,在这个策略下优先级从1(low) -> 99(high)
// 实时线程通常会比普通线程有更高的优先级
if (sched_setscheduler(0, SCHED_FIFO, ¶m)) {
ALOGW("set SCHED_FIFO failed %s", strerror(errno));
}
}
// 初始化epoll以及与ActivityManager的socket连接,等待cmd和data
if (init_reaper()) {
ALOGI("Process reaper initialized with %d threads in the pool",
reaper.thread_cnt());
}
if (!watchdog.init()) {
ALOGE("Failed to initialize the watchdog");
}
// 进入死循环epoll_wait等待fd事件
mainloop();
}
android_log_destroy(&ctx);
ALOGI("exiting");
return 0;
}
init初始化
在初始化的时候,有一个很重要的判断:use_inkernel_interface,这个是根据是否有/sys/module/lowmemorykiller/parameters/minfree的写权限来判断的,没有的情况下就使用kernel空间的逻辑
static int init(void) {
static struct event_handler_info kernel_poll_hinfo = { 0, kernel_event_handler };
struct reread_data file_data = {
.filename = ZONEINFO_PATH,
.fd = -1,
};
struct epoll_event epev;
int pidfd;
int i;
int ret;
page_k = sysconf(_SC_PAGESIZE);
if (page_k == -1)
page_k = PAGE_SIZE;
page_k /= 1024;
epollfd = epoll_create(MAX_EPOLL_EVENTS);
if (epollfd == -1) {
ALOGE("epoll_create failed (errno=%d)", errno);
return -1;
}
// mark data connections as not connected
for (int i = 0; i < MAX_DATA_CONN; i++) {
data_sock[i].sock = -1;
}
// 拿到lmkd的socket fd
ctrl_sock.sock = android_get_control_socket("lmkd");
if (ctrl_sock.sock < 0) {
ALOGE("get lmkd control socket failed");
return -1;
}
ret = listen(ctrl_sock.sock, MAX_DATA_CONN);
if (ret < 0) {
ALOGE("lmkd control socket listen failed (errno=%d)", errno);
return -1;
}
epev.events = EPOLLIN;
// ctrl_connect_handler里面完成了soclet的accpet以及read数据,并对数据进行相应的处理
ctrl_sock.handler_info.handler = ctrl_connect_handler;
epev.data.ptr = (void *)&(ctrl_sock.handler_info);
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, ctrl_sock.sock, &epev) == -1) {
ALOGE("epoll_ctl for lmkd control socket failed (errno=%d)", errno);
return -1;
}
maxevents++;
// 使用kernel空间的处理
has_inkernel_module = !access(INKERNEL_MINFREE_PATH, W_OK);
use_inkernel_interface = has_inkernel_module;
if (use_inkernel_interface) {
ALOGI("Using in-kernel low memory killer interface");
if (init_poll_kernel()) {
epev.events = EPOLLIN;
epev.data.ptr = (void*)&kernel_poll_hinfo;
if (epoll_ctl(epollfd, EPOLL_CTL_ADD, kpoll_fd, &epev) != 0) {
ALOGE("epoll_ctl for lmk events failed (errno=%d)", errno);
close(kpoll_fd);
kpoll_fd = -1;
} else {
maxevents++;
/* let the others know it does support reporting kills */
property_set("sys.lmk.reportkills", "1");
}
}
} else {
if (!init_monitors()) {
return -1;
}
/* let the others know it does support reporting kills */
property_set("sys.lmk.reportkills", "1");
}
for (i = 0; i <= ADJTOSLOT(OOM_SCORE_ADJ_MAX); i++) {
// 双向链表初始化
procadjslot_list[i].next = &procadjslot_list[i];
procadjslot_list[i].prev = &procadjslot_list[i];
}
memset(killcnt_idx, KILLCNT_INVALID_IDX, sizeof(killcnt_idx));
/*
* Read zoneinfo as the biggest file we read to create and size the initial
* read buffer and avoid memory re-allocations during memory pressure
*/
if (reread_file(&file_data) == NULL) {
ALOGE("Failed to read %s: %s", file_data.filename, strerror(errno));
}
/* check if kernel supports pidfd_open syscall */
pidfd = TEMP_FAILURE_RETRY(pidfd_open(getpid(), 0));
if (pidfd < 0) {
pidfd_supported = (errno != ENOSYS);
} else {
pidfd_supported = true;
close(pidfd);
}
ALOGI("Process polling is %s", pidfd_supported ? "supported" : "not supported" );
// 这里是运行添加白名单策略的,解析白名单,跳过白名单里的进程
if (!lmkd_init_hook()) {
ALOGE("Failed to initialize LMKD hooks.");
return -1;
}
return 0;
}
进入loop循环mainloop
static void mainloop(void) {
struct event_handler_info* handler_info;
struct polling_params poll_params;
struct timespec curr_tm;
struct epoll_event *evt;
long delay = -1;
poll_params.poll_handler = NULL;
poll_params.paused_handler = NULL;
// 进入死循环,然后调用epoll_wait阻塞等待事件的到来
while (1) {
struct epoll_event events[MAX_EPOLL_EVENTS];
int nevents;
int i;
if (poll_params.poll_handler) {
bool poll_now;
clock_gettime(CLOCK_MONOTONIC_COARSE, &curr_tm);
if (poll_params.update == POLLING_RESUME) {
/* Just transitioned into POLLING_RESUME, poll immediately. */
poll_now = true;
nevents = 0;
} else {
/* Calculate next timeout */
delay = get_time_diff_ms(&poll_params.last_poll_tm, &curr_tm);
delay = (delay < poll_params.polling_interval_ms) ?
poll_params.polling_interval_ms - delay : poll_params.polling_interval_ms;
/* Wait for events until the next polling timeout */
nevents = epoll_wait(epollfd, events, maxevents, delay);
/* Update current time after wait */
clock_gettime(CLOCK_MONOTONIC_COARSE, &curr_tm);
poll_now = (get_time_diff_ms(&poll_params.last_poll_tm, &curr_tm) >=
poll_params.polling_interval_ms);
}
if (poll_now) {
call_handler(poll_params.poll_handler, &poll_params, 0);
}
} else {
if (kill_timeout_ms && is_waiting_for_kill()) {
clock_gettime(CLOCK_MONOTONIC_COARSE, &curr_tm);
delay = kill_timeout_ms - get_time_diff_ms(&last_kill_tm, &curr_tm);
/* Wait for pidfds notification or kill timeout to expire */
nevents = (delay > 0) ? epoll_wait(epollfd, events, maxevents, delay) : 0;
if (nevents == 0) {
/* Kill notification timed out */
stop_wait_for_proc_kill(false);
if (polling_paused(&poll_params)) {
clock_gettime(CLOCK_MONOTONIC_COARSE, &curr_tm);
poll_params.update = POLLING_RESUME;
resume_polling(&poll_params, curr_tm);
}
}
} else {
/* Wait for events with no timeout */
nevents = epoll_wait(epollfd, events, maxevents, -1);
}
}
if (nevents == -1) {
if (errno == EINTR)
continue;
ALOGE("epoll_wait failed (errno=%d)", errno);
continue;
}
/*
* First pass to see if any data socket connections were dropped.
* Dropped connection should be handled before any other events
* to deallocate data connection and correctly handle cases when
* connection gets dropped and reestablished in the same epoll cycle.
* In such cases it's essential to handle connection closures first.
*/
for (i = 0, evt = &events[0]; i < nevents; ++i, evt++) {
if ((evt->events & EPOLLHUP) && evt->data.ptr) {
ALOGI("lmkd data connection dropped");
handler_info = (struct event_handler_info*)evt->data.ptr;
watchdog.start();
ctrl_data_close(handler_info->data);
watchdog.stop();
}
}
/* Second pass to handle all other events */
for (i = 0, evt = &events[0]; i < nevents; ++i, evt++) {
if (evt->events & EPOLLERR) {
ALOGD("EPOLLERR on event #%d", i);
}
if (evt->events & EPOLLHUP) {
/* This case was handled in the first pass */
continue;
}
if (evt->data.ptr) {
handler_info = (struct event_handler_info*)evt->data.ptr;
call_handler(handler_info, &poll_params, evt->events);
}
}
}
}
处理socket传递过来的数据ctrl_command_handler
前面在ctrl_connect_handler这个方法中处理了accept,并开始了ctrl_data_handler中读取数据并进行处理:ctrl_command_handler。对于ActivityManager传递来的Command以及data的主要处理逻辑就在ctrl_command_handler中。上层代码的调用时机这里就不细化了,往前追的话基本都是在ActivityManagerService中的udpateOomAdj中,也就是说上层根据四大组件的状态对进程的优先级进行调整之后,会及时的反应到lmkd中,在内存不足的时候触发杀进程,会从低优先级开始杀进程。
static void ctrl_command_handler(int dsock_idx) {
LMKD_CTRL_PACKET packet;
struct ucred cred;
int len;
enum lmk_cmd cmd;
int nargs;
int targets;
int kill_cnt;
int result;
len = ctrl_data_read(dsock_idx, (char *)packet, CTRL_PACKET_MAX_SIZE, &cred);
if (len <= 0)
return;
if (len < (int)sizeof(int)) {
ALOGE("Wrong control socket read length len=%d", len);
return;
}
cmd = lmkd_pack_get_cmd(packet);
nargs = len / sizeof(int) - 1;
if (nargs < 0)
goto wronglen;
switch(cmd) {
// 更新内存级别以及对应级别的进程adj
case LMK_TARGET:
targets = nargs / 2;
if (nargs & 0x1 || targets > (int)lowmem_adj.size()) {
goto wronglen;
}
cmd_target(targets, packet);
break;
// 根据pid更新adj
case LMK_PROCPRIO:
/* process type field is optional for backward compatibility */
if (nargs < 3 || nargs > 4)
goto wronglen;
cmd_procprio(packet, nargs, &cred);
break;
// 根据pid移除proc
case LMK_PROCREMOVE:
if (nargs != 1)
goto wronglen;
cmd_procremove(packet, &cred);
break;
case LMK_PROCPURGE:
if (nargs != 0)
goto wronglen;
cmd_procpurge(&cred);
break;
case LMK_GETKILLCNT:
if (nargs != 2)
goto wronglen;
kill_cnt = cmd_getkillcnt(packet);
len = lmkd_pack_set_getkillcnt_repl(packet, kill_cnt);
if (ctrl_data_write(dsock_idx, (char *)packet, len) != len)
return;
break;
case LMK_SUBSCRIBE:
if (nargs != 1)
goto wronglen;
cmd_subscribe(dsock_idx, packet);
break;
case LMK_PROCKILL:
/* This command code is NOT expected at all */
ALOGE("Received unexpected command code %d", cmd);
break;
case LMK_UPDATE_PROPS:
if (nargs != 0)
goto wronglen;
result = -1;
if (update_props()) {
if (!use_inkernel_interface) {
/* Reinitialize monitors to apply new settings */
destroy_monitors();
if (init_monitors()) {
result = 0;
}
} else {
result = 0;
}
}
len = lmkd_pack_set_update_props_repl(packet, result);
if (ctrl_data_write(dsock_idx, (char *)packet, len) != len) {
ALOGE("Failed to report operation results");
}
if (!result) {
ALOGI("Properties reinitilized");
} else {
/* New settings can't be supported, crash to be restarted */
ALOGE("New configuration is not supported. Exiting...");
exit(1);
}
break;
default:
ALOGE("Received unknown command code %d", cmd);
return;
}
return;
wronglen:
ALOGE("Wrong control socket read length cmd=%d len=%d", cmd, len);
}
LMK_TARGET
上面的处理逻辑主要是:
- 按照顺序取出数据,装进lmkd的数组中。
- 分别将两个数组中的数取出,用”,”分隔
- lowmem_minfree中的数据拼成的string写到 “/sys/module/lowmemorykiller/parameters/minfree”
- lowmem_adj中的数据拼成的string写到 “/sys/module/lowmemorykiller/parameters/adj”
static void cmd_target(int ntargets, LMKD_CTRL_PACKET packet) {
int i;
struct lmk_target target;
char minfree_str[PROPERTY_VALUE_MAX];
char *pstr = minfree_str;
char *pend = minfree_str + sizeof(minfree_str);
static struct timespec last_req_tm;
struct timespec curr_tm;
if (ntargets < 1 || ntargets > (int)lowmem_adj.size()) {
return;
}
/*
* Ratelimit minfree updates to once per TARGET_UPDATE_MIN_INTERVAL_MS
* to prevent DoS attacks
*/
if (clock_gettime(CLOCK_MONOTONIC_COARSE, &curr_tm) != 0) {
ALOGE("Failed to get current time");
return;
}
if (get_time_diff_ms(&last_req_tm, &curr_tm) <
TARGET_UPDATE_MIN_INTERVAL_MS) {
ALOGE("Ignoring frequent updated to lmkd limits");
return;
}
last_req_tm = curr_tm;
// 这个for循环将数据读出装进数组中
for (i = 0; i < ntargets; i++) {
lmkd_pack_get_target(packet, i, &target);
lowmem_minfree[i] = target.minfree;
lowmem_adj[i] = target.oom_adj_score;
pstr += snprintf(pstr, pend - pstr, "%d:%d,", target.minfree,
target.oom_adj_score);
if (pstr >= pend) {
/* if no more space in the buffer then terminate the loop */
pstr = pend;
break;
}
}
lowmem_targets_size = ntargets;
/* Override the last extra comma */
pstr[-1] = '\0';
property_set("sys.lmk.minfree_levels", minfree_str);
// 使用kernel空间的处理逻辑
if (has_inkernel_module) {
char minfreestr[128];
char killpriostr[128];
minfreestr[0] = '\0';
killpriostr[0] = '\0';
for (i = 0; i < lowmem_targets_size; i++) {
char val[40];
// 取出两个数组中的数据,以","分隔,分别拼接成string
if (i) {
strlcat(minfreestr, ",", sizeof(minfreestr));
strlcat(killpriostr, ",", sizeof(killpriostr));
}
snprintf(val, sizeof(val), "%d", use_inkernel_interface ? lowmem_minfree[i] : 0);
strlcat(minfreestr, val, sizeof(minfreestr));
snprintf(val, sizeof(val), "%d", use_inkernel_interface ? lowmem_adj[i] : 0);
strlcat(killpriostr, val, sizeof(killpriostr));
}
// 将生成好的string写入到文件节点minfree以及adj
writefilestring(INKERNEL_MINFREE_PATH, minfreestr, true);
writefilestring(INKERNEL_ADJ_PATH, killpriostr, true);
}
}
LMK_PROCPRIO
static void cmd_procprio(LMKD_CTRL_PACKET packet, int field_count, struct ucred *cred) {
struct proc *procp;
char path[LINE_MAX];
char val[20];
int soft_limit_mult;
struct lmk_procprio params;
bool is_system_server;
struct passwd *pwdrec;
int64_t tgid;
char buf[PAGE_SIZE];
lmkd_pack_get_procprio(packet, field_count, ¶ms);
if (params.oomadj < OOM_SCORE_ADJ_MIN ||
params.oomadj > OOM_SCORE_ADJ_MAX) {
ALOGE("Invalid PROCPRIO oomadj argument %d", params.oomadj);
return;
}
if (params.ptype < PROC_TYPE_FIRST || params.ptype >= PROC_TYPE_COUNT) {
ALOGE("Invalid PROCPRIO process type argument %d", params.ptype);
return;
}
/* Check if registered process is a thread group leader */
if (read_proc_status(params.pid, buf, sizeof(buf))) {
if (parse_status_tag(buf, PROC_STATUS_TGID_FIELD, &tgid) && tgid != params.pid) {
ALOGE("Attempt to register a task that is not a thread group leader "
"(tid %d, tgid %" PRId64 ")", params.pid, tgid);
return;
}
}
/* gid containing AID_READPROC required */
/* CAP_SYS_RESOURCE required */
/* CAP_DAC_OVERRIDE required */
// LMK_PROCPRIO的主要作用就是更新进程的oomAdj
// 将上层传递过来的数据(pid以及优先级)写到该进程对应的文件节点
// /proc/pid/oom_score_adj
snprintf(path, sizeof(path), "/proc/%d/oom_score_adj", params.pid);
snprintf(val, sizeof(val), "%d", params.oomadj);
if (!writefilestring(path, val, false)) {
ALOGW("Failed to open %s; errno=%d: process %d might have been killed",
path, errno, params.pid);
/* If this file does not exist the process is dead. */
return;
}
// 如果使用kernel的使用逻辑,return
// 即这个command传递过来只是更新了对应文件节点的oom_score_adj
if (use_inkernel_interface) {
stats_store_taskname(params.pid, proc_get_name(params.pid, path, sizeof(path)));
return;
}
/* lmkd should not change soft limits for services */
if (params.ptype == PROC_TYPE_APP && per_app_memcg) {
if (params.oomadj >= 900) {
soft_limit_mult = 0;
} else if (params.oomadj >= 800) {
soft_limit_mult = 0;
} else if (params.oomadj >= 700) {
soft_limit_mult = 0;
} else if (params.oomadj >= 600) {
// Launcher should be perceptible, don't kill it.
params.oomadj = 200;
soft_limit_mult = 1;
} else if (params.oomadj >= 500) {
soft_limit_mult = 0;
} else if (params.oomadj >= 400) {
soft_limit_mult = 0;
} else if (params.oomadj >= 300) {
soft_limit_mult = 1;
} else if (params.oomadj >= 200) {
soft_limit_mult = 8;
} else if (params.oomadj >= 100) {
soft_limit_mult = 10;
} else if (params.oomadj >= 0) {
soft_limit_mult = 20;
} else {
// Persistent processes will have a large
// soft limit 512MB.
soft_limit_mult = 64;
}
std::string path;
if (!CgroupGetAttributePathForTask("MemSoftLimit", params.pid, &path)) {
ALOGE("Querying MemSoftLimit path failed");
return;
}
snprintf(val, sizeof(val), "%d", soft_limit_mult * EIGHT_MEGA);
/*
* system_server process has no memcg under /dev/memcg/apps but should be
* registered with lmkd. This is the best way so far to identify it.
*/
is_system_server = (params.oomadj == SYSTEM_ADJ &&
(pwdrec = getpwnam("system")) != NULL &&
params.uid == pwdrec->pw_uid);
writefilestring(path.c_str(), val, !is_system_server);
}
procp = pid_lookup(params.pid);
if (!procp) {
int pidfd = -1;
if (pidfd_supported) {
pidfd = TEMP_FAILURE_RETRY(pidfd_open(params.pid, 0));
if (pidfd < 0) {
ALOGE("pidfd_open for pid %d failed; errno=%d", params.pid, errno);
return;
}
}
procp = static_cast<struct proc*>(calloc(1, sizeof(struct proc)));
if (!procp) {
// Oh, the irony. May need to rebuild our state.
return;
}
procp->pid = params.pid;
procp->pidfd = pidfd;
procp->uid = params.uid;
procp->reg_pid = cred->pid;
procp->oomadj = params.oomadj;
procp->valid = true;
proc_insert(procp);
// 将proc插入到lmkd中的数据结构中,主要包括两个数据结构
// 更新hashtable,通过pid计算hash值,然后存储,解决冲突是让新来的作为数组元素链表的头结点
// 优先级为index的双向链表组成的table
} else {
if (!claim_record(procp, cred->pid)) {
char buf[LINE_MAX];
char *taskname = proc_get_name(cred->pid, buf, sizeof(buf));
/* Only registrant of the record can remove it */
ALOGE("%s (%d, %d) attempts to modify a process registered by another client",
taskname ? taskname : "A process ", cred->uid, cred->pid);
return;
}
// 但是因为优先级的变化,需要先把这个proc从原先的优先级table中对应位置的双向链表中remove
// 然后新加到新的优先级对应的双向链表中
// 双向链表的添加是新来的放在头部
proc_unslot(procp);
procp->oomadj = params.oomadj;
proc_slot(procp);
}
}
LMK_PROCREMOVE
static void cmd_procremove(LMKD_CTRL_PACKET packet, struct ucred *cred) {
struct lmk_procremove params;
struct proc *procp;
lmkd_pack_get_procremove(packet, ¶ms);
if (use_inkernel_interface) {
/*
* Perform an extra check before the pid is removed, after which it
* will be impossible for poll_kernel to get the taskname. poll_kernel()
* is potentially a long-running blocking function; however this method
* handles AMS requests but does not block AMS.
*/
poll_kernel(kpoll_fd);
// 更新数据结构,pid的以及进程优先级的双向链表
stats_remove_taskname(params.pid);
return;
}
procp = pid_lookup(params.pid);
if (!procp) {
return;
}
if (!claim_record(procp, cred->pid)) {
char buf[LINE_MAX];
char *taskname = proc_get_name(cred->pid, buf, sizeof(buf));
/* Only registrant of the record can remove it */
ALOGE("%s (%d, %d) attempts to unregister a process registered by another client",
taskname ? taskname : "A process ", cred->uid, cred->pid);
return;
}
/*
* WARNING: After pid_remove() procp is freed and can't be used!
* Therefore placed at the end of the function.
*/
pid_remove(params.pid);
}
从上面的处理逻辑就能看出来,三种command的处理逻辑中都对use_inkernel_interface的情况下做了特殊处理,在use_inkernel_interface的情况下,做的事情都是很简单的,只是更新一下文件节点。如果不使用kernel interface,就需要lmkd自己维护两个table,在每次更新adj的时候去更新table。 且在初始化的时候也能看到,如果不使用kernel的lowmemorykiller,则需要lmkd自己获取手机内存状态,如果匹配到了minfree中的等级,则需要通过杀掉一些进程释放内存。
// 根据当前内存的状态查找需要杀掉的进程
/*
* Find one process to kill at or above the given oom_score_adj level.
* Returns size of the killed process.
*/
static int find_and_kill_process(int min_score_adj, struct kill_info *ki, union meminfo *mi,
struct wakeup_info *wi, struct timespec *tm,
struct psi_data *pd) {
int i;
int killed_size = 0;
bool lmk_state_change_start = false;
bool choose_heaviest_task = kill_heaviest_task;
for (i = OOM_SCORE_ADJ_MAX; i >= min_score_adj; i--) {
struct proc *procp;
if (!choose_heaviest_task && i <= PERCEPTIBLE_APP_ADJ) {
/*
* If we have to choose a perceptible process, choose the heaviest one to
* hopefully minimize the number of victims.
*/
choose_heaviest_task = true;
}
while (true) {
procp = choose_heaviest_task ?
proc_get_heaviest(i) : proc_adj_tail(i);
if (!procp)
break;
// 杀进程,通过发信号的方式
// 返回值是杀了该进程之后释放的内存的大小
// 如果释放内存之后依然不满足要求,则从链表上再取一个杀
killed_size = kill_one_process(procp, min_score_adj, ki, mi, wi, tm, pd);
if (killed_size >= 0) {
if (!lmk_state_change_start) {
lmk_state_change_start = true;
stats_write_lmk_state_changed(STATE_START);
}
break;
}
}
if (killed_size) {
break;
}
}
if (lmk_state_change_start) {
stats_write_lmk_state_changed(STATE_STOP);
}
return killed_size;
}
由于Android中的进程启动的很频繁,四大组件都会涉及到进程启动,进程启动之后做完组要做的事情之后就会很快被AMS把优先级降低,但是为了针对低内存的情况以及如果用户开启太多,且APP的优先级很高,AMS这边就有一些无力了,为了保证手机正常运行必须有进程清理,内存回收,根据当前手机剩余内存的状态,在minfree中找到当前等级,再根据这个等级去adj中找到这个等级应该杀掉的进程的优先级,然后去杀进程,直到释放足够的内存。目前大多都使用kernel中的lowmemorykiller,但是上层用户的APP的优先级的调整还是AMS来完成的,lmkd在中间充当了一个桥梁的角色,通过把上层的更新之后的adj写入到文件节点,提供lowmemorykiller杀进程的依据。
[init.svc.lmkd]: [running]
[init.svc_debug_pid.lmkd]: [314]
[persist.sys.lmk.reportkills]: [true]
[ro.boottime.lmkd]: [2119975666]
[ro.lmk.filecache_min_kb]: [10240]
[ro.lmk.psi_complete_stall_ms]: [100]
[ro.lmk.psi_partial_stall_ms]: [70]
[ro.lmk.thrashing_limit]: [60]
[ro.lmk.thrashing_limit_decay]: [30]
[ro.lmk.use_new_strategy]: [1]
[sys.lmk.minfree_levels]: [18432:0,23040:100,27648:200,32256:250,55296:900,80640:950]
[sys.lmk.reportkills]: [1]