Android S 输入法相关新特性整理
1.首先在S上，对输入法的target窗口有了更准确的命名
具体如下：
| R版本 | S版本 | 定义 |
|---|---|---|
| InputMethodTarget | mImeLayeringTarget | 输入法放置的窗口 |
| InputMethodInputTarget | mImeInputTarget | 输入法输入的目标窗口 |
| InputMethodControlTarget | mImeControlTarget | 输入法的控制窗口 |
2.然后谷歌主要通过6笔change来实现更好的输入法 transition体验
其主要思想是通过保存输入法的snapshot ，来保证输入法在进入最近任务界面以及在app的transition过程中能够始终保证可见性，从而提高更好的使用体验。
2.1 首先第一笔change主要有以下几笔修改：
commit 60222bf60db8e7ccfa9d025ab1b24d7f7e31eea9
Author: Ming-Shin Lu <lumark@google.com>
Date: Mon Sep 14 00:07:04 2020 +0800
Better IME transition while switching app with recents (1/N)
Changes includes:
- In DC#computeImeTarget, keeps the IME target with the last window
while swiping up to recents to prevent flickering due to IME hide
animation on top of recents.
- Screenshot Task with IME window if IME is attatched to the app.
- Ignore hiding current IME if IME is attached to the app.
DisplayContent#computeImeTarget方法主要是在window层级变化时重新计算返回当前的输入法target窗口，第一笔change在该方法内添加逻辑判断，当新的target窗口是最近任务窗口同时之前的target窗口的输入法是可见的，那么直接返回之前的target窗口，无需再计算新的target窗口，等待需要显示输入法或者重新relayout时再计算输入法target窗口
- 在
RecentsAnimationController#hideCurrentInputMethod方法内添加逻辑，首先先调用updateImeParent方法保证ImeParent始终正确，然后就是判断当前输入法有没有attach到app上，如果输入法当前已经attach到app上，那么在进入最近任务界面时就不执行其隐藏动画
在
TaskSnapShotController#createTaskSnapshot方法,在添加excludeLayer时添加判断条件，原先只要输入法窗口不为null，就把输入法的surface添加到excludelayers内，现在是同时当输入法没有attach到app上时，才将其添加到excludelayers内。换句话说，如果当前输入法attach到app上时，创建snapShot的时候是可以将其显示出来的。
2.2 第二笔change主要修改如下：
commit 73aab1dc1fce55a544f5ecffe1c4cb7939d77be9
Author: Ming-Shin Lu <lumark@google.com>
Date: Mon Sep 14 12:35:41 2020 +0800
Better IME transition while switching app with recents (2/N)
In TaskSnapshotController#handleClosingApps will capture task snapshot
while applying closing animation or during screen turning-off.
this will overwrite the captured task from RecentsAnimationController,
and makes while quick switching tasks, sometimes will not see the IME
surface on the task snapshot.
Add a check in places which triggers task snapshot to ignore addidnal
snapshot request while RecentsAnimation is active.
Also refined Task#isTaskAnimating as isAnimatingByRecents to simplify
the logic and add the test for its behavior.
Bug: 166736352
Test: manual by:
1) Tapping EditText in app and make keyboard shown
2) Quick switch back and force between apps by swiping navbar
3) Verify if the task snapshot with IME persists shown during
switching apps
Test: atest RecentsAnimationControllerTest#testIsAnimatingByRecents
TaskSnapShotController#getClosingTasks内添加逻辑判断，如果某task是正在执行最近任务动画，那么在mSkipClosingAppSnapshotTasks内添加该task。因为RecentsAnimation已经在处理task snapshot，这里就没必要处理由RecentsAnimation控制的task了。
2.3 第三笔change主要修改如下：
概括：通过对InsetsSourceControl设置skipAnimation值 ，（由于next task已经存在Ime surface）保证Insetscontroller不会重新应用输入法显示动画
commit d579c94480fa98042dd38721de0e3a18fec67e9e
Author: Ming-Shin Lu <lumark@google.com>
Date: Mon Oct 12 22:10:44 2020 +0800
Better IME transition while switching app with recents (3/N)
- Add hasImeWindowSurface in TaskSnapShot
- Add InsetsSourceControl#getAndClearSkipAnimationOnce for skiping IME showing animation once when starting window with IME surface
通过对InsetsSourceControl设置skipAnimation值 ，（由于next task已经存在Ime surface）保证Insetscontroller不会重新应用输入法显示动画
Test: manual as below steps
0) Enabling developer options -> Quick settings developer tiles ->
Window animation scale to slow down transition animation.
1) Launch an app with focusing an editor to show soft-input
2) Swipe out app task to back to launcher
3) Using quick switch or taping shortcut to bring back the app task
4) Verify that should be no IME showing animation happens during task transition.
Change-Id: I83ffc03119e01da71ad12f8ad8043cf0730dfd50
- TaskSnapShot内添加hasImeSurface值来判断当前snapShot是否存在ime的surface
- InsetsSourceControl内添加mSkipAnimationOnce值以及相关逻辑，InsetsController#applyAnimation
提供布尔值判断当前动画是否需要跳过（比如当前app内显示了输入法，按任务健进入最近任务时，此时输入法的隐藏动画就需要skip）
ImeInsetsSourceProvider#getControl方法，在这里获取InsetsSourceControl时，设置该control的mSkipAnimationOnce值，（主要依据条件就是target窗口的snapshot内是否存在ImeSurface）
InsetsController#applyAnimation，在该方法内如果当前Insets为输入法时，获取其InsetsSourceControl的mSkipAnimationOnce值，并在构造InternalAnimationControlListener时将skipAnim值传递到mDisable变量，通过这个值来控制动画是否enable
2.4 （重点）第四笔change主要修改如下：
概括： 从closing app transit到新的task时，通过showIme的snapshot始终保持输入法的可见性，而在动画结束的时候remove掉输入法的surface
commit 4964839ec3b3648ef64782a7005f6ec18cd85548
Author: Ming-Shin Lu <lumark@google.com>
Date: Fri Oct 16 01:23:55 2020 +0800
Better IME transition while switching app with recents (4/N)
This CL introduces TaskSnapshotController#snapshotImeFromAttachedTask to
attach IME screenshot when performing closing transition.
This can improve app transition without jank or flickering while the IME
insets control transits from closing task to the next task, we keep the
IME surface visibility by placing the IME screenshot with calling
DC#showImeScreenshot to be a part of task while animating
app transition, and then remove it with DC#removeImeScreenshotIfNeeded
when the transition animation finished or no longer used gracefully.
Test: manual as below steps:
1) Launch an app with focusing an editor (e.g. Dialer)
2) Swipe down status bar and tap Settings icon.
3) Verify that when doing task transition animation, app activity
with IME keeps visible.
Test: manual as below steps:
1) With 2-button or 3-button gesture, launch an app with focusing an
editor to show soft-keyboard.
2) Pressing home key
3) Verify the IME screenshot keeps visible and animates with closing
transition smoothly.
DisplayContent#setImeLayingTarget方法内添加调用attachAndShowImeScreenshotOnTarget的逻辑，在设置新的或者null ImeLayingTarget的时候，这时候为了保证优雅过渡，需要保持原先的ime surface，因此在这里添加调用show imeSurface的方法 （这里是创建ime surface的其中一种调用方式）
DisplayContent#attachAndShowImeScreenshotOnTarget，这里在原先的target窗口内创建ime surface
void attachAndShowImeScreenshotOnTarget() {
// No need to attach screenshot if the IME target not exists or screen is off.
if (!isImeAttachedToApp() || !mWmService.mPolicy.isScreenOn()) {
// 满足不需要生成ime surface的条件 直接return
return;
}
final SurfaceControl.Transaction t = getPendingTransaction();
// Prepare IME screenshot for the target if it allows to attach into.
if (mInputMethodWindow != null && mInputMethodWindow.isVisible()) {
final Task task = mImeLayeringTarget.getTask();
// Re-new the IME screenshot when it does not exist or the size changed.
final boolean renewImeSurface = mImeScreenshot == null
|| mImeScreenshot.getWidth() != mInputMethodWindow.getFrame().width()
|| mImeScreenshot.getHeight() != mInputMethodWindow.getFrame().height();
if (task != null && !task.isHomeOrRecentsRootTask()) {
SurfaceControl.ScreenshotHardwareBuffer imeBuffer = renewImeSurface
? mWmService.mTaskSnapshotController.snapshotImeFromAttachedTask(task)
: null;
// 如果原先已存在ime surface 这里先remove掉 然后重新生成
if (imeBuffer != null) {
// Remove the last IME surface when the surface needs to renew.
removeImeSurfaceImmediately();
mImeScreenshot = createImeSurface(imeBuffer, t);
}
}
}
final boolean isValidSnapshot = mImeScreenshot != null && mImeScreenshot.isValid();
// Showing the IME screenshot if the target has already in app transition stage.
// Note that if the current IME insets is not showing, no need to show IME screenshot
// to reflect the true IME insets visibility and the app task layout as possible.
if (isValidSnapshot && getInsetsStateController().getImeSourceProvider().isImeShowing()) {
if (DEBUG_INPUT_METHOD) {
Slog.d(TAG, "show IME snapshot, ime target=" + mImeLayeringTarget);
}
t.show(mImeScreenshot);
} else if (!isValidSnapshot) {
removeImeSurfaceImmediately();
}
}
DisplayContent#createImeSurface该方法主要就是创建ime的SurfaceControl
SurfaceControl createImeSurface(SurfaceControl.ScreenshotHardwareBuffer imeBuffer,
Transaction t) {
final HardwareBuffer buffer = imeBuffer.getHardwareBuffer();
if (DEBUG_INPUT_METHOD) Slog.d(TAG, "create IME snapshot for "
+ mImeLayeringTarget + ", buff width=" + buffer.getWidth()
+ ", height=" + buffer.getHeight());
final ActivityRecord activity = mImeLayeringTarget.mActivityRecord;
final SurfaceControl imeSurface = mWmService.mSurfaceControlFactory.apply(null)
.setName("IME-snapshot-surface")
.setBufferSize(buffer.getWidth(), buffer.getHeight())
.setFormat(buffer.getFormat())
.setParent(activity.getSurfaceControl())
.setCallsite("DisplayContent.attachAndShowImeScreenshotOnTarget")
.build();
// Make IME snapshot as trusted overlay
InputMonitor.setTrustedOverlayInputInfo(imeSurface, t, getDisplayId(),
"IME-snapshot-surface");
Surface surface = mWmService.mSurfaceFactory.get();
surface.copyFrom(imeSurface);
surface.attachAndQueueBufferWithColorSpace(buffer, null);
surface.release();
t.setRelativeLayer(imeSurface, activity.getSurfaceControl(), 1);
t.setPosition(imeSurface, mInputMethodWindow.getDisplayFrame().left,
mInputMethodWindow.getDisplayFrame().top);
return imeSurface;
}
DisplayContent#showImeScreentShot、 removeImeScreenshotIfPossible、removeImeSurfaceImmediately 、onWindowAnimationFinished
这些都是 显示和移除Ime surface的相关方法
showImeScreentShot是在app transition是或者swipe到最近任务时调用（WindowContainer内应用动画时调用该方法）
removeIme是在动画结束是或者ime surface淘汰时如旋转屏引起size变换
WindowContainer#applyAnimationUnchecked在显示跳转动画时，这里调用显示 ime的surface
WindowContainer#doAnimationFinished在动画结束时 回调dc的onWindowAnimationFinished方法（该方法会remove掉 ime的surface）
2.5 第五笔change主要修改如下：
概括： 在updateImeControltarget的时候调用updateImeParent，保证此时输入法的可见性已经确定了，这样可以防止闪烁问题，然后在执行进入最近任务动画过程中，再次check ime parent以及ime是否正确
commit d4d90ac8a88e601a4418c44fe45563070581c19b
Author: Ming-Shin Lu <lumark@google.com>
Date: Wed Nov 11 13:17:18 2020 +0800
Better IME transition while switching app with recents (5/N) 在updateImeControltarget的时候调用updateImeParent，保证此时输入法的可见性已经确定了，这样可以防止闪烁问题、更改InputMonitor防止错按、在旋转动画时保证imeParent的正确性
With CL[1], the IME surface will have snapshot when transtioning to the
next task.
We can now remove the previous hacky logics like dedicates to keep the
previous IME and make the true IME target while task transitioning.
And, move the call of updateImeParent() from
DC#setInputMethodTarget to DC#updateImeControlTarget,
to ensure that the reparenting of IME insets source control can be done
when the IME insets visiblity settled down after the IME insets control
changed, to prevent unnecessary flickering during the time period between
reparenting IME parent and start IME insets animation.
Also, modify UpdateInputForAllWindowsConsumer#accept to let
mRecentsAnimationInputConsumer can be above IME target activity which to
prevent mis-touch or keystoke may left while quickly taping
soft-keyboard during swping up to recents.
DisplayContent#updateImeControlTarget在该方法内更新ime parent，这个是最佳的时机，
（在setImeLayeringTargetInner方法内，原先是在updateImeControlTarget方法之前调用updateImeParent）
RecentsAnimationController#setAnimationTargetsBehindSystemBars，在执行进入最近任务界面动画时，首先更新ime parent保证正确的ime parent，然后再次确认ime是否attach到之前的task上，如果没有，执行隐藏输入法操作
2.5 第六笔change主要修改如下：
概括： 通过设置FrozenInsetsState 来保证正在执行closing的window不会收到非期望的insets变化(比如新window的insets的分发)（也就是InsetsStateController会根据是否存在freeze的InsetsState来判断是否分发insetsChanged），同时添加freeze以及unFreeze Insetsstate的操作
commit 0f536fa99489c5cfd2d266e94798b19540e2cd12
Author: Ming-Shin Lu <lumark@google.com>
Date: Thu Jan 14 21:03:15 2021 +0800
Better IME transition while switching app with recents (6/N)
Add WindowState#{freezeInsetsState, clearFrozenInsetsState} to freeze
the insets state of the window to keep the insets state when the window
is in app exiting transition, to ensure the exiting window won't
receive unexpected insets changes from the next window.
And, in order to easy maintain the logic of dispatching all windows's
insets changed event in InsetsStateController, instead of adding
complicated rule in InsetsStateController to judge if the window can
disatch insets change, we can replace with
WindowState#isReadyToDispatchInsetsState() to wrap the dispatch
judgement.
Bug: 166736352
Test: atest CtsWindowManagerDeviceTestCases WmTests
Test: atest WindowStateTests#testSetFreezeInsetsState
Test: atest WindowContainerTests#testFreezeInsetsStateWhenAppTransition
Change-Id: I56418733c1abbd73e88a8918a5d55ecc15344c5e
WindowState内添加mFrozenInsetsState的定义以及相关方法，意为冻结insets的state，在一些情况下没有必要与client端保证同步，例如在退出动画时
isReadyToDispatchInsetsState方法返回代表是否准备分发insetsState，如果不存在freeze的InsetsState，即可分发
- DisplayContent#onWindowAnimationFinished，在动画结束时清除所有的freeze的insetsState
- InsesStateController 修改mDispatchInsetsChanged 这个consumer，保证分发时不存在freeze的InsetsState
总结
关于S上输入法的新特性大概就总结到这里了，S上的输入法在动画切换以及app transition的时候更好的提高了用户的体验度。 如果大家有何疑问或者建议，欢迎评论留言～ 与大家一起共同进步！～
