Android S 输入法相关新特性整理

wayne啦
1,635 阅读10分钟

Android S 输入法相关新特性整理

1.首先在S上，对输入法的target窗口有了更准确的命名

具体如下：

R版本S版本定义
InputMethodTargetmImeLayeringTarget输入法放置的窗口
InputMethodInputTargetmImeInputTarget输入法输入的目标窗口
InputMethodControlTargetmImeControlTarget输入法的控制窗口

2.然后谷歌主要通过6笔change来实现更好的输入法 transition体验

其主要思想是通过保存输入法的snapshot ，来保证输入法在进入最近任务界面以及在app的transition过程中能够始终保证可见性，从而提高更好的使用体验。

2.1 首先第一笔change主要有以下几笔修改：

commit 60222bf60db8e7ccfa9d025ab1b24d7f7e31eea9
Author: Ming-Shin Lu <lumark@google.com>
Date: Mon Sep 14 00:07:04 2020 +0800

Better IME transition while switching app with recents (1/N)

Changes includes:
- In DC#computeImeTarget, keeps the IME target with the last window
  while swiping up to recents to prevent flickering due to IME hide
  animation on top of recents.
- Screenshot Task with IME window if IME is attatched to the app.
- Ignore hiding current IME if IME is attached to the app.
  • DisplayContent#computeImeTarget方法主要是在window层级变化时重新计算返回当前的输入法target窗口，第一笔change在该方法内添加逻辑判断，当新的target窗口是最近任务窗口同时之前的target窗口的输入法是可见的，那么直接返回之前的target窗口，无需再计算新的target窗口，等待需要显示输入法或者重新relayout时再计算输入法target窗口

  • 在RecentsAnimationController#hideCurrentInputMethod方法内添加逻辑，首先先调用updateImeParent方法保证ImeParent始终正确，然后就是判断当前输入法有没有attach到app上，如果输入法当前已经attach到app上，那么在进入最近任务界面时就不执行其隐藏动画

在TaskSnapShotController#createTaskSnapshot方法,在添加excludeLayer时添加判断条件，原先只要输入法窗口不为null，就把输入法的surface添加到excludelayers内，现在是同时当输入法没有attach到app上时，才将其添加到excludelayers内。换句话说，如果当前输入法attach到app上时，创建snapShot的时候是可以将其显示出来的。

2.2 第二笔change主要修改如下：

commit 73aab1dc1fce55a544f5ecffe1c4cb7939d77be9
Author: Ming-Shin Lu <lumark@google.com>
Date:   Mon Sep 14 12:35:41 2020 +0800

   Better IME transition while switching app with recents (2/N)
    
    In TaskSnapshotController#handleClosingApps will capture task snapshot
    while applying closing animation or during screen turning-off.
    
    this will overwrite the captured task from RecentsAnimationController,
    and makes while quick switching tasks, sometimes will not see the IME
    surface on the task snapshot.
    
    Add a check in places which triggers task snapshot to ignore addidnal
    snapshot request while RecentsAnimation is active.
    
    Also refined Task#isTaskAnimating as isAnimatingByRecents to simplify
    the logic and add the test for its behavior.
    
    Bug: 166736352
    Test: manual by:
          1) Tapping EditText in app and make keyboard shown
          2) Quick switch back and force between apps by swiping navbar
          3) Verify if the task snapshot with IME persists shown during
             switching apps
    Test: atest RecentsAnimationControllerTest#testIsAnimatingByRecents

TaskSnapShotController#getClosingTasks内添加逻辑判断，如果某task是正在执行最近任务动画，那么在mSkipClosingAppSnapshotTasks内添加该task。因为RecentsAnimation已经在处理task snapshot，这里就没必要处理由RecentsAnimation控制的task了。

2.3 第三笔change主要修改如下：

概括：通过对InsetsSourceControl设置skipAnimation值 ，（由于next task已经存在Ime surface）保证Insetscontroller不会重新应用输入法显示动画

commit d579c94480fa98042dd38721de0e3a18fec67e9e
Author: Ming-Shin Lu <lumark@google.com>
Date:   Mon Oct 12 22:10:44 2020 +0800

    Better IME transition while switching app with recents (3/N)

    - Add hasImeWindowSurface in TaskSnapShot
    - Add InsetsSourceControl#getAndClearSkipAnimationOnce for skiping IME showing animation once when starting window with IME surface    

通过对InsetsSourceControl设置skipAnimation值 ，（由于next task已经存在Ime surface）保证Insetscontroller不会重新应用输入法显示动画

    Test: manual as below steps
        0) Enabling developer options -> Quick settings developer tiles ->
           Window animation scale to slow down transition animation.
        1) Launch an app with focusing an editor to show soft-input
        2) Swipe out app task to back to launcher
        3) Using quick switch or taping shortcut to bring back the app task
        4) Verify that should be no IME showing animation happens during task transition.
    Change-Id: I83ffc03119e01da71ad12f8ad8043cf0730dfd50
  • TaskSnapShot内添加hasImeSurface值来判断当前snapShot是否存在ime的surface
  • InsetsSourceControl内添加mSkipAnimationOnce值以及相关逻辑，InsetsController#applyAnimation

提供布尔值判断当前动画是否需要跳过（比如当前app内显示了输入法，按任务健进入最近任务时，此时输入法的隐藏动画就需要skip）

  • ImeInsetsSourceProvider#getControl方法，在这里获取InsetsSourceControl时，设置该control的mSkipAnimationOnce值，（主要依据条件就是target窗口的snapshot内是否存在ImeSurface）

  • InsetsController#applyAnimation，在该方法内如果当前Insets为输入法时，获取其InsetsSourceControl的mSkipAnimationOnce值，并在构造InternalAnimationControlListener时将skipAnim值传递到mDisable变量，通过这个值来控制动画是否enable

2.4 （重点）第四笔change主要修改如下：

概括： 从closing app transit到新的task时，通过showIme的snapshot始终保持输入法的可见性，而在动画结束的时候remove掉输入法的surface

commit 4964839ec3b3648ef64782a7005f6ec18cd85548
Author: Ming-Shin Lu <lumark@google.com>
Date:   Fri Oct 16 01:23:55 2020 +0800

    Better IME transition while switching app with recents (4/N)
    
    This CL introduces TaskSnapshotController#snapshotImeFromAttachedTask to
    attach IME screenshot when performing closing transition.
    
    This can improve app transition without jank or flickering while the IME
    insets control transits from closing task to the next task, we keep the
    IME surface visibility by placing the IME screenshot with calling
    DC#showImeScreenshot to be a part of task while animating
    app transition, and then remove it with DC#removeImeScreenshotIfNeeded
    when the transition animation finished or no longer used gracefully. 
   
    Test: manual as below steps:
       1) Launch an app with focusing an editor (e.g. Dialer)
       2) Swipe down status bar and tap Settings icon.
       3) Verify that when doing task transition animation, app activity
          with IME keeps visible.
    Test: manual as below steps:
       1) With 2-button or 3-button gesture, launch an app with focusing an
          editor to show soft-keyboard.
       2) Pressing home key
       3) Verify the IME screenshot keeps visible and animates with closing
          transition smoothly.
  • DisplayContent#setImeLayingTarget方法内添加调用attachAndShowImeScreenshotOnTarget的逻辑，在设置新的或者null ImeLayingTarget的时候，这时候为了保证优雅过渡，需要保持原先的ime surface，因此在这里添加调用show imeSurface的方法 （这里是创建ime surface的其中一种调用方式）

  • DisplayContent#attachAndShowImeScreenshotOnTarget，这里在原先的target窗口内创建ime surface
void attachAndShowImeScreenshotOnTarget() {
    // No need to attach screenshot if the IME target not exists or screen is off.
    if (!isImeAttachedToApp() || !mWmService.mPolicy.isScreenOn()) {
    // 满足不需要生成ime surface的条件 直接return
        return;
    }

    final SurfaceControl.Transaction t = getPendingTransaction();
    // Prepare IME screenshot for the target if it allows to attach into.
    if (mInputMethodWindow != null && mInputMethodWindow.isVisible()) {
        final Task task = mImeLayeringTarget.getTask();
        // Re-new the IME screenshot when it does not exist or the size changed.
        final boolean renewImeSurface = mImeScreenshot == null
                || mImeScreenshot.getWidth() != mInputMethodWindow.getFrame().width()
                || mImeScreenshot.getHeight() != mInputMethodWindow.getFrame().height();
        if (task != null && !task.isHomeOrRecentsRootTask()) {
            SurfaceControl.ScreenshotHardwareBuffer imeBuffer = renewImeSurface
                    ? mWmService.mTaskSnapshotController.snapshotImeFromAttachedTask(task)
                    : null;
            // 如果原先已存在ime surface 这里先remove掉 然后重新生成        
            if (imeBuffer != null) {
                // Remove the last IME surface when the surface needs to renew.
                removeImeSurfaceImmediately();
                mImeScreenshot = createImeSurface(imeBuffer, t);
            }
        }
    }

    final boolean isValidSnapshot = mImeScreenshot != null && mImeScreenshot.isValid();
    // Showing the IME screenshot if the target has already in app transition stage.
    // Note that if the current IME insets is not showing, no need to show IME screenshot
    // to reflect the true IME insets visibility and the app task layout as possible.
    if (isValidSnapshot && getInsetsStateController().getImeSourceProvider().isImeShowing()) {
        if (DEBUG_INPUT_METHOD) {
            Slog.d(TAG, "show IME snapshot, ime target=" + mImeLayeringTarget);
        }
        t.show(mImeScreenshot);
    } else if (!isValidSnapshot) {
        removeImeSurfaceImmediately();
    }
}
  • DisplayContent#createImeSurface 该方法主要就是创建ime的SurfaceControl
SurfaceControl createImeSurface(SurfaceControl.ScreenshotHardwareBuffer imeBuffer,
        Transaction t) {
    final HardwareBuffer buffer = imeBuffer.getHardwareBuffer();
    if (DEBUG_INPUT_METHOD) Slog.d(TAG, "create IME snapshot for "
            + mImeLayeringTarget + ", buff width=" + buffer.getWidth()
            + ", height=" + buffer.getHeight());
    final ActivityRecord activity = mImeLayeringTarget.mActivityRecord;
    final SurfaceControl imeSurface = mWmService.mSurfaceControlFactory.apply(null)
            .setName("IME-snapshot-surface")
            .setBufferSize(buffer.getWidth(), buffer.getHeight())
            .setFormat(buffer.getFormat())
            .setParent(activity.getSurfaceControl())
            .setCallsite("DisplayContent.attachAndShowImeScreenshotOnTarget")
            .build();
    // Make IME snapshot as trusted overlay
    InputMonitor.setTrustedOverlayInputInfo(imeSurface, t, getDisplayId(),
            "IME-snapshot-surface");
    Surface surface = mWmService.mSurfaceFactory.get();
    surface.copyFrom(imeSurface);
    surface.attachAndQueueBufferWithColorSpace(buffer, null);
    surface.release();
    t.setRelativeLayer(imeSurface, activity.getSurfaceControl(), 1);
    t.setPosition(imeSurface, mInputMethodWindow.getDisplayFrame().left,
            mInputMethodWindow.getDisplayFrame().top);
    return imeSurface;
}

DisplayContent#showImeScreentShot、 removeImeScreenshotIfPossible、removeImeSurfaceImmediately 、onWindowAnimationFinished

这些都是 显示和移除Ime surface的相关方法

showImeScreentShot是在app transition是或者swipe到最近任务时调用（WindowContainer内应用动画时调用该方法）

removeIme是在动画结束是或者ime surface淘汰时如旋转屏引起size变换

  • WindowContainer#applyAnimationUnchecked 在显示跳转动画时，这里调用显示 ime的surface

  • WindowContainer#doAnimationFinished 在动画结束时 回调dc的onWindowAnimationFinished方法（该方法会remove掉 ime的surface）

2.5 第五笔change主要修改如下：

概括： 在updateImeControltarget的时候调用updateImeParent，保证此时输入法的可见性已经确定了，这样可以防止闪烁问题，然后在执行进入最近任务动画过程中，再次check ime parent以及ime是否正确

commit d4d90ac8a88e601a4418c44fe45563070581c19b
Author: Ming-Shin Lu <lumark@google.com>
Date:   Wed Nov 11 13:17:18 2020 +0800

    Better IME transition while switching app with recents (5/N)   在updateImeControltarget的时候调用updateImeParent，保证此时输入法的可见性已经确定了，这样可以防止闪烁问题、更改InputMonitor防止错按、在旋转动画时保证imeParent的正确性
     
    With CL[1], the IME surface will have snapshot when transtioning to the
    next task.
     
    We can now remove the previous hacky logics like dedicates to keep the
    previous IME and make the true IME target while task transitioning.
    And, move the call of updateImeParent() from
    DC#setInputMethodTarget to DC#updateImeControlTarget,
     
    to ensure that the reparenting of IME insets source control can be done
    when the IME insets visiblity settled down after the IME insets control
    changed, to prevent unnecessary flickering during the time period between
    reparenting IME parent and start IME insets animation.
    
    Also, modify UpdateInputForAllWindowsConsumer#accept to let
    mRecentsAnimationInputConsumer can be above IME target activity which to
    prevent mis-touch or keystoke may left while quickly taping
    soft-keyboard during swping up to recents.
  • DisplayContent#updateImeControlTarget 在该方法内更新ime parent，这个是最佳的时机，

（在setImeLayeringTargetInner方法内，原先是在updateImeControlTarget方法之前调用updateImeParent）

  • RecentsAnimationController#setAnimationTargetsBehindSystemBars，在执行进入最近任务界面动画时，首先更新ime parent保证正确的ime parent，然后再次确认ime是否attach到之前的task上，如果没有，执行隐藏输入法操作

2.5 第六笔change主要修改如下：

概括： 通过设置FrozenInsetsState 来保证正在执行closing的window不会收到非期望的insets变化(比如新window的insets的分发)（也就是InsetsStateController会根据是否存在freeze的InsetsState来判断是否分发insetsChanged），同时添加freeze以及unFreeze Insetsstate的操作

commit 0f536fa99489c5cfd2d266e94798b19540e2cd12
Author: Ming-Shin Lu <lumark@google.com>
Date:   Thu Jan 14 21:03:15 2021 +0800

    Better IME transition while switching app with recents (6/N) 
    Add WindowState#{freezeInsetsState, clearFrozenInsetsState} to freeze
    the insets state of the window to keep the insets state when the window
    is in app exiting transition, to ensure the exiting window won't
    receive unexpected insets changes from the next window.
    
    And, in order to easy maintain the logic of dispatching all windows's
    insets changed event in InsetsStateController, instead of adding
    complicated rule in InsetsStateController to judge if the window can
    disatch insets change, we can replace with
    WindowState#isReadyToDispatchInsetsState() to wrap the dispatch
    judgement.
    
    Bug: 166736352
    Test: atest CtsWindowManagerDeviceTestCases WmTests
    Test: atest WindowStateTests#testSetFreezeInsetsState
    Test: atest WindowContainerTests#testFreezeInsetsStateWhenAppTransition
    
    Change-Id: I56418733c1abbd73e88a8918a5d55ecc15344c5e
  • WindowState内添加mFrozenInsetsState的定义以及相关方法，意为冻结insets的state，在一些情况下没有必要与client端保证同步，例如在退出动画时

isReadyToDispatchInsetsState方法返回代表是否准备分发insetsState，如果不存在freeze的InsetsState，即可分发

  • DisplayContent#onWindowAnimationFinished，在动画结束时清除所有的freeze的insetsState

  • InsesStateController 修改mDispatchInsetsChanged 这个consumer，保证分发时不存在freeze的InsetsState

总结

关于S上输入法的新特性大概就总结到这里了，S上的输入法在动画切换以及app transition的时候更好的提高了用户的体验度。 如果大家有何疑问或者建议，欢迎评论留言～ 与大家一起共同进步！～

avatar
文章
阅读
粉丝