掌握Android图像显示原理下(一)

·  阅读 1719

前言

前面两篇文章介绍了图像生产者图像消费者,终于到了最后一篇——图像缓冲区,这三者一起构成了一个完整的Android图像显示系统。Android中的图像生产者OpenGL,Skia,Vulkan将绘制的数据存放在图像缓冲区中,Android中的图像消费SurfaceFlinger从图像缓冲区将数据取出,进行加工及合成。那么图像缓冲区是什么呢?它是如何创建出来的呢?又要如何使用它呢?它的存储原理是什么呢?读完这篇文章,你就能回答这些问题了。

图像缓冲区

在讲解图像的生产者时,多次提到了Surface,我们知道Surface可以存储用来绘制的图形数据。在硬件加速中,需要调用 ThreadedRenderer.initialize(mSurface) 函数将Surface缓冲区传递到OpenGL或者Vulkan的渲染管线;在软件绘制中调用 drawSoftware(surface, mAttachInfo, xOffset, yOffset, scalingRequired, dirty) 函数将Surface缓冲区传递到native层的SkiaCanvas。当OpenGL,Vulkan或Skia拥有Surface后,就可以获取到图形缓冲区,将需要绘制的内容存储在图形缓冲区内了。

同时,在讲解图像的消费者时,也多次提到了Layer,我们知道通过Layer可以取出缓冲区的图形数据。在SurfaceFlinger需要消费图像数据,进行图层混合时,需要遍历Layer,取出Layer中缓冲区的图形数据后,再加过或合成处理。

我们也知道一个Layer唯一对应了一个Surface,他们的关系如下。 在这里插入图片描述

所以,Surface和Layer都属于图形缓冲区的组成部分。那么Android图形缓冲区由哪些部分组成呢?Surface和Layer又是什么呢?他们是怎么关联起来的呢?他们又分别是如何创建、获取以及使用缓冲区的呢?这篇文章主要的内容都是围绕这四个问题来讲解,在深入讲解这四个问题之前,这里我们先高屋建瓴的了解这个四个问题的答案,这样在接下面的篇幅中,才不会迷失在冗长的代码中。

在这里插入图片描述

1.Android图形缓冲区由哪些部分组成呢?

  • Android的图形缓冲区由Surface,BufferQueue,Layer,GraphicBuffer四部分组成。BufferQueue中的slots数组最多能存储64个GraphicBuffer,而GraphicBuffer就是真正被分配内存,并能存储图形数据的缓冲区

2.Surface和Layer又是什么呢?

  • Surface是提供给图形生产者控制缓冲区的类,Surface持有GraphicBufferProducer,简称gbp,gbp专门用来创建或获取可用的GraphicBuffer以及提交绘制后的GraphicBuffer
  • Layer是提供给图像消费者获取缓冲区的类,Layer持有GraphicBufferConsumer,简称gbc,通过gbc用来获取和释放GraphicBuffer

3.他们是怎么关联起来的呢?

  • Surface和Layer通过BufferQueue关联起来,Surface持有BufferQueue中的gbp,Layer持有BufferQueue中的gbc,gbp和gbc的GraphicBuffer都存储在GraphicBufferCore的slots数组中。

4.如何创建、获取以及使用缓冲区呢?

  • Surface通过调用gbp的dequeue函数获取GraphicBuffer,调用queue函数提交使用完毕的GraphicBuffer。
  • Layer通过调用gbc的acquire函数获取有数据的GraphicBuffer,调用release释放GraphicBuffer 在这里插入图片描述

总结一下上面提到的知识:当我们想要绘制图像时,需要创建Surface和Layer,图像生产者如Skia,OpenGL通过Surface调用dequeue函数获取一块缓冲区GraphicBuffer,有了这块缓冲区,就可以在缓冲区上绘制图像了,当绘制完毕后,通过Surface调用queue函数,将GraphicBuffer归还到BufferQueue。之后,Vsync通知图像消费者SurfaceFlinger调用Layer的acquire函数,获取绘制好内容的GraphicBuffer进行合成与处理,处理完毕后通过release函数释放这块缓冲区。

了解了大概的背景和流程,我们接着开始对图像缓冲区的深入学习。

缓冲区的创建

我们已经知道缓冲区由Surface,Layer,BufferQueue和GraphicBuffer组成。这一节主要讲这四个部分是如何创建的,先从Surface和Layer的创建开始。

Surface和Layer的创建

为了更容易的理解如何创建Surface和Layer,这里先从开机动画这个案例讲起。

开机动画创建Surface和Layer

在前面图像生产者中讲如何通过OpenGL ES播放开启动画时,已经提到了创建Surface的流程,它的流程很简单,这里再回顾一下通过Android开机动画的启动流程。开机动画的对象为BootAnimation,它的构造函数如下。

/frameworks/base/cmds/bootanimation/BootAnimation.cpp

BootAnimation::BootAnimation() : Thread(false), mClockEnabled(true), mTimeIsAccurate(false),
        mTimeFormat12Hour(false), mTimeCheckThread(NULL) {
    //创建SurfaceComposerClient
    mSession = new SurfaceComposerClient();
    ……
}
复制代码

可以看到构造函数里面创建了SurfaceComposerClient。

我们接着看BootAnimation的初始化函数——readyToRun函数

/frameworks/base/cmds/bootanimation/BootAnimation.cpp

status_t BootAnimation::readyToRun() {
    mAssets.addDefaultAssets();

    sp<IBinder> dtoken(SurfaceComposerClient::getBuiltInDisplay(
            ISurfaceComposer::eDisplayIdMain));
    DisplayInfo dinfo;
    //获取屏幕信息
    status_t status = SurfaceComposerClient::getDisplayInfo(dtoken, &dinfo);
    if (status)
        return -1;

    // 通知SurfaceFlinger创建Surface,创建成功会返回一个SurfaceControl代理
    sp<SurfaceControl> control = session()->createSurface(String8("BootAnimation"),
            dinfo.w, dinfo.h, PIXEL_FORMAT_RGB_565);

    SurfaceComposerClient::openGlobalTransaction();
    //设置这个layer在SurfaceFlinger中的层级顺序
    control->setLayer(0x40000000);

    //获取surface
    sp<Surface> s = control->getSurface();

    // 以下是EGL的初始化流程
    const EGLint attribs[] = {
            EGL_RED_SIZE,   8,
            EGL_GREEN_SIZE, 8,
            EGL_BLUE_SIZE,  8,
            EGL_DEPTH_SIZE, 0,
            EGL_NONE
    };
    EGLint w, h;
    EGLint numConfigs;
    EGLConfig config;
    EGLSurface surface;
    EGLContext context;

    //步骤1:获取Display
    EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
    //步骤2:初始化EGL
    eglInitialize(display, 0, 0);
    //步骤3:选择参数
    eglChooseConfig(display, attribs, &config, 1, &numConfigs);
    //步骤4:传入SurfaceFlinger生成的surface,并以此构造EGLSurface
    surface = eglCreateWindowSurface(display, config, s.get(), NULL);
    //步骤5:构造egl上下文
    context = eglCreateContext(display, config, NULL, NULL);
    //步骤6:绑定EGL上下文
    if (eglMakeCurrent(display, surface, surface, context) == EGL_FALSE)
        return NO_INIT;
    ……
}
复制代码

通过BootAnimation的构造函数和readyToRun函数可以看到,创建Surface的步骤如下:

  1. 创建SurfaceComponentClient

  2. 通过SurfaceComponentClient的createSurface函数创建SurfaceControl,SurfaceControl是Surface的控制类

  3. 有了SurfaceControl之后,我们就可以并能通过getSurface获取到Surface

只需要三步,Surface的创建就完成了,非常的简单。但是不我们不能止步于如何创建Surface,我们还需要了解SurfaceComponentClient什么?createSurface经历了哪些流程?getSurface又经历了哪些流程?

SurfaceComponentClient

我们先看看SurfaceComponentClient对象,它的构造函数如下。

/frameworks/native/libs/gui/SurfaceComposerClient.cpp

ComposerService::ComposerService()
: Singleton<ComposerService>() {
    connectLocked();
}

void ComposerService::connectLocked() {
    const String16 name("SurfaceFlinger");
    //获取SurfaceFlinger
    while (getService(name, &mComposerService) != NO_ERROR) {
        usleep(250000);
    }
    
    //注册binder death的通知
    ……
        
}
复制代码

从SurfaceComposerClient可以看到,它在connectLocked函数中获取了SurfaceFlinger的客户端Binder代理mComposerService。

接着在看看它初始化函数onFirstRef

/frameworks/native/libs/gui/SurfaceComposerClient.cpp

void SurfaceComposerClient::onFirstRef() {
    sp<ISurfaceComposer> sm(ComposerService::getComposerService());
    if (sm != 0) {
        auto rootProducer = mParent.promote();
        sp<ISurfaceComposerClient> conn;
        conn = (rootProducer != nullptr) ? sm->createScopedConnection(rootProducer) :
                sm->createConnection();
        if (conn != 0) {
            mClient = conn;
            mStatus = NO_ERROR;
        }
    }
}
复制代码

onFirstRef函数中通过mComposerService调用createConnection方法。mComposerService就是SurfaceFlinger的binder代理,所以这里会最终调用SurfaceFlinger的createConnection函数。

接着看SurfaceFlinger的createConnection函数

/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

sp<ISurfaceComposerClient> SurfaceFlinger::createConnection() {
    return initClient(new Client(this));
}

static sp<ISurfaceComposerClient> initClient(const sp<Client>& client) {
    status_t err = client->initCheck();
    if (err == NO_ERROR) {
        return client;
    }
    return nullptr;
}
复制代码

可以看到createConnection方法创建了Client,这个Client封装了对Layer和Surface的操作,我们看一下Client的头文件

/frameworks/native/services/surfaceflinger/Client.h


class Client : public BnSurfaceComposerClient
{
public:
    explicit Client(const sp<SurfaceFlinger>& flinger);
    Client(const sp<SurfaceFlinger>& flinger, const sp<Layer>& parentLayer);
    ~Client();

    status_t initCheck() const;

    // protected by SurfaceFlinger::mStateLock
    void attachLayer(const sp<IBinder>& handle, const sp<Layer>& layer);

    void detachLayer(const Layer* layer);

    sp<Layer> getLayerUser(const sp<IBinder>& handle) const;

    void setParentLayer(const sp<Layer>& parentLayer);

private:
    // ISurfaceComposerClient interface
    virtual status_t createSurface(
            const String8& name,
            uint32_t w, uint32_t h,PixelFormat format, uint32_t flags,
            const sp<IBinder>& parent, uint32_t windowType, uint32_t ownerUid,
            sp<IBinder>* handle,
            sp<IGraphicBufferProducer>* gbp);

    virtual status_t destroySurface(const sp<IBinder>& handle);

    virtual status_t clearLayerFrameStats(const sp<IBinder>& handle) const;

    virtual status_t getLayerFrameStats(const sp<IBinder>& handle, FrameStats* outStats) const;

    virtual status_t onTransact(
        uint32_t code, const Parcel& data, Parcel* reply, uint32_t flags);

    sp<Layer> getParentLayer(bool* outParentDied = nullptr) const;

    // constant
    sp<SurfaceFlinger> mFlinger;

    // protected by mLock
    DefaultKeyedVector< wp<IBinder>, wp<Layer> > mLayers;
    wp<Layer> mParentLayer;

    // thread-safe
    mutable Mutex mLock;
};
}
复制代码

可以看到,Client继承自BnSurfaceComposerClient,并提供了创建和销毁Layer和Surface的操作函数。

createSurface

创建好了SurfaceComponentClinet,并有了一个在SurfaceFlinger对应的Clinet。我们可以接着看第二步:createSurface函数。

/frameworks/native/libs/gui/SurfaceComposerClient.cpp

sp<SurfaceControl> SurfaceComposerClient::createSurface(
        const String8& name,
        uint32_t w,
        uint32_t h,
        PixelFormat format,
        uint32_t flags,
        SurfaceControl* parent,
        uint32_t windowType,
        uint32_t ownerUid)
{
    sp<SurfaceControl> sur;
    if (mStatus == NO_ERROR) {
        sp<IBinder> handle;
        sp<IBinder> parentHandle;
        sp<IGraphicBufferProducer> gbp;

        if (parent != nullptr) {
            parentHandle = parent->getHandle();
        }
        status_t err = mClient->createSurface(name, w, h, format, flags, parentHandle,
                windowType, ownerUid, &handle, &gbp);
        if (err == NO_ERROR) {
            sur = new SurfaceControl(this, handle, gbp);
        }
    }
    return sur;
}
复制代码

createSurface函数主要做了两件事情

  1. 调用Client的createSurface方法
  2. 创建SurfaceControl

先看第一件事情:调用Client的createSurface函数

/frameworks/native/services/surfaceflinger/Client.cpp

status_t Client::createSurface(
        const String8& name,
        uint32_t w, uint32_t h, PixelFormat format, uint32_t flags,
        const sp<IBinder>& parentHandle, uint32_t windowType, uint32_t ownerUid,
        sp<IBinder>* handle,
        sp<IGraphicBufferProducer>* gbp)
{
    ……
    
    class MessageCreateLayer : public MessageBase {
        SurfaceFlinger* flinger;
        Client* client;
        sp<IBinder>* handle;
        sp<IGraphicBufferProducer>* gbp;
        status_t result;
        const String8& name;
        uint32_t w, h;
        PixelFormat format;
        uint32_t flags;
        sp<Layer>* parent;
        uint32_t windowType;
        uint32_t ownerUid;
    public:
        MessageCreateLayer(SurfaceFlinger* flinger,
                const String8& name, Client* client,
                uint32_t w, uint32_t h, PixelFormat format, uint32_t flags,
                sp<IBinder>* handle, uint32_t windowType, uint32_t ownerUid,
                sp<IGraphicBufferProducer>* gbp,
                sp<Layer>* parent)
            : flinger(flinger), client(client),
              handle(handle), gbp(gbp), result(NO_ERROR),
              name(name), w(w), h(h), format(format), flags(flags),
              parent(parent), windowType(windowType), ownerUid(ownerUid) {
        }
        status_t getResult() const { return result; }
        virtual bool handler() {
            result = flinger->createLayer(name, client, w, h, format, flags,
                    windowType, ownerUid, handle, gbp, parent);
            return true;
        }
    };

    sp<MessageBase> msg = new MessageCreateLayer(mFlinger.get(),
            name, this, w, h, format, flags, handle,
            windowType, ownerUid, gbp, &parent);
    mFlinger->postMessageSync(msg);
    return static_cast<MessageCreateLayer*>( msg.get() )->getResult();
}
复制代码

可以看到,createSurface实际是通过SurfaceFlinger的消息队列调用createLayer函数,接着看SurfaceFlinger中createLayer函数的实现。

/frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

status_t SurfaceFlinger::createLayer(
        const String8& name,
        const sp<Client>& client,
        uint32_t w, uint32_t h, PixelFormat format, uint32_t flags,
        uint32_t windowType, uint32_t ownerUid, sp<IBinder>* handle,
        sp<IGraphicBufferProducer>* gbp, sp<Layer>* parent)
{
    

    status_t result = NO_ERROR;
    sp<Layer> layer;
    String8 uniqueName = getUniqueLayerName(name);

    switch (flags & ISurfaceComposerClient::eFXSurfaceMask) {
        case ISurfaceComposerClient::eFXSurfaceNormal:
            //创建普通的Layer
            result = createNormalLayer(client,
                    uniqueName, w, h, flags, format,
                    handle, gbp, &layer);
            break;
        case ISurfaceComposerClient::eFXSurfaceDim:
            //创建有遮罩效果的Layer
            result = createDimLayer(client,
                    uniqueName, w, h, flags,
                    handle, gbp, &layer);
            break;
        default:
            result = BAD_VALUE;
            break;
    }

    if (result != NO_ERROR) {
        return result;
    }

    layer->setInfo(windowType, ownerUid);

    result = addClientLayer(client, *handle, *gbp, layer, *parent);
    if (result != NO_ERROR) {
        return result;
    }
    mInterceptor.saveSurfaceCreation(layer);

    setTransactionFlags(eTransactionNeeded);
    return result;
}

status_t SurfaceFlinger::createNormalLayer(const sp<Client>& client,
        const String8& name, uint32_t w, uint32_t h, uint32_t flags, PixelFormat& format,
        sp<IBinder>* handle, sp<IGraphicBufferProducer>* gbp, sp<Layer>* outLayer)
{
    // initialize the surfaces
    switch (format) {
    case PIXEL_FORMAT_TRANSPARENT:
    case PIXEL_FORMAT_TRANSLUCENT:
        format = PIXEL_FORMAT_RGBA_8888;
        break;
    case PIXEL_FORMAT_OPAQUE:
        format = PIXEL_FORMAT_RGBX_8888;
        break;
    }

    *outLayer = new Layer(this, client, name, w, h, flags);
    status_t err = (*outLayer)->setBuffers(w, h, format, flags);
    if (err == NO_ERROR) {
        *handle = (*outLayer)->getHandle();
        *gbp = (*outLayer)->getProducer();
    }

    ALOGE_IF(err, "createNormalLayer() failed (%s)", strerror(-err));
    return err;
}
复制代码

到这里我们可以发现,原来createSurface函数实际上并不是创建Surface,而是创建了Layer。接着看Layer的初始化函数onFirstRef

/frameworks/native/services/surfaceflinger/Layer.cpp

void Layer::onFirstRef() {    
    sp<IGraphicBufferProducer> producer;
    sp<IGraphicBufferConsumer> consumer;
    //创建BufferQueue
    BufferQueue::createBufferQueue(&producer, &consumer, true);
    mProducer = new MonitoredProducer(producer, mFlinger, this);
    mSurfaceFlingerConsumer = new SurfaceFlingerConsumer(consumer, mTextureName, this);
    mSurfaceFlingerConsumer->setConsumerUsageBits(getEffectiveUsage(0));
    mSurfaceFlingerConsumer->setContentsChangedListener(this);
    mSurfaceFlingerConsumer->setName(mName);

    if (mFlinger->isLayerTripleBufferingDisabled()) {
        mProducer->setMaxDequeuedBufferCount(2);
    }

    const sp<const DisplayDevice> hw(mFlinger->getDefaultDisplayDevice());
    updateTransformHint(hw);
}
复制代码

可以看到,Layer的初始化函数里通过BufferQueue::createBufferQueue创建了BufferQueue,而BufferQueue又会创建GraphicBufferProducer和GraphicBufferConsumer,关于BufferQueue后面在详讲。

在看第二件事情:创建SurfaceControl

/frameworks/native/libs/gui/SurfaceControl.cpp

SurfaceControl::SurfaceControl(
        const sp<SurfaceComposerClient>& client,
        const sp<IBinder>& handle,
        const sp<IGraphicBufferProducer>& gbp)
    : mClient(client), mHandle(handle), mGraphicBufferProducer(gbp)
{
}
复制代码

SurfaceControl的构造函数没有任何操作,仅仅只是传入Client,以及在BufferQueue中创建的GraphicBufferProducer。

getSurface

Layer和SurfaceControl都创建好了,就差最后一步了:调用getSurface函数获取Surface。

/frameworks/native/libs/gui/SurfaceControl.cpp

sp<Surface> SurfaceControl::getSurface() const
{
    Mutex::Autolock _l(mLock);
    if (mSurfaceData == 0) {
        return generateSurfaceLocked();
    }
    return mSurfaceData;
}

sp<Surface> SurfaceControl::generateSurfaceLocked() const
{
    
    mSurfaceData = new Surface(mGraphicBufferProducer, false);

    return mSurfaceData;
}
复制代码

可以看到,这里会创建一个Surface。为了对Surface有一个了解,我们看一下Surface的头文件

/frameworks/native/include/gui/Surface.h


class Surface
    : public ANativeObjectBase<ANativeWindow, Surface, RefBase>
{
public:
    explicit Surface(const sp<IGraphicBufferProducer>& bufferProducer,
            bool controlledByApp = false);
   ……
protected:
    virtual int dequeueBuffer(ANativeWindowBuffer** buffer, int* fenceFd);
    virtual int queueBuffer(ANativeWindowBuffer* buffer, int fenceFd);
   ……
protected:
    uint32_t mReqWidth;
    uint32_t mReqHeight;
    PixelFormat mReqFormat;
    uint32_t mDefaultWidth;
    uint32_t mDefaultHeight;
    ……
}
复制代码

从头文件可以看到,Surface继承自ANativeWindow,并且封装了许多对Buffer进行操作的方法,以及维护了Surface的大小即属性等参数数据。到这里,Surface也创建完成了,Surface的构造函数的入参就是GraphicBufferProducer。有了Surface,就可以交给OpenGL进行开机动画的绘制了,绘制的过程在图像生产者中已经讲了,这里就不再说了。

通过开机动画这个案例,我们可以发现,通过简单三步,就能创建好Surface和Layer了。下面接着来看看位于Java层的Activity是如何创建Surface,由于Activity的的界面显示流程涉及到很多JNI的通信以及对界面的测量布局等流程,所以它创建Surface和Layer并没有开机动画这么直观易懂。

Activity创建Surface和Layer

Activity创建Surface和Layer的流程会和界面绘制、AMS对界面Z轴排序等流程耦合在一块,所以在讲Surface和Layer的创建过程中,会将这些流程一起讲,Activity的界面我都以Window,即窗口来表示。

先看看Java层的Surface,它位于ViewRootImpl中,并且声明成员变量就创建好了。

/frameworks/base/core/java/android/view/ViewRootImpl.java

public final class ViewRootImpl implements ViewParent,View.AttachInfo.Callbacks, ThreadedRenderer.DrawCallbacks {
    //……
    final Surface mSurface = new Surface();
    //……  
}
复制代码

这里的Surface只是Java层的Surface,它需要绑定Native的Surface,我们一般在构造函数中,可以看到jni方法创建native层对应的对象,比如图像生产者中提到的DisplayListCanvas就会在构造函数中创建native层的RecordingCanvas,所以我们先看看Surface的构造函数

/frameworks/base/core/java/android/view/Surface.java

public Surface() {
}
复制代码

这里发现他是一个空实现,找不到创建创建native层Surface的JNI方法调用。那么native层的Surface是如何创建的呢?又是如何和java层的Surface绑定起来的呢?带着这个疑问,接着往下看。

这需要从Activity界面的绘制流程开始讲起,当我们在Activity的OnCreate函数中通过setContentView设置界面后,最终会执行到ViewRootImpl的setView方法,我们从这儿开始看起。

/frameworks/base/core/java/android/view/ViewRootImpl.java

public void setView(View view, WindowManager.LayoutParams attrs, View panelParentView) {
    synchronized (this) {
        if (mView == null) {
            mView = view;

            //……

            // 开启硬件加速
            if (mSurfaceHolder == null) {
                enableHardwareAcceleration(attrs);
            }

            
            //1,测量,布局和绘制流程
            requestLayout();
            if ((mWindowAttributes.inputFeatures
                 & WindowManager.LayoutParams.INPUT_FEATURE_NO_INPUT_CHANNEL) == 0) {
                //触摸事件回调信道
                mInputChannel = new InputChannel();
            }
            mForceDecorViewVisibility = (mWindowAttributes.privateFlags
                                         & PRIVATE_FLAG_FORCE_DECOR_VIEW_VISIBILITY) != 0;
         
                mOrigWindowType = mWindowAttributes.type;
                mAttachInfo.mRecomputeGlobalAttributes = true;
                collectViewAttributes();
            	//2,调用wms的addToDisplay函数
                res = mWindowSession.addToDisplay(mWindow, mSeq, mWindowAttributes,
                                                  getHostVisibility(), mDisplay.getDisplayId(),
                                                  mAttachInfo.mContentInsets, mAttachInfo.mStableInsets,
                                                  mAttachInfo.mOutsets, mInputChannel);
            

            //添加窗口错误的处理
            ……
            
            //触摸事件监听初始化处理
            ……
        }
    }
}
复制代码

setView函数中关键流程主要是这两件事情:

1,执行requestLayout函数,这个流程里会创建Surface和Layer

2,调用wms的addTodisplay(),这个流程里会创建SurfaceComponentClient

先看看requestLayout函数

/frameworks/base/core/java/android/view/ViewRootImpl.java

public void requestLayout() {
    if (!mHandlingLayoutInLayoutRequest) {
        checkThread();
        mLayoutRequested = true;
        scheduleTraversals();
    }
}

void scheduleTraversals() {
    if (!mTraversalScheduled) {
        mTraversalScheduled = true;
        mTraversalBarrier = mHandler.getLooper().getQueue().postSyncBarrier();
        mChoreographer.postCallback(
            Choreographer.CALLBACK_TRAVERSAL, mTraversalRunnable, null);
        if (!mUnbufferedInputDispatch) {
            scheduleConsumeBatchedInput();
        }
        notifyRendererOfFramePending();
        pokeDrawLockIfNeeded();
    }
}
复制代码

requestLayout函数通过mChoreographer添加了回调对象mTraversalRunnable,mChoreographer会在收到VSync执行mTraversalRunnable,这里就不介绍mChoreographer了。mTraversalRunnable里做的事情主要是测量,布局和绘制流程,但是现在就进行这些工作还太早了,因为我们的SurfaceComponentClient还没创建好,所以mTraversalRunnable的回调会在addTodisplay后执行。

SurfaceComponentClinet

我们先看mWindowSession.addToDisplay()的流程,他会在mTraversalRunnable回调之前执行。在这这里我们先了解一下mWindowSession,凡是带Session的类都是用来通信的,WindowSession就是Java层的窗口和WMS通信的会话对象,它在ViewRootImp的l构造函数中通过WindowManagerGlobal.getWindowSession()获取。

/frameworks/base/core/java/android/view/ViewRootImpl.java

public ViewRootImpl(Context context, Display display) {
    ……
	mWindowSession = WindowManagerGlobal.getWindowSession();
    ……
}

public final class WindowManagerGlobal {
    ……
        
    public static IWindowSession getWindowSession() {
        synchronized (WindowManagerGlobal.class) {
            if (sWindowSession == null) {
                try {
                    InputMethodManager imm = InputMethodManager.getInstance();
                    IWindowManager windowManager = getWindowManagerService();
                    sWindowSession = windowManager.openSession(
                        new IWindowSessionCallback.Stub() {
                            @Override
                            public void onAnimatorScaleChanged(float scale) {
                                ValueAnimator.setDurationScale(scale);
                            }
                        },
                        imm.getClient(), imm.getInputContext());
                } catch (RemoteException e) {
                    throw e.rethrowFromSystemServer();
                }
            }
            return sWindowSession;
        }
    }
    
    ……
}
复制代码

可以看到getWindowSession函数中获取了WindowManagerService的Binder代理,然后调用WindowManagerService的openSession方法来创建Session。

/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java

public IWindowSession openSession(IWindowSessionCallback callback, IInputMethodClient client,
                                  IInputContext inputContext) {
    if (client == null) throw new IllegalArgumentException("null client");
    if (inputContext == null) throw new IllegalArgumentException("null inputContext");
    Session session = new Session(this, callback, client, inputContext);
    return session;
}
复制代码

为什么不通过WMS的Proxy直接进行通信呢,而要创建Session来进行通信呢?我的理解是WMS要和多个窗口通信,如果直接通过WMS对话会不方便维护对应窗口的上下文,这个Session作用是表示当前这个窗口和WMS的对话。继续回到mWindowSession.addToDisplay函数上来。

/frameworks/base/services/core/java/com/android/server/wm/Session.java


public int addToDisplay(IWindow window, int seq, WindowManager.LayoutParams attrs,
                        int viewVisibility, int displayId, Rect outContentInsets, Rect outStableInsets,
                        Rect outOutsets, InputChannel outInputChannel) {
    return mService.addWindow(this, window, seq, attrs, viewVisibility, displayId,
                              outContentInsets, outStableInsets, outOutsets, outInputChannel);
}


复制代码

addToDisplay方法只是调用了WMS的addWindow方法

/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java

public int addWindow(Session session, IWindow client, int seq,
                     WindowManager.LayoutParams attrs, int viewVisibility, int displayId,
                     Rect outContentInsets, Rect outStableInsets, Rect outOutsets,
                     InputChannel outInputChannel) {
    int[] appOp = new int[1];
    //权限检查
    int res = mPolicy.checkAddPermission(attrs, appOp);
    if (res != WindowManagerGlobal.ADD_OKAY) {
        return res;
    }

    boolean reportNewConfig = false;
    WindowState parentWindow = null;
    long origId;
    final int callingUid = Binder.getCallingUid();
    final int type = attrs.type;

    synchronized(mWindowMap) {
        if (!mDisplayReady) {
            throw new IllegalStateException("Display has not been initialialized");
        }

        //创建或获取DisplayContent
        final DisplayContent displayContent = mRoot.getDisplayContentOrCreate(displayId);
        if (displayContent == null) {
            Slog.w(TAG_WM, "Attempted to add window to a display that does not exist: "
                   + displayId + ".  Aborting.");
            return WindowManagerGlobal.ADD_INVALID_DISPLAY;
        }
        
        //窗口类型校验
        …… 

        //获取WindowToken
        AppWindowToken atoken = null;
        final boolean hasParent = parentWindow != null;
        WindowToken token = displayContent.getWindowToken(
            hasParent ? parentWindow.mAttrs.token : attrs.token);
       
        final int rootType = hasParent ? parentWindow.mAttrs.type : type;

        boolean addToastWindowRequiresToken = false;

        if (token == null) {
            
            // token为空的情况下,根据窗口类型判断是否返回错误,有些窗口,如系统窗口运行token为空,但是子窗口等窗口不允许
            ……
            
            //为允许token为空的窗口创建token
            final IBinder binder = attrs.token != null ? attrs.token : client.asBinder();
            token = new WindowToken(this, binder, type, false, displayContent,
                                    session.mCanAddInternalSystemWindow);
        } else if (rootType >= FIRST_APPLICATION_WINDOW && rootType <= LAST_APPLICATION_WINDOW) {
            // APPLICATION类型的窗口检查,对应的WindowToken的类型也为APPLICATION
            ……
        } else if (rootType == TYPE_INPUT_METHOD) {
            // INPUT_METHOD类型的窗口检查
            ……
        } else if (rootType == TYPE_VOICE_INTERACTION) {
            // VOICE类型的窗口检查
            ……
        } else if (rootType == TYPE_WALLPAPER) {
            // WALLPAPER类型的窗口检查
            ……
        } else if (rootType == TYPE_DREAM) {
            // TYPE_DREAM类型的窗口检查
            ……
        } else if (rootType == TYPE_ACCESSIBILITY_OVERLAY) {
            ……
        } else if (type == TYPE_TOAST) {
            ……
        } else if (type == TYPE_QS_DIALOG) {
            ……
        } else if (token.asAppWindowToken() != null) {
            …… 
        }

        //为这个窗口创建WindowState
        final WindowState win = new WindowState(this, session, client, token, parentWindow,
                                                appOp[0], seq, attrs, viewVisibility, session.mUid,
                                                session.mCanAddInternalSystemWindow);       
        ……
        
        //执行attach
        win.attach();
        //将window存入mWindowMap
        mWindowMap.put(client.asBinder(), win);
  

        final AppWindowToken aToken = token.asAppWindowToken();
        
		//同一个token会有多个window
        win.mToken.addWindow(win);       
         ……      
    }

    
    return res;
}
复制代码

addToDisplay比较长,但主要做的事情只有这几件

  1. 对添加窗口的Token和类型合法性校验,Token是这个窗口的标识,Activity窗口的Token会在AMS创建Acitvity的时候创建,系统窗口会在这里创建。
  2. 创建WindowState,WindowState会持有在上面创建的用于WMS和窗口通信的Session以及一些窗口状态。
  3. 执行attach函数,这个函数中会创建SurfaceComponentClient
  4. 存储WindowState

这里只关心SurfaceComponentClient的创建,所以我们只看win.attach()函数

/frameworks/base/services/core/java/com/android/server/wm/WindowState.java

void attach() {
    mSession.windowAddedLocked(mAttrs.packageName);
}
复制代码

这里调用了Session的windowAddedLocked函数,这个session就是我们在上面创建的Session,

/frameworks/base/services/core/java/com/android/server/wm/Session.java

void windowAddedLocked(String packageName) {
    mPackageName = packageName;   
    if (mSurfaceSession == null) {     
        mSurfaceSession = new SurfaceSession();      
        mService.mSessions.add(this);
        if (mLastReportedAnimatorScale != mService.getCurrentAnimatorScale()) {
            mService.dispatchNewAnimatorScaleLocked(this);
        }
    }
    mNumWindow++;
}
复制代码

可以看到这里创建了SurfaceSession,又遇到了一个以Session命名的类,它是和谁通信用的呢?带着疑问接着往下看

/frameworks/base/core/java/android/view/SurfaceSession.java

public SurfaceSession() {
    mNativeClient = nativeCreate();
}
复制代码

SurfaceSession的构造函数调用了native函数nativeCreate

/frameworks/base/core/jni/android_view_SurfaceSession.cpp

static jlong nativeCreate(JNIEnv* env, jclass clazz) {
    SurfaceComposerClient* client = new SurfaceComposerClient();
    client->incStrong((void*)nativeCreate);
    return reinterpret_cast<jlong>(client);
}
复制代码

到这里终于发现了SurfaceComposerClient的身影,在前面已经知道,只要有了SurfaceComposerClient,就能通过createSurface和getSurface创建Layer和Surface了。

我们接着探索Activity的界面显示流程中是在哪儿调用这两个函数的。

createSurface

那么createSurface的调用在哪儿呢?它其实在mTraversalRunnable回调里

/frameworks/base/core/java/android/view/ViewRootImpl.java

final class TraversalRunnable implements Runnable {
    @Override
    public void run() {
        doTraversal();
    }
}

void doTraversal() {
    if (mTraversalScheduled) {
        mTraversalScheduled = false;
        mHandler.getLooper().getQueue().removeSyncBarrier(mTraversalBarrier);

        performTraversals();

        if (mProfile) {
            Debug.stopMethodTracing();
            mProfile = false;
        }
    }
}
复制代码

mTraversalRunnable回调函数执行了performTraversals函数,这个函数主要是用来进行布局,测量和绘制的,接着看这个函数。

/frameworks/base/core/java/android/view/ViewRootImpl.java

private void performTraversals() {
    final View host = mView;
    if (host == null || !mAdded)
        return;

    // 窗口参数初始化和调整
    ……

    boolean layoutRequested = mLayoutRequested && (!mStopped || mReportNextDraw);
    if (layoutRequested) {
        ……

        //1,预测量
        windowSizeMayChange |= measureHierarchy(host, lp, res,
                desiredWindowWidth, desiredWindowHeight);
    }

    ……

    if (mApplyInsetsRequested) {
        mApplyInsetsRequested = false;
        if (mLayoutRequested) {
            // window窗口发生变化后重新测量
            windowSizeMayChange |= measureHierarchy(host, lp,
                    mView.getContext().getResources(),
                    desiredWindowWidth, desiredWindowHeight);
        }
    }

    //判断窗口是否需要重新调整大小
    ……


    final int surfaceGenerationId = mSurface.getGenerationId();

    final boolean isViewVisible = viewVisibility == View.VISIBLE;
    final boolean windowRelayoutWasForced = mForceNextWindowRelayout;
    //第一次加载,或者 window 需要重新调整大小,或者 insets 发生了改变,或者 view 可见性变化了
    if (mFirst || windowShouldResize || insetsChanged ||
            viewVisibilityChanged || params != null || mForceNextWindowRelayout) {
        mForceNextWindowRelayout = false;

        ……

        boolean hadSurface = mSurface.isValid();

        try {
            ……

            //2,调用wms,创建native Surface等工作。
            relayoutResult = relayoutWindow(params, viewVisibility, insetsPending);

            
            // 判断 surface 大小等数据是否发生变化
            ……

            //Surface的处理
            …… 

        } catch (RemoteException e) {
        }
        if (!mStopped || mReportNextDraw) {
                boolean focusChangedDueToTouchMode = ensureTouchModeLocally(
                        (relayoutResult&WindowManagerGlobal.RELAYOUT_RES_IN_TOUCH_MODE) != 0);
                //touch mode以及之前预测量的宽高不一样,则再次进行测量
                if (focusChangedDueToTouchMode || mWidth != host.getMeasuredWidth()
                        || mHeight != host.getMeasuredHeight() || contentInsetsChanged ||
                        updatedConfiguration) {
                    int childWidthMeasureSpec = getRootMeasureSpec(mWidth, lp.width);
                    int childHeightMeasureSpec = getRootMeasureSpec(mHeight, lp.height);

                     //3,进行测量
                    performMeasure(childWidthMeasureSpec, childHeightMeasureSpec);

                    ……
                    layoutRequested = true;
                }
            }
    } else {
       
        maybeHandleWindowMove(frame);
    }
    ……

    final boolean didLayout = layoutRequested && (!mStopped || mReportNextDraw);
    if (didLayout) {
        //4,执行布局工作
        performLayout(lp, mWidth, mHeight);
        //计算透明区域
        ……

    }

    //回调,请求焦点,输入法处理等工作
    …… 

    boolean cancelDraw = mAttachInfo.mTreeObserver.dispatchOnPreDraw() || !isViewVisible;
    if (!cancelDraw && !newSurface) {
        ……

        //4,绘制工作
        performDraw();
    } else {
        ……
    }

    mIsInTraversal = false;
}
复制代码

performTraversals函数非常的长,做的事情也非常多,它的关键流程只有下面这件事

  1. 预测量
  2. relayoutWindow,这一步会创建Layer和Surface
  3. 重新进行测量
  4. 布局
  5. 绘制

这篇文章主要讲图像缓冲区,所以测量,布局和绘制的流程都不讲了,我们这里只关心第二步,relayoutWindow是如何创建Surface的。

/frameworks/base/core/java/android/view/ViewRootImpl.java

private int relayoutWindow(WindowManager.LayoutParams params, int viewVisibility,
                           boolean insetsPending) throws RemoteException {

    float appScale = mAttachInfo.mApplicationScale;
    boolean restore = false;

    int relayoutResult = mWindowSession.relayout(
        mWindow, mSeq, params,
        (int) (mView.getMeasuredWidth() * appScale + 0.5f),
        (int) (mView.getMeasuredHeight() * appScale + 0.5f),
        viewVisibility, insetsPending ? WindowManagerGlobal.RELAYOUT_INSETS_PENDING : 0,
        mWinFrame, mPendingOverscanInsets, mPendingContentInsets, mPendingVisibleInsets,
        mPendingStableInsets, mPendingOutsets, mPendingBackDropFrame,
        mPendingMergedConfiguration, mSurface);

    return relayoutResult;
}
复制代码

relayoutWindow方法里面mWindowSession.relayout函数

/frameworks/base/services/core/java/com/android/server/wm/Session.java

public int relayout(IWindow window, int seq, WindowManager.LayoutParams attrs,
                    int requestedWidth, int requestedHeight, int viewFlags,
                    int flags, Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
                    Rect outVisibleInsets, Rect outStableInsets, Rect outsets, Rect outBackdropFrame,
                    MergedConfiguration mergedConfiguration, Surface outSurface) {

    int res = mService.relayoutWindow(this, window, seq, attrs,
                                      requestedWidth, requestedHeight, viewFlags, flags,
                                      outFrame, outOverscanInsets, outContentInsets, outVisibleInsets,
                                      outStableInsets, outsets, outBackdropFrame, mergedConfiguration, outSurface);
    return res;
}


复制代码

WindowSession的relayout函数调用了wms的relayoutWindow函数

/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java


public int relayoutWindow(Session session, IWindow client, int seq,
                          WindowManager.LayoutParams attrs, int requestedWidth,
                          int requestedHeight, int viewVisibility, int flags,
                          Rect outFrame, Rect outOverscanInsets, Rect outContentInsets,
                          Rect outVisibleInsets, Rect outStableInsets, Rect outOutsets, Rect outBackdropFrame,
                          MergedConfiguration mergedConfiguration, Surface outSurface) {
    //权限校验
    ……
    synchronized(mWindowMap) {
        WindowState win = windowForClientLocked(session, client, false);
        ……
        if (viewVisibility == View.VISIBLE &&
            (win.mAppToken == null || win.mAttrs.type == TYPE_APPLICATION_STARTING
             || !win.mAppToken.isClientHidden())) {

            // 进行布局,以分配正确大小的surface
            if (win.mLayoutSeq == -1) {
                win.setDisplayLayoutNeeded();
                mWindowPlacerLocked.performSurfacePlacement(true);
            }
            result = win.relayoutVisibleWindow(mergedConfiguration, result, attrChanges,
                                               oldVisibility);

            try {
                //创建SurfaceControl
                result = createSurfaceControl(outSurface, result, win, winAnimator);
            } catch (Exception e) {
                // 错误处理
                ……
                return 0;
            }
           
        } else {
            …… 
        }

       ……
    }

    
    return result;
}
复制代码

relayoutWindow函数主要做了两件事情

  1. 对所有Window进行测量布局和排序,前面讲到performTraversals会进行测量布局的流程,为什么这里还要进行一次呢?因为WMS中需要对所有的窗口进行一次整体的测量,比如activity的窗口会位于状态栏窗口和虚拟按键窗口之间,所以确定activity窗口的实际大小需要wms中结合其他的窗口才能进行。
  2. 执行createSurfaceControl

测量和排序的流程不在这儿详说了,还是只看和Surface创建相关函数createSurfaceControl

/frameworks/base/services/core/java/com/android/server/wm/WindowManagerService.java

private int createSurfaceControl(Surface outSurface, int result, WindowState win,
                                 WindowStateAnimator winAnimator) {
    if (!win.mHasSurface) {
        result |= RELAYOUT_RES_SURFACE_CHANGED;
    }

    WindowSurfaceController surfaceController;
    try { 
        //1,创建SurfaceControl
        surfaceController = winAnimator.createSurfaceLocked(win.mAttrs.type, win.mOwnerUid);
    } finally {
        Trace.traceEnd(TRACE_TAG_WINDOW_MANAGER);
    }
    if (surfaceController != null) {
        //2,创建Surface
        surfaceController.getSurface(outSurface);     
    } else {       
        outSurface.release();
    }

    return result;
}
复制代码

createSurfaceControl做了两件事情

  1. 通过createSurfaceLocked创建SurfaceController,这个过程会创建Layer
  2. 通过surfaceController.getSurface创建Surface

createSurfaceControl实际是调用了winAnimator的createSurfaceLocked函数。winAnimator是WinState的成员变量,是主要负责窗口动画的类。我们接着看createSurfaceLocked函数

/frameworks/base/services/core/java/com/android/server/wm/WindowStateAnimator.java

WindowSurfaceController createSurfaceLocked(int windowType, int ownerUid) {
        final WindowState w = mWin;
        //……

    	mSurfaceController = new WindowSurfaceController(mSession.mSurfaceSession,
                                                     attrs.getTitle().toString(),
                                                     width, height, format, flags, this, windowType, ownerUid);
		//……
            
        return mSurfaceController;
    }
复制代码

可以看到,这里以SurfaceSession为入参创建了WindowSurfaceController,SurfaceSession在前面addWindow中已经提到了,它其实就是SurfaceCompserClient。接着看WindowSurfaceController的构造函数。

/frameworks/base/services/core/java/com/android/server/wm/WindowSurfaceController.java

public WindowSurfaceController(SurfaceSession s, String name, int w, int h, int format,
                               int flags, WindowStateAnimator animator, int windowType, int ownerUid) {
    mAnimator = animator;

    mSurfaceW = w;
    mSurfaceH = h;

    title = name;

    mService = animator.mService;
    final WindowState win = animator.mWin;
    mWindowType = windowType;
    mWindowSession = win.mSession;


    mSurfaceControl = new SurfaceControl(
        s, name, w, h, format, flags, windowType, ownerUid);


    if (mService.mRoot.mSurfaceTraceEnabled) {
        mSurfaceControl = new RemoteSurfaceTrace(
            mService.mRoot.mSurfaceTraceFd.getFileDescriptor(), mSurfaceControl, win);
    }
}
复制代码

WindowSurfaceController的构造函数里面实际是以SurfaceSession为入参创建了SurfaceControl。

SurfaceControl的构造函数如下

/frameworks/base/core/java/android/view/SurfaceControl.java

public SurfaceControl(SurfaceSession session, String name, int w, int h, int format, int flags,
                      SurfaceControl parent, int windowType, int ownerUid)
    throws OutOfResourcesException {

    mName = name;
    mNativeObject = nativeCreate(session, name, w, h, format, flags,
                                 parent != null ? parent.mNativeObject : 0, windowType, ownerUid);
    ……
}
复制代码

这个SurfaceControl是java层的SurfaceControl,它在构造函数中通过jni方法nativeCreate创建了Native层的SurfaceControl。

接着看nativeCreated的实现。

/frameworks/base/core/jni/android_view_SurfaceControl.cpp

static jlong nativeCreate(JNIEnv* env, jclass clazz, jobject sessionObj,
                          jstring nameStr, jint w, jint h, jint format, jint flags, jlong parentObject,
                          jint windowType, jint ownerUid) {
    ScopedUtfChars name(env, nameStr);
    //通过SurfaceSession获取SurfaceComposerClient
    sp<SurfaceComposerClient> client(android_view_SurfaceSession_getClient(env, sessionObj));
    SurfaceControl *parent = reinterpret_cast<SurfaceControl*>(parentObject);
    //调用createSurface,创建Layer
    sp<SurfaceControl> surface = client->createSurface(
        String8(name.c_str()), w, h, format, flags, parent, windowType, ownerUid);
    if (surface == NULL) {
        jniThrowException(env, OutOfResourcesException, NULL);
        return 0;
    }

    surface->incStrong((void *)nativeCreate);
    return reinterpret_cast<jlong>(surface.get());
}
复制代码

在JNI方法nativeCreate中,我们发现了createSurface函数,在这里,Layer便创建好了,剩下的只剩调用getSurface创建Surface了。

getSurface

接着回到createSurfaceLocked流程中的第二步:执行surfaceController.getSurface()函数

/frameworks/base/services/core/java/com/android/server/wm/WindowSurfaceController.java

void getSurface(Surface outSurface) {
    outSurface.copyFrom(mSurfaceControl);
}
复制代码

这里的outSurface是前面ViewRootImpl创建的Surface,它这个时候的还只是一个空的Surface,需要SurfaceComposerClient在native层创建真正的Surface后,将Native层的Surface和这个Java层空的Surface绑定之后,ViewRootImpl中的Surface才是一个可用的Surface。

接着看copyFrom的实现

/frameworks/base/core/java/android/view/Surface.java

 public void copyFrom(SurfaceControl other) {
    
    long surfaceControlPtr = other.mNativeObject;
     //调用native方法创建Surface
    long newNativeObject = nativeGetFromSurfaceControl(surfaceControlPtr);

    synchronized (mLock) {
        if (mNativeObject != 0) {
            nativeRelease(mNativeObject);
        }
        //将native层的Surface和Java层的Surface绑定
        setNativeObjectLocked(newNativeObject);
    }
}
复制代码

接着看native函数nativeGetFromSurfaceControl

/frameworks/base/core/jni/android_view_Surface.cpp

static jlong nativeGetFromSurfaceControl(JNIEnv* env, jclass clazz,
        jlong surfaceControlNativeObj) {
   
    sp<SurfaceControl> ctrl(reinterpret_cast<SurfaceControl *>(surfaceControlNativeObj));
    //创建Surface
    sp<Surface> surface(ctrl->getSurface());
    if (surface != NULL) {
        surface->incStrong(&sRefBaseOwner);
    }
    return reinterpret_cast<jlong>(surface.get());
}
复制代码

nativeGetFromSurfaceControl函数的实现中,我们终于看到缓冲区创建流程的最后一步SurfaceControl->getSurface(),至此,在Activity的绘制流程中,Layer和Surface都已经创建完成了。

这里再总结一下Activity界面显示中缓冲区的创建流程,ViewRootImpl的setView函数中会执行**requestLayout()**和 mWindowSession.addToDisplay(),其中,在mWindowSession.addToDisplay()流程中,会调用WMS来创建SurfaceComponentClient;requestLayout()流程中会使用SurfaceComponentClient来创建Layer和Surface,Layer的创建函数为SurfaceCompoentClient.createSurface(),Surface的创建函数为SurfaceCompoentClient.getSurface。创建好了图像缓冲区后,requestLayout()就可以接着进行测量布局和绘制的操作流程了。

讲完了Surface和Layer的创建,我们接着看如何创建BufferQueue。

由于掘金的字数限制,这篇文章拆分为两篇,接下来的内容在掌握Android图像显示原理下(二),或者这里掌握Android图像显示原理(下)查看完整的内容

分类:
Android
标签:
收藏成功!
已添加到「」, 点击更改