SurfaceFlinger04-默认屏幕加载过程

1,131 阅读16分钟

前言

屏幕作为不可或缺的硬件设备之一,负责将显示内容呈现给用户。"Display"作为surfaceflinger进程中的元数据之一(另外一个是图层"Layer"),无论哪类屏幕设备,都是由surfaceflinger进程统一从HAL层进行加载和管理,并将屏幕设备信息向上提供给system_server和应用进程。

在对硬件屏幕抽象化为软件对象过程中,不同软件层有不同类型的类封装,从应用层API到surfaceflinger中,所有用于描述屏幕的类如下图所示:

display_arche.jpg

本篇文章中将对surfaceflinger启动过程中的加载默认屏流程进行详细分析。

一、onComposerHalHotplug()响应热插拔事件

SurfaceFlinger启动过程中,HWC HAL会主动触发一次热插拔事件,来进行默认内置屏的加载。

接《SurfaceFlinger01-SurfaceFlinger概述及启动过程 》第5.2小节分析,在初始化HWComposer组件过程中,会向HWC HAL注册HWC2::ComposerCallback接口,用于接收硬件HWC相关事件,如热插拔、VSYNC等:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWC2.h

struct ComposerCallback {
    // 发生热插拔事件时回调
    virtual void onComposerHalHotplug(hal::HWDisplayId, hal::Connection) = 0;
    ......
};

SurfaceFlinger类实现了该接口,并重写了onComposerHalHotplug()等方法。

初始化HWComposer过程中注册Callback方法如下:

// frameworks/native/services/surfaceflinger/DisplayHardware/AidlComposerHal.cpp

void AidlComposer::registerCallback(HWC2::ComposerCallback& callback) {
    ......
    // 向HWC HAL注册Callback
    mAidlComposerCallback = ndk::SharedRefBase::make<AidlIComposerCallbackWrapper>(callback);
    const auto status = mAidlComposerClient->registerCallback(mAidlComposerCallback);
    ......
}

当HWC HAL收到注册调用后,会主动发起一次热插拔事件——onComposerHalHotplug()回调,SurfaceFlinger中将利用这次热插拔事件来完成默认屏的加载:

// frameworks/native/services/SurfaceFlinger/SurfaceFlinger.cpp

void SurfaceFlinger::onComposerHalHotplug(hal::HWDisplayId hwcDisplayId,
                                          hal::Connection connection) {
    {
        std::lock_guard<std::mutex> lock(mHotplugMutex);
        // 将hotplug事件保存在mPendingHotplugEvents中
        mPendingHotplugEvents.push_back(HotplugEvent{hwcDisplayId, connection});
    }
    ......
}

在这个方法中,传入了Hal层屏幕ID和连接状态,SurfaceFlinger将参数封装到HotplugEvent中,并保存在mPendingHotplugEvents列表中。HotplugEvent定义如下:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.h

    struct HotplugEvent {
        // HAL层屏幕标识,uint64_t位整型
        hal::HWDisplayId hwcDisplayId;      
        // 热插拔事件类型,uint32_t位整型
        hal::Connection connection = hal::Connection::INVALID;   
    };

在接下来的configureLocked()方法中,将从mPendingHotplugEvents取出HotplugEvent,处理此次热插拔事件。

二、configureLocked()处理热插拔事件

configureLocked()方法中,将处理mPendingHotplugEvents中的热插拔事件,并从HWC HAL中加载默认屏幕:

// frameworks/native/services/SurfaceFlinger/SurfaceFlinger.cpp

bool SurfaceFlinger::configureLocked() {
    // 取出mPendingHotplugEvents中的事件
    std::vector<HotplugEvent> events;
    {
        std::lock_guard<std::mutex> lock(mHotplugMutex);
        events = std::move(mPendingHotplugEvents);
    }
    
    // 将HotplugEvent传递给HWComposer对象
    for (const auto [hwcDisplayId, connection] : events) {
        // HWComposer中根据hwcDisplayId加载默认屏硬件参数
        if (auto info = getHwComposer().onHotplug(hwcDisplayId, connection)) {
            const auto displayId = info->id;
            const bool connected = connection == hal::Connection::CONNECTED;
            // SurfaceFlinger中进一步处理热插拔事件
            if (const char* const log =
                        processHotplug(displayId, hwcDisplayId, connected, std::move(*info))) {
            }
        }
    }

    return !events.empty();
}

以上方法中:

  1. 执行HWComposer::onHotplug()方法,通过HWC HAL进行硬件屏幕标识信息读取,并进行HWC2::Display的创建;
  2. 执行SurfaceFlinger::processHotplug()方法,进行PhysicalDisplay的创建。

下面分别来看以上两操作。

2.1、HWC::onHotplugConnect()读取屏幕标识信息

在HWComposer::onHotplug()方法中,会根据connection值,决定执行热插拔连接或热插拔断开流程:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWComposer.cpp

std::optional<DisplayIdentificationInfo> HWComposer::onHotplug(hal::HWDisplayId hwcDisplayId,
                                                               hal::Connection connection) {
    switch (connection) {
        case hal::Connection::CONNECTED:
            // 执行热插拔屏幕连接操作
            return onHotplugConnect(hwcDisplayId);
        case hal::Connection::DISCONNECTED:
            // 执行热插拔屏幕断开操作
            return onHotplugDisconnect(hwcDisplayId);
        case hal::Connection::INVALID:
            return {};
    }
}

HWComposer::onHotplugConnect()方法中,将根据hwcDisplayId去加载所有硬件屏幕信息:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWComposer.cpp

std::optional<DisplayIdentificationInfo> HWComposer::onHotplugConnect(
        hal::HWDisplayId hwcDisplayId) {
    // 硬件屏幕标识信息
    std::optional<DisplayIdentificationInfo> info;
    if (const auto displayId = toPhysicalDisplayId(hwcDisplayId)) {
        // 首次加载,不满足条件,略过
        .......
    } else {
        // 从HWC中获取DisplayIdentification
        uint8_t port;
        DisplayIdentificationData data;
        const bool hasDisplayIdentificationData =
                getDisplayIdentificationData(hwcDisplayId, &port, &data);
        ......
        info = [this, hwcDisplayId, &port, &data, hasDisplayIdentificationData] {
            const bool isPrimary = !mPrimaryHwcDisplayId;
            ......
            return DisplayIdentificationInfo{.id = PhysicalDisplayId::fromPort(port),
                                             .name = isPrimary ? "Primary display"
                                                               : "Secondary display",
                                             .deviceProductInfo = std::nullopt};
        }();

        mComposer->onHotplugConnect(hwcDisplayId);
    }

    if (!isConnected(info->id)) {
        // 创建HWC2::Display对象
        allocatePhysicalDisplay(hwcDisplayId, info->id);
    }
    return info;
}

以上方法中:

  1. 通过getDisplayIdentificationData()方法会再次从HWC HAL中获取硬件屏幕标识信息,经过解析后并生成唯一的物理屏幕ID(Physical Display ID)、物理屏幕名称、制造商信息等数据,保存到了DisplayIdentificationInfo中;
  2. 通过allocatePhysicalDisplay()创建HWC2::Display对象。

2.1.1、获取硬件屏幕标识信息

这里从HWC HAL层读取到的屏幕标识数据经过解析后,将物理屏幕ID、物理屏幕名称、制造商信息保存到DisplayIdentificationInfo中,DisplayIdentificationInfo封装了屏幕对应的id、name、以及其他屏幕参数信息:

// frameworks/native/libs/ui/include/ui/DisplayIdentification.h

struct DisplayIdentificationInfo {
    PhysicalDisplayId id;                                  // physical display id
    std::string name;                                      // display name
    std::optional<DeviceProductInfo> deviceProductInfo;    // 屏幕制造信息
};

PhysicalDisplayId代表物理屏幕的唯一ID,并且将用于system_server进程DisplayManager中。PhysicalDisplayId实际是一个uint64_t类型id的包装类,结构体如下:

// frameworks/native/libs/ui/include/ui/DisplayId.h

// 表示屏幕id,实际是一个uint64_t类型id的包装类
struct DisplayId {
    // 实际ID值
    uint64_t value;
    // As above, but also upcast to Id.
    template <typename Id>
    static constexpr std::optional<Id> fromValue(uint64_t value) {
        if (const auto id = Id::tryCast(DisplayId(value))) {
            return id;
        }
        return {};
    }

};

// DisplayId of a physical display, such as the internal display or externally connected display.
struct PhysicalDisplayId : DisplayId {
......
private:
    constexpr PhysicalDisplayId(uint64_t flags, uint8_t port, uint16_t manufacturerId,
                                uint32_t modelHash)
          : DisplayId(flags | (static_cast<uint64_t>(manufacturerId) << 40) |
                      (static_cast<uint64_t>(modelHash) << 8) | port) {}
    .......
};

关于物理屏ID的来源与具体的解析流程,这里不再深入,见DisplayIdentification::parseDisplayIdentificationData()方法。

2.1.2、HWComposer::allocatePhysicalDisplay()创建HWC2::Display

完成硬件屏幕标识信息读取后,接下来,将执行allocatePhysicalDisplay()方法,创建HWC2::Display对象和DisplayData对象。

HWC2::Display是硬件屏幕在SurfaceFlinger进程内的第一层抽象封装,使用范围在HWComposer操作阶段,提供了操作硬件屏幕状态、亮度、VSYNC信号等方法。

DisplayData可以理解为HWC2::Display的"Owner",它持有HWC2::Display对象所有权,以及和该HWC2::Display相关的Fence相关信息。

allocatePhysicalDisplay()方法如下:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWComposer.cpp

void HWComposer::allocatePhysicalDisplay(hal::HWDisplayId hwcDisplayId,
                                         PhysicalDisplayId displayId) {
    // 关联hwcDisplayId和PhysicalDisplayId的键值对
    mPhysicalDisplayIdMap[hwcDisplayId] = displayId;
    // 设置主屏幕ID
    if (!mPrimaryHwcDisplayId) {
        mPrimaryHwcDisplayId = hwcDisplayId;
    }
    // 关联PhysicalDisplayId和DisplayData对象的键值对
    // 返回或创建DisplayData对象
    auto& displayData = mDisplayData[displayId];
    // 创建HWC2::Display对象
    auto newDisplay =
            std::make_unique<HWC2::impl::Display>(*mComposer.get(), mCapabilities, hwcDisplayId,
                                                  hal::DisplayType::PHYSICAL);
    // 设置HWC2::Display为连接状态
    newDisplay->setConnected(true);
    // 将HWC2::Display所有权交给DisplayData
    displayData.hwcDisplay = std::move(newDisplay);
}
2.1.2.1、创建HWC2::Display

HWC2::Display作为硬件屏幕的第一层抽象封装,定义如下:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWC2.h

class Display : public HWC2::Display {
public:
    Display(android::Hwc2::Composer&,
            const std::unordered_set<aidl::android::hardware::graphics::composer3::Capability>&,
            hal::HWDisplayId, hal::DisplayType);
    ~Display() override;
    // 设置屏幕状态
    hal::Error setPowerMode(hal::PowerMode) override;
    // 设置VSYNC开启/关闭
    hal::Error setVSYNCEnabled(hal::VSYNC enabled) override;
    // 设置亮度
    ftl::Future<hal::Error> setDisplayBrightness(
            float brightness, float brightnessNits,
            const Hwc2::Composer::DisplayBrightnessOptions& options) override;
    ......
};

// frameworks/native/services/surfaceflinger/DisplayHardware/HWC2.cpp
Display::Display(android::Hwc2::Composer& composer,
                 const std::unordered_set<AidlCapability>& capabilities, HWDisplayId id,
                 DisplayType type)
      : mComposer(composer), mCapabilities(capabilities), mId(id), mType(type) {
    ALOGV("Created display %" PRIu64, id);
}
2.1.2.2、创建DisplayData

DisplayData作为HWC2::Display的"Owner",定义如下:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWComposer.h

    struct DisplayData {
        // HWC2::Display对象
        std::unique_ptr<HWC2::Display> hwcDisplay;
        // 最近一次合成过程中的presentFence
        sp<Fence> lastPresentFence = Fence::NO_FENCE; // signals when the last set op retires
        nsecs_t lastPresentTimestamp = 0;
        // releaseFence列表
        std::unordered_map<HWC2::Layer*, sp<Fence>> releaseFences;

        bool validateWasSkipped;
        hal::Error presentError;

        bool VSYNCTraceToggle = false;

        std::mutex VSYNCEnabledLock;
        hal::VSYNC VSYNCEnabled GUARDED_BY(VSYNCEnabledLock) = hal::VSYNC::DISABLE;
    };

完成创建后,在allocatePhysicalDisplay()方法中用了两个Map来对HalDisplayId、PhysicalDisplayId、HWC2::Display进行关联:

// frameworks/native/services/surfaceflinger/DisplayHardware/HWComposer.h

// Key为Hal Display ID, 值为DisplayData对象
std::unordered_map<HalDisplayId, DisplayData> mDisplayData;
// Key为Hal Display ID, 值为物理屏ID
std::unordered_map<hal::HWDisplayId, PhysicalDisplayId> mPhysicalDisplayIdMap;
  • mPhysicalDisplayIdMap:键为Hal Display ID、值为物理屏ID的Map;
  • mDisplayData:键为物理屏ID、值为DisplayData对象的Map。

mPhysicalDisplayIdMap一般用于执行从下到上方向的调用(HWC HAL向SurfaceFlinger类),HWComposer收到接收到来自HWC HAL的回调事件后,根据事件传递的HalDisplayId找到对应的物理屏ID,并向上调用;

mDisplayData则一般用于执行从上到下方向的调用(SurfaceFlinger向HWC HAL),在HWComposer中,mDisplayData中根据物理屏ID获得DisplayData.hwcDisplay对象,然后进行向下调用。

display_sequence.jpg

经过以上流程,屏幕标识信息的加载,以及HWC2::Display、DisplayData对象创建完成。从Hotplug事件的上报,到最终HWC2::Display对象的创建,整个过程表示如下图:

display_abstract.jpg

HWComposer::onHotplug()方法执行完毕后,接下来会带着hwcDisplayId、PhysicalDisplayId对象等参数,执行processHotplug(),进行更上一层的屏幕对象的初始化。

2.2、SurfaceFlinger::processHotplug()初始化PhysicalDisplay

SurfaceFlinger::processHotplug()方法处理所有类型的热插拔事件,如连接、断开、重新连接等,本篇流程中仅看下处理热插拔连接事件的部分:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

const char* SurfaceFlinger::processHotplug(PhysicalDisplayId displayId,
                                           hal::HWDisplayId hwcDisplayId, bool connected,
                                           DisplayIdentificationInfo&& info) {
    // mPhysicalDisplays表示关联PhysicalDisplayId和PhysicalDisplay对象的Map
    const auto displayOpt = mPhysicalDisplays.get(displayId);
    // Disconnecting、Reconnecting流程略过
    ......
    // 获取当前屏幕的DisplayMode
    auto [displayModes, activeMode] = loadDisplayModes(displayId);
    // 获取当前屏幕的ColorMode
    ui::ColorModes colorModes = getHwComposer().getColorModes(displayId);

    // 新建一个Token标记,作为PhysicalDisplay的唯一标识
    const sp<IBinder> token = sp<BBinder>::make();
    // 创建PhysicalDisplay对象,并放入mPhysicalDisplays中
    mPhysicalDisplays.try_emplace(displayId, token, displayId,
                                  getHwComposer().getDisplayConnectionType(displayId),
                                  std::move(displayModes), std::move(colorModes),
                                  std::move(info.deviceProductInfo));
    // 创建DisplayDeviceState对象
    DisplayDeviceState state;
    // 物理参数
    state.physical = {.id = displayId,
                      .hwcDisplayId = hwcDisplayId,
                      .activeMode = std::move(activeMode)};
    state.isSecure = true; // All physical displays are currently considered secure.
    state.displayName = std::move(info.name);
    // 更新mCurrentState
    mCurrentState.displays.add(token, state);
    return "Connecting";
}

以上方法中:

  1. 创建PhysicalDisplay对象,并将其添加到mPhysicalDisplays映射表中;
  2. 创建DisplayDeviceState对象,并将其添加到mCurrentState.displays列表中。

2.2.1、创建PhysicalDisplay对象

PhysicalDisplay代表SurfaceFlinger范围内的硬件屏幕的抽象类,相比HWComposer范围内的HWC2::Display类更上一层,定义如下:

// frameworks/native/services/surfaceflinger/Display/PhysicalDisplay.h

class PhysicalDisplay {
public:
    // 构造函数
    template <typename... Args>
    PhysicalDisplay(sp<DisplayToken> token, Args&&... args)
          : mToken(std::move(token)), mSnapshot(std::forward<Args>(args)...) {}
.......
private:
    // 物理屏标识
    const sp<DisplayToken> mToken;

    // 物理屏配置参数快照
    DisplaySnapshot mSnapshot;
};

PhysicalDisplay只有两个成员变量:

  • mToken:该物理屏的唯一标识,该值贯穿了整个SurfaceFlinger和system_server进程屏幕管理机制,DisplayManager中通过该Token指定屏幕进行状态的更新;
  • DisplaySnapshot是热插拔事件发生后的屏幕快照,保存了物理屏相关属性,如ColorMode、DisplayModes、连接类型等等。

mPhysicalDisplays是一个键为PhysicalDisplayId、值为PhysicalDisplay对象的Map:

// frameworks/native/services/surfaceflinger/Display/PhysicalDisplay.h

using PhysicalDisplays = PhysicalDisplayMap<PhysicalDisplayId, PhysicalDisplay>;

这个Map作用相当重要,system_server进程DisplayManager中获取物理屏信息时,就是通过PhysicalDisplayId或token从mPhysicalDisplays获取,比如要获取物理屏ID时:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

std::vector<PhysicalDisplayId> SurfaceFlinger::getPhysicalDisplayIdsLocked() const {
    std::vector<PhysicalDisplayId> displayIds;
    ......
    for (const auto& [id, display] : mPhysicalDisplays) {
        if (id != defaultDisplayId) {
            displayIds.push_back(id);
        }
    }
    return displayIds;
}

再比如根据Token获取PhysicalDisplayId时:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.h

    sp<display::DisplayToken> getPhysicalDisplayTokenLocked(PhysicalDisplayId displayId) const
            REQUIRES(mStateLock) {
        return mPhysicalDisplays.get(displayId)
                .transform([](const display::PhysicalDisplay& display) { return display.token(); })
                .or_else([] { return std::optional<sp<display::DisplayToken>>(nullptr); })
                .value();
    }

2.2.2、创建DisplaySnapshot

DisplaySnapshot作为PhysicalDisplay除Token标识之外的第二个属性,保存了硬件屏幕相关信息,如DisplayMode、ColorMode等:

// frameworks/native/services/surfaceflinger/Display/DisplaySnapshot.cpp

DisplaySnapshot::DisplaySnapshot(PhysicalDisplayId displayId,
                                 ui::DisplayConnectionType connectionType,
                                 DisplayModes&& displayModes, ui::ColorModes&& colorModes,
                                 std::optional<DeviceProductInfo>&& deviceProductInfo)
      : mDisplayId(displayId),
        mConnectionType(connectionType),
        mDisplayModes(std::move(displayModes)),
        mColorModes(std::move(colorModes)),
        mDeviceProductInfo(std::move(deviceProductInfo)) {}

SurfaceFlinger中获取物理屏DisplayMode、ColorMode属性时,就会从DisplaySnapShot中获取。

2.2.3、创建DisplayDeviceState对象

DisplayDeviceState表示具体的显示设备的状态集,其中定义了显示设备的宽、高、LayerStack等信息,无论内置屏、扩展屏、虚拟屏,每个屏幕都会有各自的DisplayDeviceState对象。

DisplayDeviceState用来描述DisplayDevice的状态,DisplayDevice根据DisplayDeviceState中的配置进行创建或更新(DisplayDevice表示一个显示设备)。该类定义如下:

// frameworks/native/services/surfaceflinger/DisplayDevice.h

struct DisplayDeviceState {
    // 物理屏相关属性
    struct Physical {
        PhysicalDisplayId id;
        hardware::graphics::composer::hal::HWDisplayId hwcDisplayId;
        DisplayModePtr activeMode;

        bool operator==(const Physical& other) const {
            return id == other.id && hwcDisplayId == other.hwcDisplayId;
        }
    };
    // 是否为虚拟屏
    bool isVirtual() const { return !physical; }
    // 序列ID,用户区分DisplayDeviceState
    int32_t sequenceId = sNextSequenceId++;
    // 物理屏属性
    std::optional<Physical> physical;
    sp<IGraphicBufferProducer> surface;
    // LayerStack ID值
    ui::LayerStack layerStack;
    uint32_t flags = 0;
    // LayerStack区域大小
    Rect layerStackSpaceRect;
    // 内容显示区域大小
    Rect orientedDisplaySpaceRect;
    // 显示方向
    ui::Rotation orientation = ui::ROTATION_0;
    // 屏幕宽高
    uint32_t width = 0;
    uint32_t height = 0;
    std::string displayName;
    bool isSecure = false;
    // Refer to DisplayDevice::mRequestedRefreshRate, for virtual display only
    Fps requestedRefreshRate;

private:
    static std::atomic<int32_t> sNextSequenceId;
};

完成DisplayDeviceState创建后,会以Token为键将它保存到mCurrentState.displays中。

此时,configureLocked()方法执行完毕。之后将根据mCurrentState.displays中存储的DisplayDeviceState进一步初始化屏幕相关的对象。

三、SurfaceFlinger::processDisplayAdded()

继续回到SurfaceFlinger::init()方法中,接下来,将从mCurrentState.displays中取出刚刚创建的DisplayDeviceState对象和Token对象,执行processDisplayAdded()方法:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::init() FTL_FAKE_GUARD(kMainThreadContext) {
    ......
    LOG_ALWAYS_FATAL_IF(!configureLocked(),
                    "Initial display configuration failed: HWC did not hotplug");
                    
    // Commit primary display.
    sp<const DisplayDevice> display;
    if (const auto indexOpt = mCurrentState.getDisplayIndex(getPrimaryDisplayIdLocked())) {
        const auto& displays = mCurrentState.displays;

        const auto& token = displays.keyAt(*indexOpt);
        const auto& state = displays.valueAt(*indexOpt);
        // Display添加流程
        processDisplayAdded(token, state);
        // 添加到mDrawingState.displays中
        mDrawingState.displays.add(token, state);

        display = getDefaultDisplayDeviceLocked();
    }
    ......
}

在processDisplayAdded()方法中,会依次完成Composition::Display、DisplayDevice等对象的初始化:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

void SurfaceFlinger::processDisplayAdded(const wp<IBinder>& displayToken,
                                         const DisplayDeviceState& state) {
    .......
    // compositionengine::Display构造器对象
    compositionengine::DisplayCreationArgsBuilder builder;
    // 设置display id
    if (const auto& physical = state.physical) {
        builder.setId(physical->id);
    } else {
        builder.setId(acquireVirtualDisplay(resolution, pixelFormat));
    }
    
    builder.setPixels(resolution);                  // 设置resolution
    builder.setIsSecure(state.isSecure);            // 设置是否是Secure屏幕
    builder.setPowerAdvisor(mPowerAdvisor.get());   // 设置PowerAdvisor
    builder.setName(state.displayName);             // 设置Name
    // 创建compositionengine::Display对象
    auto compositionDisplay = getCompositionEngine().createDisplay(builder.build());
    compositionDisplay->setLayerCachingEnabled(mLayerCachingEnabled);

    // 创建BufferQueue、BufferProducer和BufferComsumer,用于对应屏幕的GPU合成
    sp<compositionengine::DisplaySurface> displaySurface;
    sp<IGraphicBufferProducer> producer;
    sp<IGraphicBufferProducer> bqProducer;
    sp<IGraphicBufferConsumer> bqConsumer;
    getFactory().createBufferQueue(&bqProducer, &bqConsumer, /*consumerIsSurfaceFlinger =*/false);

    // 创建compositionengine::DisplaySurface
    if (state.isVirtual()) {
        // 虚拟屏处理,略去
    } else {
        const auto displayId = PhysicalDisplayId::tryCast(compositionDisplay->getId());
        LOG_FATAL_IF(!displayId);
        // 创建FramebufferSurface对象,用于GPU合成
        displaySurface =
                sp<FramebufferSurface>::make(getHwComposer(), *displayId, bqConsumer,
                                             state.physical->activeMode->getResolution(),
                                             ui::Size(maxGraphicsWidth, maxGraphicsHeight));
        producer = bqProducer;
    }
    // 创建DisplayDevice对象
    auto display = setupNewDisplayDeviceInternal(displayToken, std::move(compositionDisplay), state,
                                                 displaySurface, producer, qtiDSExtnIntf);

    ......
    
    // 添加到mDisplaysMap中
    mDisplays.try_emplace(displayToken, std::move(display));
    ......
}

以上方法中:

  1. 通过getCompositionEngine().createDisplay()创建Compositionengine::Display实例,在合成阶段作为合成输出屏幕;
  2. 通过getFactory().createBufferQueue()创建一个BufferQueue队列,用于在GPU合成时获取GraphicBuffer;
  3. 创建DisplaySurface实例,也用于GPU合成;
  4. 通过setupNewDisplayDeviceInternal()创建DisplayDevice实例。

下面分别进行介绍。

3.1、创建Compositionengine::Display

compositionengine::Display是继承于Output接口的一个接口。

surfaceflinger中包含来自不同进程的Layer,经过DPU或GPU进行合成后输出到屏幕上,此时屏幕可以理解为合成结果的输出目标,而Output类就是这个合成输出的抽象封装。

Output作为Layer合成输出最顶层接口,针对不同场景分别有两个子类实现:

  • compositionengine::Display:输出端为具体屏幕(如物理屏、虚拟屏),一个屏幕对应一个compositionengine::Display;
  • ScreenCaptureOutput:输出端为预留缓冲区,如截图场景,把多个layer截取并合成输出到一张图片;

还有一点需要了解的是,所有的Output实现类都是通过Output类中的模板函数createOutputTemplated()方法创建:

// frameworks/native/services/surfaceflinger/CompositionEngine/include/compositionengine/impl/Output.h

template <typename BaseOutput, typename CompositionEngine, typename... Args>
std::shared_ptr<BaseOutput> createOutputTemplated(const CompositionEngine& compositionEngine,
                                                  Args... args) {
    // 实际创建继承于泛型类BaseOutput的子类                                              
    class Output final : public BaseOutput {
    public:
        ......
        explicit Output(const CompositionEngine& compositionEngine, Args... args)
              : BaseOutput(std::forward<Args>(args)...), mCompositionEngine(compositionEngine) {}
        ~Output() override = default;
        ......
    };
    
    // 返回实际的Output类对象
    return std::make_shared<Output>(compositionEngine, std::forward<Args>(args)...);
}

关于该模板函数具体细节,见合成相关流程文档中对这部分做细节说明。

现在来看下compositionengine::Display对象的创建,在经过

CompositionEngine::createDisplay() -> compositionengine::impl::createDisplay() -> createDisplayTemplated() ,最终在createOutputTemplated()方法中进行compositionengine::Display对象的创建:

// frameworks/native/services/surfaceflinger/CompositionEngine/include/compositionengine/impl/Output.h

std::shared_ptr<BaseOutput> createOutputTemplated(const CompositionEngine& compositionEngine,
                                                  Args... args) {
    class Output final : public BaseOutput {
    public:
        ......
        explicit Output(const CompositionEngine& compositionEngine, Args... args)
              : BaseOutput(std::forward<Args>(args)...), mCompositionEngine(compositionEngine) {}
        ~Output() override = default;

    private:
        // compositionengine::Output overrides
        const OutputCompositionState& getState() const override { return mState; }

        OutputCompositionState& editState() override { return mState; }
        ......

        const CompositionEngine& mCompositionEngine;
        OutputCompositionState mState;
    };

    return std::make_shared<Output>(compositionEngine, std::forward<Args>(args)...);
}

以上代码采用模板编程方式实现,BaseOutput即是compositionengine::Display,实际创建的是则它的子类对象。该对象中的其他参数和方法先暂不关注,在合成过程中才会用到。

这里对OutputCompositionState属性提前做个说明:它是合成过程中用于记录状态的属性,如屏幕的合成方式、内容投射区域、色彩空间、色彩矩阵等等.....

创建完成后,在createDisplayTemplated()方法中,将compositionengine::DisplayCreationArgs构造器对象中的参数设置给compositionengine::Display:

// frameworks/native/services/surfaceflinger/CompositionEngine/include/compositionengine/impl/Display.h

template <typename BaseDisplay, typename CompositionEngine>
std::shared_ptr<BaseDisplay> createDisplayTemplated(
        const CompositionEngine& compositionEngine,
        const compositionengine::DisplayCreationArgs& args) {
    auto display = createOutputTemplated<BaseDisplay>(compositionEngine);

    display->setConfiguration(args);

    return display;
}

// frameworks/native/services/surfaceflinger/CompositionEngine/src/Display.cpp

void Display::setConfiguration(const compositionengine::DisplayCreationArgs& args) {
    mId = args.id;
    mPowerAdvisor = args.powerAdvisor;
    editState().isSecure = args.isSecure;
    // 设置displaySpace区域范围,用于指定显示内容区域
    editState().displaySpace.setBounds(args.pixels);
    setName(args.name);
}

此时,compositionengine::Display创建完成。

3.2、创建BufferQueue

BufferQueue表示图形缓冲数据队列,用于图形绘制过程中客户端(Surface)和服务端(GPU)间图形Buffer的交换和转移。SurfaceFlinger中创建的BufferQueue,主要有两个用途:

  1. GPU合成过程时,进行Buffer的转移和消费;
  2. 虚拟屏上,获取客户端的Buffer数据。

关于BufferQueue机制具体细节,见《BBQ-01-BLASTBufferQueue架构概述 》。

3.3、创建DisplaySurface

DisplaySurface用于和BufferQueue协同工作。它继承了ComsumerBase类,因此可以理解为BufferQueue中的"消费者"。其实现类有两个:

  • FramebufferSurface:用户物理屏合成时;
  • VirtualDisplaySurface:用于虚拟屏。

3.4、创建DisplayDevice对象

接下来,将通过SurfaceFlinger::setupNewDisplayDeviceInternal()方法创建DisplayDevice对象。DisplayDevice表示一个显示设备,不仅包括物理屏属性,还包括逻辑属性、合成输出状态集、以及BufferQueue队列、DisplaySurface等对象状态。该方法如下:

// frameworks/native/services/surfaceflinger/SurfaceFlinger.cpp

sp<DisplayDevice> SurfaceFlinger::setupNewDisplayDeviceInternal(
        const wp<IBinder>& displayToken,
        std::shared_ptr<compositionengine::Display> compositionDisplay,
        const DisplayDeviceState& state,
        const sp<compositionengine::DisplaySurface>& displaySurface,
        const sp<IGraphicBufferProducer>& producer) {
    // DisplayDevice构造参数
    DisplayDeviceCreationArgs creationArgs(sp<SurfaceFlinger>::fromExisting(this), getHwComposer(),
                                           displayToken, compositionDisplay);
    // DisplayDevice序列号
    creationArgs.sequenceId = state.sequenceId;
    // 是否安全显示设备
    creationArgs.isSecure = state.isSecure;
    // DisplaySurface对象
    creationArgs.displaySurface = displaySurface;
    creationArgs.hasWideColorGamut = false;
    creationArgs.supportedPerFrameMetadata = 0;
    // 针对物理屏幕参数设置
    if (const auto& physical = state.physical) {
        // 当前使用的DisplayMode
        creationArgs.activeModeId = physical->activeMode->getId();
        const auto [kernelIdleTimerController, idleTimerTimeoutMs] =
                getKernelIdleTimerProperties(compositionDisplay->getId());

        // 刷新率设置
        using Config = scheduler::RefreshRateSelector::Config;
        const auto enableFrameRateOverride = sysprop::enable_frame_rate_override(true)
                ? Config::FrameRateOverride::Enabled
                : Config::FrameRateOverride::Disabled;
        Config config =
                {.enableFrameRateOverride = enableFrameRateOverride,
                 .frameRateMultipleThreshold =
                         base::GetIntProperty("debug.sf.frame_rate_multiple_threshold", 0),
                 .idleTimerTimeout = idleTimerTimeoutMs,
                 .kernelIdleTimerController = kernelIdleTimerController};

        creationArgs.refreshRateSelector =
                mPhysicalDisplays.get(physical->id)
                        .transform(&PhysicalDisplay::snapshotRef)
                        .transform([&](const display::DisplaySnapshot& snapshot) {
                            return std::make_shared<
                                    scheduler::RefreshRateSelector>(snapshot.displayModes(),
                                                                    creationArgs.activeModeId,
                                                                    config);
                        })
                        .value_or(nullptr);
        // 是否为主屏幕
        creationArgs.isPrimary = physical->id == getPrimaryDisplayIdLocked();
        // 色彩管理
        if (useColorManagement) {
            mPhysicalDisplays.get(physical->id)
                    .transform(&PhysicalDisplay::snapshotRef)
                    .transform(ftl::unit_fn([&](const display::DisplaySnapshot& snapshot) {
                        for (const auto mode : snapshot.colorModes()) {
                            creationArgs.hasWideColorGamut |= ui::isWideColorMode(mode);
                            creationArgs.hwcColorModes
                                    .emplace(mode,
                                             getHwComposer().getRenderIntents(physical->id, mode));
                        }
                    }));
        }
    }

    if (const auto id = HalDisplayId::tryCast(compositionDisplay->getId())) {
        // HDR能力
        getHwComposer().getHdrCapabilities(*id, &creationArgs.hdrCapabilities);
        creationArgs.supportedPerFrameMetadata = getHwComposer().getSupportedPerFrameMetadata(*id);
    }
    // 创建一个Surface作为BufferQueue Producer
    auto nativeWindowSurface = getFactory().createNativeWindowSurface(producer);
    auto nativeWindow = nativeWindowSurface->getNativeWindow();
    creationArgs.nativeWindow = nativeWindow;

    // Make sure that composition can never be stalled by a virtual display
    // consumer that isn't processing buffers fast enough. We have to do this
    // here, in case the display is composed entirely by HWC.
    if (state.isVirtual()) {
        nativeWindow->setSwapInterval(nativeWindow.get(), 0);
    }
    // 屏幕方向
    creationArgs.physicalOrientation =
            getPhysicalDisplayOrientation(compositionDisplay->getId(), creationArgs.isPrimary);

    // 默认屏幕状态
    creationArgs.initialPowerMode =
            state.isVirtual() ? std::make_optional(hal::PowerMode::ON) : std::nullopt;
    // 刷新率
    creationArgs.requestedRefreshRate = state.requestedRefreshRate;

    // 创建DisplayDevice
    sp<DisplayDevice> display = getFactory().createDisplayDevice(creationArgs);
    // 预申请GraphicBuffer
    nativeWindowSurface->preallocateBuffers();

    // 设置色彩空间
    ui::ColorMode defaultColorMode = ui::ColorMode::NATIVE;
    Dataspace defaultDataSpace = Dataspace::UNKNOWN;
    if (display->hasWideColorGamut()) {
        defaultColorMode = ui::ColorMode::SRGB;
        defaultDataSpace = Dataspace::V0_SRGB;
    }
    display->getCompositionDisplay()->setColorProfile(
            compositionengine::Output::ColorProfile{defaultColorMode, defaultDataSpace,
                                                    RenderIntent::COLORIMETRIC,
                                                    Dataspace::UNKNOWN});

    if (const auto& physical = state.physical) {
        const auto& mode = *physical->activeMode;
        display->setActiveMode(mode.getId(), mode.getFps(), mode.getFps());
    }
    // 设置LayerFilter
    display->setLayerFilter(makeLayerFilterForDisplay(display->getId(), state.layerStack));
    // 设置显示区域
    display->setProjection(state.orientation, state.layerStackSpaceRect,
                           state.orientedDisplaySpaceRect);
    // 设置DisplayDevice名称
    display->setDisplayName(state.displayName);
    display->setFlags(state.flags);

    return display;
}

DisplayDevice构造方法如下:

// frameworks/native/services/surfaceflinger/DisplayDevice.cpp

DisplayDevice::DisplayDevice(DisplayDeviceCreationArgs& args)
      : mFlinger(args.flinger),
        mHwComposer(args.hwComposer),
        mDisplayToken(args.displayToken),
        mSequenceId(args.sequenceId),
        mCompositionDisplay{args.compositionDisplay},
        mActiveModeFPSTrace("ActiveModeFPS -" + to_string(getId())),
        mActiveModeFPSHwcTrace("ActiveModeFPS_HWC -" + to_string(getId())),
        mRenderFrameRateFPSTrace("RenderRateFPS -" + to_string(getId())),
        mPhysicalOrientation(args.physicalOrientation),
        mIsPrimary(args.isPrimary),
        mRequestedRefreshRate(args.requestedRefreshRate),
        mRefreshRateSelector(std::move(args.refreshRateSelector)) {
    mCompositionDisplay->editState().isSecure = args.isSecure;
    // 创建RenderSurface
    mCompositionDisplay->createRenderSurface(
            compositionengine::RenderSurfaceCreationArgsBuilder()
                    .setDisplayWidth(ANativeWindow_getWidth(args.nativeWindow.get()))
                    .setDisplayHeight(ANativeWindow_getHeight(args.nativeWindow.get()))
                    .setNativeWindow(std::move(args.nativeWindow))
                    .setDisplaySurface(std::move(args.displaySurface))
                    .setMaxTextureCacheSize(
                            static_cast<size_t>(SurfaceFlinger::maxFrameBufferAcquiredBuffers))
                    .qtiSetDisplaySurfaceExtension(args.mQtiDSExtnIntf)
                    .build());

    if (!mFlinger->mDisableClientCompositionCache &&
        SurfaceFlinger::maxFrameBufferAcquiredBuffers > 0) {
        mCompositionDisplay->createClientCompositionCache(
                static_cast<uint32_t>(SurfaceFlinger::maxFrameBufferAcquiredBuffers));
    }

    mCompositionDisplay->setPredictCompositionStrategy(mFlinger->mPredictCompositionStrategy);
    mCompositionDisplay->setTreat170mAsSrgb(mFlinger->mTreat170mAsSrgb);
    mCompositionDisplay->createDisplayColorProfile(
            compositionengine::DisplayColorProfileCreationArgsBuilder()
                    .setHasWideColorGamut(args.hasWideColorGamut)
                    .setHdrCapabilities(std::move(args.hdrCapabilities))
                    .setSupportedPerFrameMetadata(args.supportedPerFrameMetadata)
                    .setHwcColorModes(std::move(args.hwcColorModes))
                    .Build());

    ......
    mCompositionDisplay->getRenderSurface()->initialize();

    if (const auto powerModeOpt = args.initialPowerMode) {
        setPowerMode(*powerModeOpt);
    }

    // initialize the display orientation transform.
    setProjection(ui::ROTATION_0, Rect::INVALID_RECT, Rect::INVALID_RECT);

    char value[PROPERTY_VALUE_MAX];
}

在DisplayDevice构造方法中,会创建一个RenderSurface对象,它用于GPU合成方式时,作为Surface进行图形缓冲数据GraphicBuffer的dequeueBufferqueueBuffer操作。

完成创建后,将DisplayDevice对象作为值,Binder token为键保存在了mDisplays中。

// May be read from any thread, but must only be written from the main thread.
display::DisplayMap<wp<IBinder>, const sp<DisplayDevice>> mDisplays GUARDED_BY(mStateLock);

从PhysicalDisplay到最终DisplayDevice对象的创建,整个过程表示如下图:

create_displaydevice.jpg

此时,屏幕加载的整个流程执行完毕。整个过程核心时序图如下: