1. 用PlatformView来做什么
用PlatformView做音视频直播!Flutter在UI呈现上具有极强的能力,但Widget在视频渲染方面还是存在很多不足的。目前市面上使用Flutter做视频直播的主流方案有Texture Widget和PlatformView,Google官方开源的视频播放插件video_player plugin是基于Texture Widget的。开发同学的Flutter环境是1.9.1+hotfix-7,经过实验我们最终选择PlatformView的方案来做。
原因有一下几点:
1)经过实际的实验数据对比,PlatfomView的实际性能指标比纯Native还是要逊色一点的,但相差不远
2)结合我们现有的媒体sdk的接口,使用PlatformView比较实际
3)兄弟团队早在Flutter1.0版本上有使用PlatformView的经验
4)应对直播间复杂的UI交互,PlatformView实际使用起来更友好。
2. PlatformView到底是个啥
2.1 PlatformView是一个通过flutter plugin的形式来创建NativeView的技术。
PlatformView在Dart中的类对应到iOS和Android平台分别是UiKitView和AndroidView,AndroidView和 UiKitView的定义本质是一个StatefulWidget。 以UiKitView为例,先从Framework层来看看PlatformView定义以及PlatformView的创建流程。
class UiKitView extends StatefulWidget {
const UiKitView({
Key key,
@required this.viewType,
this.onPlatformViewCreated,
this.hitTestBehavior = PlatformViewHitTestBehavior.opaque,
this.layoutDirection,
this.creationParams,
this.creationParamsCodec,
this.gestureRecognizers,
}) : assert(viewType != null),
assert(hitTestBehavior != null),
assert(creationParams == null || creationParamsCodec != null),
super(key: key);
````
@override
State<UiKitView> createState() => _UiKitViewState();
}
class _UiKitViewState extends State<UiKitView> {
````
@override
Widget build(BuildContext context) {
return _UiKitPlatformView(
controller: _controller,
hitTestBehavior: widget.hitTestBehavior,
gestureRecognizers: widget.gestureRecognizers ?? _emptyRecognizersSet,
);
}
Future<void> _createNewUiKitView() async {
//id 是个++i的操作,会传到在Engine中作为key存储当前创建的View
final int id = platformViewsRegistry.getNextPlatformViewId();
final UiKitViewController controller = await PlatformViewsService.initUiKitView(
id: id,
viewType: widget.viewType,
layoutDirection: _layoutDirection,
creationParams: widget.creationParams,
creationParamsCodec: widget.creationParamsCodec,
);
if (!mounted) {
controller.dispose();
return;
}
if (widget.onPlatformViewCreated != null) {
widget.onPlatformViewCreated(id);
}
setState(() { _controller = controller; });
}
}
需要说明下viewType是注册pluign时,Dart侧和Native侧约定的是必须要传的一个字符传,用来注册PlatformView的类型。继续看PlatformViewsService中initUiKitView的实现:
class PlatformViewsService {
````
static Future<UiKitViewController> initUiKitView({
@required int id,
@required String viewType,
@required TextDirection layoutDirection,
dynamic creationParams,
MessageCodec<dynamic> creationParamsCodec,
}) async {
assert(id != null);
assert(viewType != null);
assert(layoutDirection != null);
assert(creationParams == null || creationParamsCodec != null);
final Map<String, dynamic> args = <String, dynamic>{
'id': id,
'viewType': viewType,
};
if (creationParams != null) {
final ByteData paramsByteData = creationParamsCodec.encodeMessage(creationParams);
args['params'] = Uint8List.view(
paramsByteData.buffer,
0,
paramsByteData.lengthInBytes,
);
}
//重点在这里,SystemChannels中定义了一些Flutter 与Engine之前进行通信的channel, 如lifeCycle
await SystemChannels.platform_views.invokeMethod<void>('create', args);
return UiKitViewController._(id, layoutDirection);
}
}
不难看出initUiKitView是通过SystemChannels中的platform_views这个channel与Native进行通信,发送create消息到Native层。SystemChannels中定义了的channel给Dart与Engine之间进行通信,因此我们需要编译下Flutter的Engine。继续往下看create在Engine中的实现,最终定位到FlutterPlatformViews类OnCreate这个方法,核心实现代码如下
void FlutterPlatformViewsController::OnCreate(FlutterMethodCall* call, FlutterResult& result) {
````
NSDictionary<NSString*, id>* args = [call arguments];
long viewId = [args[@"id"] longValue];
std::string viewType([args[@"viewType"] UTF8String]);
//先根据viewId去查缓存
if (views_.count(viewId) != 0) {
result([FlutterError errorWithCode:@"recreating_view"
message:@"trying to create an already created view"
details:[NSString stringWithFormat:@"view id: '%ld'", viewId]]);
}
//检测viewType的合法性
NSObject<FlutterPlatformViewFactory>* factory = factories_[viewType].get();
if (factory == nil) {
result([FlutterError errorWithCode:@"unregistered_view_type"
message:@"trying to create a view with an unregistered type"
details:[NSString stringWithFormat:@"unregistered view type: '%@'",
args[@"viewType"]]]);
return;
}
id params = nil;
//参数编码格式是StandardMethodCodec类型,这个在platform_views这个channel初始化时有定义
if ([factory respondsToSelector:@selector(createArgsCodec)]) {
NSObject<FlutterMessageCodec>* codec = [factory createArgsCodec];
if (codec != nil && args[@"params"] != nil) {
FlutterStandardTypedData* paramsData = args[@"params"];
params = [codec decode:paramsData.data];
}
}
//根据FlutterPlatformView中的实现创建View
NSObject<FlutterPlatformView>* embedded_view = [factory createWithFrame:CGRectZero
viewIdentifier:viewId
arguments:params];
views_[viewId] = fml::scoped_nsobject<NSObject<FlutterPlatformView>>([embedded_view retain]);
//创建一个Touch交互的View作为我们定义的embedded_view.view的父View
FlutterTouchInterceptingView* touch_interceptor = [[[FlutterTouchInterceptingView alloc]
initWithEmbeddedView:embedded_view.view
flutterViewController:flutter_view_controller_.get()] autorelease];
touch_interceptors_[viewId] =
fml::scoped_nsobject<FlutterTouchInterceptingView>([touch_interceptor retain]);
root_views_[viewId] = fml::scoped_nsobject<UIView>([touch_interceptor retain]);
result(nil);
}
到此PlatformView创建调用流程就结束了。PlatformView在Dart侧以UiKitView和AndroidView的形式暴露给开发者来创建NativeView;当PlatformView销毁时,UiKitView的dispose方法会通过platform_views来向Engine发送dispose消息,在FlutterPlatformViews.mm的OnDispose方法中被销毁。
2.2 使用PlatformView的套路
从Flutter Engine层关于OnCreate的实现来看,我们需要创建一个Flutter Plugin项目并注册该plugin,Native端我们要实现FlutterPlatformViewFactory以及FlutterPlatformView这两个协议,在Plugin中根据viewType来注册ViewFactory.直接上代码。
2.2.1 FlutterPlatformViewFactory的实现
@implementation RenderViewFactory
- (nonnull NSObject<FlutterPlatformView> *)createWithFrame:(CGRect)frame viewIdentifier:(int64_t)viewId arguments:(id _Nullable)args
{
RenderView *rendererView = [[RenderView alloc] initWithFrame:frame viewIdentifier:viewId];
return rendererView;
}
- (NSObject<FlutterMessageCodec> *)createArgsCodec
{
return [FlutterStandardMessageCodec sharedInstance];
}
@end
2.2.2 FlutterPlatformView的实现
@implementation RenderView
- (instancetype)initWithFrame:(CGRect)frame viewIdentifier:(int64_t)viewId
{
if (self = [super init]) {
self.mainView = [[UIView alloc] initWithFrame:frame];
}
return self;
}
- (UIView *)view
{
return self.mainView;
}
@end
2.2.3 注册viewType和ViewFactory
+ (void)registerWithRegistrar:(NSObject<FlutterPluginRegistrar>*)registrar
{
FlutterMethodChannel *channel = [FlutterMethodChannel methodChannelWithName:kRenderChannel
binaryMessenger:[registrar messenger]];
//注册ViewFactory和viewType
ThunderRenderViewFactory *renderViewFactory = [[ThunderRenderViewFactory alloc] init];
[registrar registerViewFactory:renderViewFactory withId:kRenderViewType];
}
2.2.4 Dart层创建关联的类
class RenderViewPlugin {
static const MethodChannel _channel = MethodChannel(kRenderChannel);
//提供一个创建NativeView的方法
static Widget createRenderView(Function(int viewId) created, {Key key}) {
if (defaultTargetPlatform == TargetPlatform.iOS) {
return UiKitView(
key: key,
viewType: kRenderViewType,
onPlatformViewCreated: (viewId) {
if (created != null) {
created(viewId);
}
},
);
} else if (defaultTargetPlatform == TargetPlatform.android){
return AndroidView(
key: key,
viewType: kRenderViewType,
onPlatformViewCreated: (viewId) {
if (created != null) {
created(viewId);
}
},
);
}
return null;
}
3. 在实际业务中使用PlatformView
3.1 一个实际的需求
直播间多人连麦场景下,要求视频流渲染的View布局随着人数动态变化,且主播点击任一路视频流可以全屏。
3.2 实现核心的视频功能
Dart层推流localWidget和拉流remoteWidget均为UiKitView来,localWidget和remoteWidget对应的Native View我们给到媒体sdk视频流渲染,localWidget和remoteWidget采用Stack布局方便后面全屏模式下的层级切换。核心代码如下:
class _LiveViewStackState extends State<LiveViewStack> {
····
List<Widget> views = [
Positioned(
width: 200,
height: MediaQuery.of(context).size.height,
child: RenderViewPlugin.createNativeView((viewId) {
_localViewId = viewId;
}),
),
Positioned(
width: MediaQuery.of(context).size.width,
height: 300,
child: RenderViewPlugin.createNativeView((viewId) {
_remoteViewId = viewId;
}),
),
];
@override
Widget build(BuildContext context) {
return Scaffold(
body: Stack(
children: videoViews(),
),
floatingActionButton: RaisedButton(
child: Text("切换层级"),
onPressed: () {
if (_localViewId != null && _remoteViewId != null) {
if (mounted) {
setState(() {
views.insert(1, views.removeAt(0));
});
}
}
},
),
);
}
为了避免setState导致的NativeView被多次创建,设置给媒体SDK的view跟当前在页面上的view不是同一个导致黑屏, 因此我们将localWidget和remoteWidget给Cache住。
3.3 遇到的问题
以上代码代码测试视频流连麦没有问题了,试试全屏切换,localWidget和remoteWidget的层级和尺寸没有正常更新。 点击层级切换,我们期望的效果:
而实际我们看到的效果:
3.4 解决方案
以iOS为例,前面有提到Engine在创建UiKitView时会给UikitView添加一个父试图FlutterTouchInterceptingView用来处理点击事件,如果要切换两个View的层级可以这么来做。
[frontView.superview.superview insertSubview:frontView.superview aboveSubview:backView.superview];
起初我们也是这么做的。但发现UiKitView在渲染之前根据viewId创建一个FlutterOverlayView的实例,这个overlay会覆盖在UiKitView上,FlutterTouchInterceptingView有着共同的父试图,我们通过insertSubview切换了UiKitView,那么FlutterOverlayView也应该切换才对。FlutterOverlayView这个类是Engine私有并没有对外公开,虽然我们可以通过一些手段拿到FlutterOverlayView但是成本太高,再者这么做不太“Flutter”!
在Flutter中如果更新了Widget树,我们需要调用setState去触发Element和RenderObject树的更新,从而达我们期望的UI效果。
树的更新规则:
1)找到Widget对应的element节点,设置element为dirty,触发drawframe, drawframe会调用element的 performRebuild()进行树重建
2)widget.build() == null, deactive element.child,删除子树,流程结束
3)element.child.widget == NULL, mount 的新子树,流程结束
4)element.child.widget == widget.build() 无需重建,否则进入流程5
5)Widget.canUpdate(element.child.widget, newWidget) == true,更新child的slot,element.child.update(newWidget)(如果child还有子节点,则递归上面的流程进行子树更新),流程结束,否则转6
6)Widget.canUpdate(element.child.widget, newWidget) != true(widget的classtype 或者 key 不相等),deactivew element.child,mount 新子树
核心方法在于canUpdate(element.child.widget, newWidget)当我们没有给Widget任何key的时候,将会只比较这两个Widget的runtimeType 。这里两个Widget的runtimeType均为我们注册PlatformView的viewType,canUpdate 方法将会返回true,于是更新StatefulWidget的位置,这两个Element将不会交换位置。但是原有 Element 只会从它持有的state实例中build新的widget。因为element没变,它持有的state也没变, 因此就出现了上面的UI异常。
给localWidget和remoteWidgte分别加一个UniqueKey之后,canUpdate方法将会比较两个Widget的runtimeType 以及 key。并返回false。此时 RenderObjectElement 会用新 Widget的key在老Element列表里面查找,找到匹配的则会更新Element的位置并更新对应RenderObject的位置,因此就能更新成功了。
class _LiveViewStackState extends State<LiveViewStack> {
····
List<Widget> views = [
Positioned(
key: UniqueKey(),
width: 200,
height: MediaQuery.of(context).size.height,
child: RenderViewPlugin.createNativeView((viewId) {
_localViewId = viewId;
}),
),
Positioned(
key: UniqueKey(),
width: MediaQuery.of(context).size.width,
height: 300,
child: RenderViewPlugin.createNativeView((viewId) {
_remoteViewId = viewId;
}),
),
];
····
4.总结
PlatformView的使用还是很简单的,在解决PlatformView上的内存问题后,从我们实际的性能测试数据来看,音视频的渲染性能表现基本是贴近原生。组内大佬这篇文章 手把手教你解决PlatformView内存泄漏 将PlatformView的内存泄漏以及Engine层对PlatfomrView的layout 、paint剖析的已经很清晰了,我就不做过多的赘述,这里对比下某应用进出直播间的内存测试数据。
修复前:
修复后:
后面有机会尝试Texture Widget的话,会对比下PlatformView的实际性能。
祝大家玩的开心!