iOS webrtc在P2P场景下实现系统级屏幕共享（flutter版本）

白话666
837 阅读3分钟

背景

用flutter webrtc实现了屏幕共享功能，但是在iOS上，默认只能共享当前APP的屏幕，需要自己使用Broadcast Extension扩展才能实现系统级屏幕共享。

实现方式

以下内容来源：github.com/flutter-web…

iOS Screen Sharing allows you to publish your screen as a video track. This document explains how to setup screen sharing on iOS.

Broadcasting Modes

iOS supports two broadcasting modes:

  • In-App screen recording: Publishes the contents of your app only.
  • Broadcast Extension screen recording: Publishes the contents of your screen, on any app.

Screen sharing is started by calling navigator.mediaDevices.getDisplayMedia(Map<String, dynamic> constraints). By default, in-app screen recording is used. No extra setup is needed for in-app screen recording.

Broadcast extension screen recording can be accessed by setting the 'deviceId' key to 'broadcast' in constraints. Broadcast extension screen sharing requires extra setup in your iOS project.

Broadcast Extension Quick Setup

  1. Add a new Broadcast Upload Extension, without UI, to your project in XCode. Update the deployment info for your app and broadcast extension to run in iOS 14 or newer.
  2. Add your app and broadcast extension to the same app group.
  3. Add the app group id value to the app's Info.plist for the RTCAppGroupIdentifier key.
  4. Copy SampleHandler.swift, SampleUploader.swift, SocketConnection.swift, DarwinNotificationCenter.swift and Atomic.swift files from this sample project to your extension. Make sure they are added to the extension's target.
  5. Update appGroupIdentifier in `SampleHandler to return your app group identifier.

Add a new Broadcast Extension

Add a Broadcast Upload Extension through File -> New -> Target.

Fill out the required information for your extension and click Finish.

Screen Shot 2022-06-10 at 9 57 09 PM

Ensure your app and broadcast extension's deployment info is set to iOS 14 or newer.

Setup app groups

Add your extension to an app group by going to your extension's target in the project; in the Signings & Capabilities tab, click the + button in the top left and add App Groups. If you haven't done so already, add App Groups to your main app as well, ensuring that the App Group identifier is the same for both.

Setup SampleHandler

  • Copy SampleHandler.swift, SampleUploader.swift, SocketConnection.swift, DarwinNotificationCenter.swift and Atomic.swift files from this sample project to your extension. You may already have a SampleHandler.swift auto-generated by XCode; just overwrite those contents.
  • Update appGroupIdentifier in SampleHandler to return the app group identifier for your extension.

Your broadcast extension is now ready for broadcast through getDisplayMedia.

其它

参数修改

appGroupIdentifier 要改成自己的groupId

调用方式

'deviceId': 'broadcast'


final Map<String, dynamic> mediaConstraints = {
  'audio':  false,
  'video': {
    'mandatory': {
      'minWidth': '720',
      'minHeight': '1280',
      'maxWidth': '1080',
      'maxHeight': '1920',
      'minFrameRate': '15',
    },
    'facingMode': 'user',
    'optional': [],
    'deviceId': 'broadcast',
  }
};


MediaStream stream = await navigator.mediaDevices.getDisplayMedia(mediaConstraints);

后台运行

因为iOS会限制后台程序的网络、文件读写功能等，所以即使是实现了以上功能，也会出现后台屏幕共享运行一段时间后，屏幕共享功能暂停，此时如果手动回到自己的APP，屏幕共享又会恢复 解决办法： github.com/flutter-web…

使用callkit功能，保持后台运行

  1. 在flutter的依赖中增加依赖
flutter_callkit_incoming: 2.0.4+1

APP 能力增加后台运行功能

  • Info.plist
<key>UIBackgroundModes</key>
<array>
    <string>voip</string>
    <string>remote-notification</string>
    <string>processing</string> //you can add this if needed
</array>

2. 实现callkit调用

import 'dart:async';
import 'dart:io';

import 'package:flutter_callkit_incoming/entities/call_event.dart';
import 'package:flutter_callkit_incoming/entities/call_kit_params.dart';
import 'package:flutter_callkit_incoming/entities/ios_params.dart';
import 'package:flutter_callkit_incoming/flutter_callkit_incoming.dart';
import 'package:flutter_remote/utils/toast.dart';
import 'package:uuid/uuid.dart';

class CallKit {
  static Event? _eventAction;
  static String? currentUuid;
  static StreamSubscription? callListener;

  static Future<bool> startOutGoingCall() async {
    await stopOutGoingCall();
    if (Platform.isIOS) {
      try {
        // Generate a UUID
        String uuid = const Uuid().v4().toString();
        // Ready Params
        CallKitParams params = CallKitParams(
            id: uuid,
            nameCaller: "screen_share",
            handle: '0123456789',
            type: 1,
            extra: <String, dynamic>{'userId': 'screen_share'},
            ios: const IOSParams(
                iconName: 'images/app_icon.png', // ios/Runner/Assets.xcassets/Logo.imageset
                handleType: 'generic',
                audioSessionActive: false,
                configureAudioSession: false));

        Completer<bool> completer = Completer();
        callListener = FlutterCallkitIncoming.onEvent.listen((event) {
          _eventAction = event?.event;
          switch (event!.event) {
            case Event.actionCallStart:
              print("call started");
              completer.complete(true);
              break;
            case Event.actionCallEnded:
              print("call Stopped");
              break;
            default:
              break;
          }
        });

        await FlutterCallkitIncoming.startCall(params);
        currentUuid = uuid;

        // setState(() {
        //   currentUuid = uuid;
        // });
        return completer.future;
      } catch (e) {
        showToast("Something went wrong! Please start again");
      }
    }
    return false;
  }

  static Future<void> stopOutGoingCall() async {
    if (currentUuid != null && Platform.isIOS) {
      try {
        await FlutterCallkitIncoming.endCall(currentUuid!);
        // setState(() {
          currentUuid = null;
        // });
        await callListener?.cancel();
        _eventAction = null;
      } catch (e) {
        print("Error while stop outgoing call: $e");
      }
    }
  }
}

3. 调用 使用系统屏幕需要手动操作开始录屏，可以根据需求决策是否需要

Future<bool> _startWebRtcPushScreen(bool systemScreen) async {
  if (systemScreen && Platform.isIOS) {
    bool result = await CallKit.startOutGoingCall();
    if (!result) {
      return false;
    }
    //会影响投屏的系统选择界面，先延时处理
    Future.delayed(const Duration(milliseconds: 1000), () async {
      _startWebRtcPushScreenInvite(systemScreen);
    });
  }
  else {
    return _startWebRtcPushScreenInvite(systemScreen);
  }
  return true;
}

审核

  1. 有可能在上传ipa包时，遇到静态代码分析的问题，解决办法：
  • info.plist
<key>BGTaskSchedulerPermittedIdentifiers</key>
<array>
    <string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
</array>
  1. 人工审核
  • 由于使用了callkit，但是callkit是被中国工信部禁止使用的，所以会因此审核被拒，尝试通过邮件沟通解决，说明只在非中国地区使用了此功能。我自己的iOS应用，和审核人员反复沟通几次后，花了一周左右时间审核上线。
avatar
文章
阅读
粉丝