一. 系统集成第三方库

一般情况下，系统集成第三方so，提供方都会分别给出32bit和64bit的库。这样我们在集成的时候，只需要根据当前android版本的arch来自行选择集成哪个即可，以libbytelink.so举例

ifeq ($(TARGET_IS_64_BIT),true)
    LOCAL_PREBUILT_LIBS +=libbytelink:libs/arm64-v8a/libbytelink.so
else
    LOCAL_PREBUILT_LIBS +=libbytelink:libs/armeabi-v7a/libbytelink.so
endif
LOCAL_MODULE_TAGS := optional
include $(BUILD_MULTI_PREBUILT)

首先需要在系统app的根目录，新建一个libs目录，然后在这基础上分别创建armeabi-v7a和arm64-v8a分别存放对应的32bit so和64bit so 这里armeabi-v7a和arm64-v8a代表android平台支持的CPU架构 armeabi-v7a: 第七代以上的arm处理器, 支持浮点运算 arm64-v8a: 第八代64bit arm处理器
针对so文件，在引用之前，需要预编译，在对应目录的mk中，通过TARGET_IS_64_BIT来判断当前Android系统的32bit的还是64bit的，然后分别用LOCAL_PREBUILT_LIBS来拷贝上面已经存放到对应目录下的so文件 TARGET_IS_64_BIT是在android的make编译的board config中设定好的，它会根据当前cpu的架构来指定上面的值为true还是false，以高通865平台为例，这个值为true

ifneq ($(filter %64,$(TARGET_ARCH)),)
  TARGET_IS_64_BIT := true
endif

然后在build对应的package的时候，通过下面的方法引入即可

LOCAL_JNI_SHARED_LIBRARIES := libbytelink
LOCAL_REQUIRED_MODULES := libbytelink
include $(BUILD_PACKAGE)

二. 系统集成第三方32bit库

如果因为某些原因，供应商暂时只能提供32bit的so，这种情况下，我们该如何去在64bit android系统下去集成这个它

如果我们还是按照上面的办法，只在armeabi-v7a存放供应商给的so，然后正常去预编译此时，android系统的去loadLibrary的时候就会报如下错误:

05-08 10:43:29.260  7747  7747 E AndroidRuntime: java.lang.UnsatisfiedLinkError: dalvik.system.PathClassLoader[DexPathList[[zip file "/system/priv-app/ScreenMirrorServer/ScreenMirrorServer.apk"],nativeLibraryDirectories=[/system/priv-app/ScreenMirrorServer/lib/arm, /system/priv-app/ScreenMirrorServer/ScreenMirrorServer.apk!/lib/armeabi-v7a, /system/lib, /system/product/lib, /system/lib, /system/product/lib]nativeLibraryFindDir=[directory "/system/priv-app/ScreenMirrorServer/lib/arm", zip file "/system/priv-app/ScreenMirrorServer/ScreenMirrorServer.apk", dir "lib/armeabi-v7a", directory "/system/lib", directory "/system/product/lib", directory "/system/lib", directory "/system/product/lib", directory "/apex/com.android.runtime/lib", directory "/vendor/lib"]]] couldn't find "libbytelink.so"
05-08 10:43:29.260  7747  7747 E AndroidRuntime:         at java.lang.Runtime.loadLibrary0(Runtime.java:1087)
05-08 10:43:29.260  7747  7747 E AndroidRuntime:         at java.lang.Runtime.loadLibrary0(Runtime.java:1027)
05-08 10:43:29.260  7747  7747 E AndroidRuntime:         at java.lang.System.loadLibrary(System.java:1691)

三. Android系统加载so的机制

从上文的报错可以看出，是在loadLibrary的时候找不到libbytelink.so这个库

couldn't find "libbytelink.so"

为什么会报这个错误，Android系统是如何加载so的呢，我们这里来分析一下

1. 系统如何loadLibrary

libcore/ojluni/src/main/java/java/lang/System.java

@CallerSensitive
public static void loadLibrary(String libname) {
    Runtime.getRuntime().loadLibrary0(Reflection.getCallerClass(), libname);
}

libcore/ojluni/src/main/java/java/lang/Runtime.java

if (loader != null && !(loader instanceof BootClassLoader)) {
    String filename = loader.findLibrary(libraryName);
    if (filename == null) {
        // It's not necessarily true that the ClassLoader used
        // System.mapLibraryName, but the default setup does, and it's
        // misleading to say we didn't find "libMyLibrary.so" when we
        // actually searched for "liblibMyLibrary.so.so".
        throw new UnsatisfiedLinkError(loader + " couldn't find \"" +
                                       System.mapLibraryName(libraryName) + "\"");
    }

从上面可以看到，报错是走到了UnsatisfiedLinkError异常，进一步看下，findLibrary是如何查找的，这里的loader是ClassLoader对象，findLibrary的实现则在它的子类BaseDexClassLoader.java

libcore/dalvik/src/main/java/dalvik/system/BaseDexClassLoader.java

@Override
public String findLibrary(String name) {
    return pathList.findLibrary(name);
}

然后这里进入到DexPathList的findLibrary里面

libcore/dalvik/src/main/java/dalvik/system/DexPathList.java

public String findLibrary(String libraryName) {
    String fileName = System.mapLibraryName(libraryName);

    for (NativeLibraryElement element : nativeLibraryPathElements) {
        String path = element.findNativeLibrary(fileName);

        if (path != null) {
            return path;
        }
    }

    return null;
}

首先我们这里看一下，mapLibraryName做了什么操作，这是一个native调用。

实际上结合给的libname，加上一个prexfix和一个suffix，将文件名称整合称我们最终看到的样子如例子中我们的filename是bytelink，通过这个方法，最终得到的我们实际要在路径中要寻找的so名称即为libbytelink.so

libcore/ojluni/src/main/native/System.c

System_mapLibraryName(JNIEnv *env, jclass ign, jstring libname)
{
    int len;
    // #define JNI_LIB_PREFIX "lib"
    int prefix_len = (int) strlen(JNI_LIB_PREFIX);
    // #define JNI_LIB_SUFFIX ".so"
    int suffix_len = (int) strlen(JNI_LIB_SUFFIX);

    jchar chars[256];
    if (libname == NULL) {
        JNU_ThrowNullPointerException(env, 0);
        return NULL;
    }
    len = (*env)->GetStringLength(env, libname);
    if (len > 240) {
        JNU_ThrowIllegalArgumentException(env, "name too long");
        return NULL;
    }
    cpchars(chars, JNI_LIB_PREFIX, prefix_len);
    (*env)->GetStringRegion(env, libname, 0, len, chars + prefix_len);
    len += prefix_len;
    cpchars(chars + len, JNI_LIB_SUFFIX, suffix_len);
    len += suffix_len;

    return (*env)->NewString(env, chars, len);

接着回到上面的findLibrary方法，可以看到是遍历nativeLibraryPathElements这个数组来查找对应的路径，那么接下来就去看一下nativeLibraryPathElements这个数组是怎么被赋值的

libcore/dalvik/src/main/java/dalvik/system/DexPathList.java

// Native libraries may exist in both the system and
// application library paths, and we use this search order:
//
//   1. This class loader's library path for application libraries (librarySearchPath):
//   1.1. Native library directories
//   1.2. Path to libraries in apk-files
//   2. The VM's library path from the system property for system libraries
//      also known as java.library.path
//
// This order was reversed prior to Gingerbread; see http://b/2933456.
this.nativeLibraryDirectories = splitPaths(librarySearchPath, false);
this.systemNativeLibraryDirectories =
        splitPaths(System.getProperty("java.library.path"), true);
this.nativeLibraryPathElements = makePathElements(getAllNativeLibraryDirectories());

libcore/dalvik/src/main/java/dalvik/system/DexPathList.java

private List<File> getAllNativeLibraryDirectories() {
    List<File> allNativeLibraryDirectories = new ArrayList<>(nativeLibraryDirectories);
    allNativeLibraryDirectories.addAll(systemNativeLibraryDirectories);
    return allNativeLibraryDirectories;
}

从上面的代码可以看到，nativeLibraryPathElements是nativeLibraryDirectories和systemNativeLibraryDirectories两个List的总和这里因为nativeLibraryDirectories被优先加入到list里面，所以后续查找也会优先从app对应的目录下开始查找

从上面的注释中可以看到，nativeLibraryDirectories是application的library path，顾名思义，就是在app的根目录下的去寻找，这里重点关注下systemNativeLibraryDirectories这个是如何查找的

这里主要是通过System.getProperty("java.library.path")这个properties，找了一圈没找到直接setProp的地方，所以我们换个方向去找，很明显，像system library path这种肯定是不能被改变的，所以我们看下System这个类当中的静态代码块中，initUnchangeableSystemProperties这个方法

libcore/ojluni/src/main/java/java/lang/System.java

// Android-added: Undocumented properties that exist only on Android.
p.put("android.icu.library.version", ICU.getIcuVersion());
p.put("android.icu.unicode.version", ICU.getUnicodeVersion());
p.put("android.icu.cldr.version", ICU.getCldrVersion());

// Property override for ICU4J : this is the location of the ICU4C data. This
// is prioritized over the properties in ICUConfig.properties. The issue with using
// that is that it doesn't play well with jarjar and it needs complicated build rules
// to change its default value.
String icuDataPath = TimeZoneDataFiles.generateIcuDataPath();
p.put("android.icu.impl.ICUBinary.dataPath", icuDataPath);

parsePropertyAssignments(p, specialProperties());

这个方法里，会put一堆不可被改变的system prop，没有找到我们需要的"java.library.path"，但我们看到这里有一个specialProperties()的方法，上面put的所有prop都会通过这个通过这个作为参数来解析，作为一个native function，看一下它内部的实现

libcore/ojluni/src/main/native/System.c

 const char* library_path = getenv("LD_LIBRARY_PATH");
#if defined(__ANDROID__)
    if (library_path == NULL) {
        android_get_LD_LIBRARY_PATH(path, sizeof(path));
        library_path = path;
    }
#endif
    if (library_path == NULL) {
        library_path = "";
    }
    char* java_path = malloc(strlen("java.library.path=") + strlen(library_path) + 1);
    strcpy(java_path, "java.library.path=");
    strcat(java_path, library_path);

如果library_path为null，则通过 android_get_LD_LIBRARY_PATH(path, sizeof(path))去获取，然后赋值给library_path，然后用一个java_path来拼接，返回给java层，所以关键是path的值是如何get的，看下android_get_LD_LIBRARY_PATH的实现

bionic/linker/linker.cpp

void do_android_get_LD_LIBRARY_PATH(char* buffer, size_t buffer_size) {
  const auto& default_ld_paths = g_default_namespace.get_default_library_paths();

  size_t required_size = 0;
  for (const auto& path : default_ld_paths) {
    required_size += path.size() + 1;
  }

  if (buffer_size < required_size) {
    async_safe_fatal("android_get_LD_LIBRARY_PATH failed, buffer too small: "
                     "buffer len %zu, required len %zu", buffer_size, required_size);
  }

  char* end = buffer;
  for (size_t i = 0; i < default_ld_paths.size(); ++i) {
    if (i > 0) *end++ = ':';
    end = stpcpy(end, default_ld_paths[i].c_str());
  }
}

这里通过层层的调用，最终是走到了do_android_get_LD_LIBRARY_PATH这个方法，可以看到buffer最后赋值给end，然后从default_ld_paths中去遍历取值，所以看下g_default_namespace.get_default_library_paths()

bionic/linker/linker_namespaces.h

const std::vector<std::string>& get_default_library_paths() const {
  return default_library_paths_;
}
void set_default_library_paths(std::vector<std::string>&& library_paths) {
  default_library_paths_ = std::move(library_paths);
}
void set_default_library_paths(const std::vector<std::string>& library_paths) {
  default_library_paths_ = library_paths;
}

这里是一个get set方法，我们看下set_default_library_paths在哪里被调用

bionic/linker/linker.cpp

static std::vector<android_namespace_t*> init_default_namespace_no_config(bool is_asan) {
  g_default_namespace.set_isolated(false);
  auto default_ld_paths = is_asan ? kAsanDefaultLdPaths : kDefaultLdPaths;

  char real_path[PATH_MAX];
  std::vector<std::string> ld_default_paths;
  for (size_t i = 0; default_ld_paths[i] != nullptr; ++i) {
    if (realpath(default_ld_paths[i], real_path) != nullptr) {
      ld_default_paths.push_back(real_path);
    } else {
      ld_default_paths.push_back(default_ld_paths[i]);
    }
  }

  g_default_namespace.set_default_library_paths(std::move(ld_default_paths));

如上，在init_default_namespaces中，会去调用init_default_namespace_no_config方法，这个里面可以看到如果支持asan(是一种基于编译器的快速检测工具，用于检测原生代码中的内存错误,android支持普通Asan和硬件加速的Asan)，所以default_ld_paths = kAsanDefaultLdPaths

bionic/linker/linker.cpp

static const char* const kAsanDefaultLdPaths[] = {
  kAsanSystemLibDir,
  kSystemLibDir,
  kAsanOdmLibDir,
  kOdmLibDir,
  kAsanVendorLibDir,
  kVendorLibDir,
  nullptr
};

bionic/linker/linker.cpp

#if defined(__LP64__)
static const char* const kSystemLibDir        = "/system/lib64";
static const char* const kOdmLibDir           = "/odm/lib64";
static const char* const kVendorLibDir        = "/vendor/lib64";
#else
static const char* const kSystemLibDir        = "/system/lib";
static const char* const kOdmLibDir           = "/odm/lib";
static const char* const kVendorLibDir        = "/vendor/lib";
#endif

看到这里，我们已经清楚了，DexPathList中findLibrary会根据加上前缀和后缀的filename，优先从app对应的目录下去查找，如果没找到，则去系统lib目录查找，对应android64 bit平台，则是/system/lib64/，/vendor/lib64/

我们再回到一开始的报错，Android10平台现在编译出来的都是64bit的app，为什么从nativeLibrary去找，打印出来的路径都是32bit的呢，如/system/lib/, /vendor/lib/, 我们就继续分析一下，系统是如何选择app是32bit还是64bit进程的

2. 系统如何判断app应该运行32bit还是64bit进程

大家都知道，当zygote folk出SystemServer进程之后，zygoteServer会起一个死循环等待AMS发送的创建app进程的请求

frameworks/base/core/java/com/android/internal/os/ZygoteInit.java

if (startSystemServer) {
    Runnable r = forkSystemServer(abiList, zygoteSocketName, zygoteServer);

    // {@code r == null} in the parent (zygote) process, and {@code r != null} in the
    // child (system_server) process.
    if (r != null) {
        r.run();
        return;
    }
}

Log.i(TAG, "Accepting command socket connections");

// The select loop returns early in the child process after a fork and
// loops forever in the zygote.
caller = zygoteServer.runSelectLoop(abiList);

这里是SystemServer请求创建进程的方法，发现其中有一个参数是ABI override ABI，全称是Application binary interface，定义了一套规则,允许编译好的二进制目标代码能在所有兼容该ABI的操作系统中无需改动就能运行

目前android10上，对于ABI的支持如下:

darwin:/ # getprop | grep "ro.product.cpu.abilist"
[ro.product.cpu.abilist]: [arm64-v8a,armeabi-v7a,armeabi]
[ro.product.cpu.abilist32]: [armeabi-v7a,armeabi]
[ro.product.cpu.abilist64]: [arm64-v8a]

frameworks/base/services/core/java/com/android/server/am/ActivityManagerService.java

final ProcessRecord startProcessLocked(String processName,
        ApplicationInfo info, boolean knownToBeDead, int intentFlags,
        HostingRecord hostingRecord, boolean allowWhileBooting,
        boolean isolated, boolean keepIfLarge) {
    // Smt: {@
    if (processName != mLaunchingProcessName) {
        trimSystemMemoryIfNeeded(processName, info);
        mLaunchingProcessName = processName;
    }
    // @}
    return mProcessList.startProcessLocked(processName, info, knownToBeDead, intentFlags,
            hostingRecord, allowWhileBooting, isolated, 0 /* isolatedUid */, keepIfLarge,
            null /* ABI override */, null /* entryPoint */, null /* entryPointArgs */,
            null /* crashHandler */);
}

frameworks/base/services/core/java/com/android/server/am/ProcessList.java

String requiredAbi = (abiOverride != null) ? abiOverride : app.info.primaryCpuAbi;
if (requiredAbi == null) {
    requiredAbi = Build.SUPPORTED_ABIS[0];
}

继续往下走，在ProcessList中经过多次构造函数的重载调用，在startProcessLocked中，会根据传过来的abiOverride做判断，因为前面传递的为null，所以这里requiredAbi = app.info.primaryCpuAbi 看下app.info.primaryCpuAbi这值是从哪边传过来的

这个值在apk安装后，会记录在data/system/packages.xml这个文件里头，比如蓝牙app，这里就能看到primaryCpuAbi="arm64-v8a", 所以Bluetotoh.apk默认是64bit的

<package name="com.android.bluetooth" codePath="/system/app/Bluetooth" nativeLibraryPath="/system/app/Bluetooth/lib" primaryCpuAbi="arm64-v8a" publicFlags="541638213" privateFlags="0" ft="11e8f7d4c00"       it="11e8f7d4c00" ut="11e8f7d4c00" version="29" sharedUserId="1002" isOrphaned="true">

当Android系统启动以后，在PMS的构造函数里，会调用scanDirTracedLI，扫描所有的system app

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

// Collect ordinary system packages.
final File systemAppDir = new File(Environment.getRootDirectory(), "app");
scanDirTracedLI(systemAppDir,
        mDefParseFlags
        | PackageParser.PARSE_IS_SYSTEM_DIR,
        scanFlags
        | SCAN_AS_SYSTEM,
        0);

继续调用scanPackageChildLI

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

try {
    scanPackageChildLI(parseResult.pkg, parseFlags, scanFlags,
            currentTime, null);
} catch (PackageManagerException e) {
    errorCode = e.error;
    Slog.w(TAG, "Failed to scan " + parseResult.scanFile + ": " + e.getMessage());
}

最终会走到scanPackageOnlyLI

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

//如果system app不是首次启动或者升级
boolean needToDeriveAbi = (scanFlags & SCAN_FIRST_BOOT_OR_UPGRADE) != 0;

if (needToDeriveAbi) {
    Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "derivePackageAbi");
    final boolean extractNativeLibs = !pkg.isLibrary();
    derivePackageAbi(pkg, cpuAbiOverride, extractNativeLibs);
    Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);

    // Some system apps still use directory structure for native libraries
    // in which case we might end up not detecting abi solely based on apk
    // structure. Try to detect abi based on directory structure.
    if (isSystemApp(pkg) && !pkg.isUpdatedSystemApp() &&
            pkg.applicationInfo.primaryCpuAbi == null) {
        setBundledAppAbisAndRoots(pkg, pkgSetting);
        setNativeLibraryPaths(pkg, sAppLib32InstallDir);
    }

如果system app不是首次启动或者升级，就会调用derivePackageAbi

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

private static void derivePackageAbi(PackageParser.Package pkg, String cpuAbiOverride,
        boolean extractLibs)
                throws PackageManagerException {
    // Give ourselves some initial paths; we'll come back for another
    // pass once we've determined ABI below.
    setNativeLibraryPaths(pkg, sAppLib32InstallDir);

    // We shouldn't attempt to extract libs from system app when it was not updated.
    if (isSystemApp(pkg) && !pkg.isUpdatedSystemApp()) {
        extractLibs = false;
    }

    final String nativeLibraryRootStr = pkg.applicationInfo.nativeLibraryRootDir;
    final boolean useIsaSpecificSubdirs = pkg.applicationInfo.nativeLibraryRootRequiresIsa;

    NativeLibraryHelper.Handle handle = null;
    try {
        handle = NativeLibraryHelper.Handle.create(pkg);
        // TODO(multiArch): This can be null for apps that didn't go through the
        // usual installation process. We can calculate it again, like we
        // do during install time.
        //
        // TODO(multiArch): Why do we need to rescan ASEC apps again ? It seems totally
        // unnecessary.
        final File nativeLibraryRoot = new File(nativeLibraryRootStr);

        // Null out the abis so that they can be recalculated.
        pkg.applicationInfo.primaryCpuAbi = null;
        pkg.applicationInfo.secondaryCpuAbi = null;
        //如果app支持多种abi架构
        if (isMultiArch(pkg.applicationInfo)) {
            // Warn if we've set an abiOverride for multi-lib packages..
            // By definition, we need to copy both 32 and 64 bit libraries for
            // such packages.
            if (pkg.cpuAbiOverride != null
                    && !NativeLibraryHelper.CLEAR_ABI_OVERRIDE.equals(pkg.cpuAbiOverride)) {
                Slog.w(TAG, "Ignoring abiOverride for multi arch application.");
            }

            int abi32 = PackageManager.NO_NATIVE_LIBRARIES;
            int abi64 = PackageManager.NO_NATIVE_LIBRARIES;
            //如果通过getprop拿到的"ro.product.cpu.abilist32"的abi存在
            if (Build.SUPPORTED_32_BIT_ABIS.length > 0) {
                if (extractLibs) {
                    Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "copyNativeBinaries");
            //调用copyNativeBinariesForSupportedAbi查询nativeLibraryPath下面支持的abi，并进行so拷贝
                    abi32 = NativeLibraryHelper.copyNativeBinariesForSupportedAbi(handle,
                            nativeLibraryRoot, Build.SUPPORTED_32_BIT_ABIS,
                            useIsaSpecificSubdirs);
                } else {
                    Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "findSupportedAbi");
                    abi32 = NativeLibraryHelper.findSupportedAbi(handle, Build.SUPPORTED_32_BIT_ABIS);
                }
                Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);
            }

            // Shared library native code should be in the APK zip aligned
            if (abi32 >= 0 && pkg.isLibrary() && extractLibs) {
                throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                        "Shared library native lib extraction not supported");
            }

            maybeThrowExceptionForMultiArchCopy(
                    "Error unpackaging 32 bit native libs for multiarch app.", abi32);
            //如果通过getprop拿到的"ro.product.cpu.abilist64"的abi存在
            if (Build.SUPPORTED_64_BIT_ABIS.length > 0) {
                if (extractLibs) {
                    Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "copyNativeBinaries");
                    //调用copyNativeBinariesForSupportedAbi查询nativeLibraryPath下面支持的abi，并进行so拷贝
                    abi64 = NativeLibraryHelper.copyNativeBinariesForSupportedAbi(handle,
                            nativeLibraryRoot, Build.SUPPORTED_64_BIT_ABIS,
                            useIsaSpecificSubdirs);
                } else {
                    Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "findSupportedAbi");
                    abi64 = NativeLibraryHelper.findSupportedAbi(handle, Build.SUPPORTED_64_BIT_ABIS);
                }
                Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);
            }

            maybeThrowExceptionForMultiArchCopy(
                    "Error unpackaging 64 bit native libs for multiarch app.", abi64);

            if (abi64 >= 0) {
                // Shared library native libs should be in the APK zip aligned
                if (extractLibs && pkg.isLibrary()) {
                    throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                            "Shared library native lib extraction not supported");
                }
                //如果支持64位so，则primaryCpuAbi被赋值为prop中对应的abi属性
                pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[abi64];
            }

            if (abi32 >= 0) {
                final String abi = Build.SUPPORTED_32_BIT_ABIS[abi32];
                if (abi64 >= 0) {
                    if (pkg.use32bitAbi) {
                        //如果同时支持32bit和64bit的so，并且当前的app使用的是32bit so
                        //则将上面的64abi作为secondaryCpuAbi，32abi作为首要的CpuAbi
                        pkg.applicationInfo.secondaryCpuAbi = pkg.applicationInfo.primaryCpuAbi;
                        pkg.applicationInfo.primaryCpuAbi = abi;
                    } else {
                        pkg.applicationInfo.secondaryCpuAbi = abi;
                    }
                } else {
                    pkg.applicationInfo.primaryCpuAbi = abi;
                }
            }
        } else {
            //如果app不支持多种abi架构，adbList被赋值为当前系统支持的ABIS
            String[] abiList = (cpuAbiOverride != null) ?
                    new String[] { cpuAbiOverride } : Build.SUPPORTED_ABIS;

            // Enable gross and lame hacks for apps that are built with old
            // SDK tools. We must scan their APKs for renderscript bitcode and
            // not launch them if it's present. Don't bother checking on devices
            // that don't have 64 bit support.
            boolean needsRenderScriptOverride = false;
            if (Build.SUPPORTED_64_BIT_ABIS.length > 0 && cpuAbiOverride == null &&
                    NativeLibraryHelper.hasRenderscriptBitcode(handle)) {
                abiList = Build.SUPPORTED_32_BIT_ABIS;
                needsRenderScriptOverride = true;
            }

            final int copyRet;
            if (extractLibs) {
                Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "copyNativeBinaries");
                copyRet = NativeLibraryHelper.copyNativeBinariesForSupportedAbi(handle,
                        nativeLibraryRoot, abiList, useIsaSpecificSubdirs);
            } else {
                Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "findSupportedAbi");
                copyRet = NativeLibraryHelper.findSupportedAbi(handle, abiList);
            }
            Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);

            if (copyRet < 0 && copyRet != PackageManager.NO_NATIVE_LIBRARIES) {
                throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                        "Error unpackaging native libs for app, errorCode=" + copyRet);
            }

            if (copyRet >= 0) {
                // Shared libraries that have native libs must be multi-architecture
                if (pkg.isLibrary()) {
                    throw new PackageManagerException(INSTALL_FAILED_INTERNAL_ERROR,
                            "Shared library with native libs must be multiarch");
                }
                //如果nativeLib下面存在abi，则将对应的abi赋值
                pkg.applicationInfo.primaryCpuAbi = abiList[copyRet];
            } else if (copyRet == PackageManager.NO_NATIVE_LIBRARIES && cpuAbiOverride != null) {
                //如果没有native lib，但是cpuAbiOverride不为null，则赋值cpuAbiOverride
                pkg.applicationInfo.primaryCpuAbi = cpuAbiOverride;
            } else if (needsRenderScriptOverride) {
                //否则将abi list中第一项赋值
                pkg.applicationInfo.primaryCpuAbi = abiList[0];
            }
        }
    } catch (IOException ioe) {
        Slog.e(TAG, "Unable to get canonical file " + ioe.toString());
    } finally {
        IoUtils.closeQuietly(handle);
    }

    // Now that we've calculated the ABIs and determined if it's an internal app,
    // we will go ahead and populate the nativeLibraryPath.
    setNativeLibraryPaths(pkg, sAppLib32InstallDir);
}

通过上面的代码可以看到，derivePackageAbi规则如下:

如果app支持多abi架构，如果支持64bit so，则将对应的abi赋值给primaryCpuAbi，如果只支持32bit so，则将对应的abi赋值给primaryCpuAbi，如果同时支持64bit so和32bit so，并且当前使用的是32bit so，则将primaryCpuAbi赋值为32abi，64abi赋值给secondaryCpuAbi
如果app不支持多abi架构，会去当前的nativeLibraryPath下面去查询当前的so的abi，如果abi存在，则取abiList中对应的abi赋值给primaryCpuAbi，如果不存在，并且cpuAbiOverride不为null，则将cpuAbiOverride赋值给primaryCpuAbi，否则primaryCpuAbi为abiList中第一个存在的abi

如果是system app，并且通过上面的derivePackageAbi方法没有拿到对应的primaryCpuAbi，这时候，PMS会去调用setBundledAppAbisAndRoots方法

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

private static void setBundledAppAbisAndRoots(PackageParser.Package pkg,
                                       PackageSetting pkgSetting) {
    final String apkName = deriveCodePathName(pkg.applicationInfo.getCodePath());

    // If "/system/lib64/apkname" exists, assume that is the per-package
    // native library directory to use; otherwise use "/system/lib/apkname".
    final String apkRoot = calculateBundledApkRoot(pkg.applicationInfo.sourceDir);
    setBundledAppAbi(pkg, apkRoot, apkName);
    // pkgSetting might be null during rescan following uninstall of updates
    // to a bundled app, so accommodate that possibility.  The settings in
    // that case will be established later from the parsed package.
    //
    // If the settings aren't null, sync them up with what we've just derived.
    // note that apkRoot isn't stored in the package settings.
    if (pkgSetting != null) {
        pkgSetting.primaryCpuAbiString = pkg.applicationInfo.primaryCpuAbi;
        pkgSetting.secondaryCpuAbiString = pkg.applicationInfo.secondaryCpuAbi;
    }
}

frameworks/base/services/core/java/com/android/server/pm/PackageManagerService.java

private static void setBundledAppAbi(PackageParser.Package pkg, String apkRoot, String apkName) {
    final File codeFile = new File(pkg.codePath);

    final boolean has64BitLibs;
    final boolean has32BitLibs;
    if (isApkFile(codeFile)) {
        // Monolithic install
        // 分别检查32bit和64bit的lib是否存在于当前的apk所在目录
        has64BitLibs = (new File(apkRoot, new File(LIB64_DIR_NAME, apkName).getPath())).exists();
        has32BitLibs = (new File(apkRoot, new File(LIB_DIR_NAME, apkName).getPath())).exists();
    } else {
        // Cluster install
        final File rootDir = new File(codeFile, LIB_DIR_NAME);
        if (!ArrayUtils.isEmpty(Build.SUPPORTED_64_BIT_ABIS)
                && !TextUtils.isEmpty(Build.SUPPORTED_64_BIT_ABIS[0])) {
            final String isa = VMRuntime.getInstructionSet(Build.SUPPORTED_64_BIT_ABIS[0]);
            has64BitLibs = (new File(rootDir, isa)).exists();
        } else {
            has64BitLibs = false;
        }
        if (!ArrayUtils.isEmpty(Build.SUPPORTED_32_BIT_ABIS)
                && !TextUtils.isEmpty(Build.SUPPORTED_32_BIT_ABIS[0])) {
            final String isa = VMRuntime.getInstructionSet(Build.SUPPORTED_32_BIT_ABIS[0]);
            has32BitLibs = (new File(rootDir, isa)).exists();
        } else {
            has32BitLibs = false;
        }
    }

    if (has64BitLibs && !has32BitLibs) {
        // The package has 64 bit libs, but not 32 bit libs. Its primary
        // ABI should be 64 bit. We can safely assume here that the bundled
        // native libraries correspond to the most preferred ABI in the list.
       
        // 如果64bit lib支持，32bit lib不支持，则primaryCpuAbi为当前系统支持的64bit lib list的第一位
        pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
        pkg.applicationInfo.secondaryCpuAbi = null;
    } else if (has32BitLibs && !has64BitLibs) {
        // The package has 32 bit libs but not 64 bit libs. Its primary
        // ABI should be 32 bit.

        // 如果32bit lib支持而64bit lib不支持，则primaryCpuAbi为当前系统支持的32bit lib list的第一位
        pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
        pkg.applicationInfo.secondaryCpuAbi = null;
        // 如果32bit lib和64bit lib同时支持
    } else if (has32BitLibs && has64BitLibs) {
        // The application has both 64 and 32 bit bundled libraries. We check
        // here that the app declares multiArch support, and warn if it doesn't.
        //
        // We will be lenient here and record both ABIs. The primary will be the
        // ABI that's higher on the list, i.e, a device that's configured to prefer
        // 64 bit apps will see a 64 bit primary ABI,

        if ((pkg.applicationInfo.flags & ApplicationInfo.FLAG_MULTIARCH) == 0) {
            Slog.e(TAG, "Package " + pkg + " has multiple bundled libs, but is not multiarch.");
        }

        if (VMRuntime.is64BitInstructionSet(getPreferredInstructionSet())) {
            //如果当前虚拟机的运行环境是64bit,则primaryCpuAbi为当前系统支持的64bit lib list的第一位abi, secondaryCpuAbi为当前系统支持的32bit lib list的第一位
            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
        } else {
            //如果当前虚拟机的运行环境是32bit，则primaryCpuAbi为大年系统支持IDE32bit lib list的第一位abi，secondaryCpuAbi为当前系统支持的64bit lib list的第一位
            pkg.applicationInfo.primaryCpuAbi = Build.SUPPORTED_32_BIT_ABIS[0];
            pkg.applicationInfo.secondaryCpuAbi = Build.SUPPORTED_64_BIT_ABIS[0];
        }
    } else {
        pkg.applicationInfo.primaryCpuAbi = null;
        pkg.applicationInfo.secondaryCpuAbi = null;
    }
}

通过上面的代码分析得到，setBundledAppAbi的规则如下:

如果系统app支持64bit lib，但不支持32bit的lib，则primaryCpuAbi为当前系统支持的64bit abi list的第一位
如果系统app不支持64bit lib，但支持32bit的lib，则primaryCpuAbi为当前系统支持的32bit abi list的第一位
如果系统app同时支持64bit和32bit的lib，这时候需要判断当前虚拟机的运行环境，如果是64bit的，则 primaryCpuAbi为当前系统支持的64bit abi list的第一位，secondaryCpuAbi为当前系统支持的32bit abi list的第一位，如果是32bit的，则primaryCpuAbi为当前系统支持的32bit abi list的第一位，secondaryCpuAbi为当前系统支持的64bit abi list的第一位，

到了这里，我们大概捋清楚了app进程在启动是如何运行在32bit还是64bit的进程里了

回到开始我们遇到的问题，如果供应商只能提供32bit的so，那么我们该如何集成呢

虽然64bit的app向下兼容32bit，但是系统在加载so的时候却不能同时加载32bit和64bit的so，如果app需要加载的全部so为32bit，则使用32bit的方式加载so，如果需要加载的so中有一个是64bit的so，则必须使用64bit的方式加载so，所以综合上面的分析，我们只能选择让app运行在32的进程下

include $(CLEAR_VARS)
LOCAL_MODULE_TAGS := optional
LOCAL_MODULE_CLASS := SHARED_LIBRARIES
LOCAL_MODULE_SUFFIX := .so
LOCAL_MODULE := libbytelink
LOCAL_SRC_FILES := libs/armeabi-v7a/libbytelink.so
LOCAL_32_BIT_ONLY := true
include $(BUILD_PREBUILT)

LOCAL_32_BIT_ONLY := true

//将混淆给关闭
LOCAL_PROGUARD_ENABLED:= disabled

LOCAL_JNI_SHARED_LIBRARIES := libbytelink

LOCAL_REQUIRED_MODULES := libbytelink

include $(BUILD_PACKAGE)

我们在app的Android.mk中，首先对于预编译的so，需要将对应的so文件放在libs/armeabi-v7a/目录下面，然后加入LOCAL_32_BIT_ONLY := true，这会保证so将以32bit的形式编译同时对于最终app，我们也需要指定32bit编译

darwin:/ # ps -A | grep zygote
root           730     1 5509808 185960 poll_schedule_timeout 0 S zygote64
root           731     1 1821052 166792 poll_schedule_timeout 0 S zygote
webview_zygote 2753  731 1822560  85128 poll_schedule_timeout 0 S webview_zygote

darwin:/ # ps -A | grep screenmirrorclient                                                                                                  
system        3655   731 1418008  98028 ep_poll             0 S com.bytedance.screenmirrorclient

可以看到，现在com.bytedance.screenmirrorclient进程是从731即zygote folk出来的，至此，app就可以正常加载使用32bit 的第三方so提供的功能。

安卓操作系统加载第三方so问题分析

一. 系统集成第三方库

二. 系统集成第三方32bit库

三. Android系统加载so的机制

1. 系统如何loadLibrary

2. 系统如何判断app应该运行32bit还是64bit进程