记一次线程死锁导致的anr问题分析

问题描述：连接蓝牙鼠标后，关闭蓝牙，再重新开启蓝牙，会出现系统界面没有响应提示

分析tracelog：

"main" prio=5 tid=1 Blocked

group="main" sCount=1 dsCount=0 flags=1 obj=0x741943c8 self=0x7d154c2a00
sysTid=9311 nice=-10 cgrp=default sched=0/0 handle=0x7d9a4aa9a8
state=S schedstat=( 6930154951 867104714 7237 ) utm=596 stm=97 core=3 HZ=100
stack=0x7fd873f000-0x7fd8741000 stackSize=8MB
held mutexes=
at com.android.settingslib.bluetooth.CachedBluetoothDeviceManager.getCachedDevicesCopy(CachedBluetoothDeviceManager.java:-1)
waiting to lock <0x0957278e> (a com.android.settingslib.bluetooth.CachedBluetoothDeviceManager) held by thread 23
at com.android.systemui.statusbar.policy.BluetoothControllerImpl.getDevices(BluetoothControllerImpl.java:196)
at com.android.systemui.statusbar.policy.BluetoothControllerImpl.updateConnected(BluetoothControllerImpl.java:209)
at com.android.systemui.statusbar.policy.BluetoothControllerImpl.onDeviceAttributesChanged(BluetoothControllerImpl.java:267)
at com.android.settingslib.bluetooth.CachedBluetoothDevice.dispatchAttributesChanged(CachedBluetoothDevice.java:644)
locked <0x0dfd4baf> (a java.util.ArrayList)
at com.android.settingslib.bluetooth.CachedBluetoothDevice.refresh(CachedBluetoothDevice.java:451)
at com.android.settingslib.bluetooth.HidProfile$InputDeviceServiceListener.onServiceConnected(HidProfile.java:68)
at android.bluetooth.BluetoothInputDevice$2.onServiceConnected(BluetoothInputDevice.java:504)
at android.app.LoadedApk$ServiceDispatcher.doConnected(LoadedApk.java:1676)
at android.app.LoadedApk$ServiceDispatcher$RunConnection.run(LoadedApk.java:1705)
at android.os.Handler.handleCallback(Handler.java:794)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.TinnoLoop(Looper.java:189)
at android.os.Looper.loop(Looper.java:270)
at android.app.ActivityThread.main(ActivityThread.java:6730)
at java.lang.reflect.Method.invoke(Native method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

"SysUiBg" prio=5 tid=23 Blocked

group="main" sCount=1 dsCount=0 flags=1 obj=0x187017e0 self=0x7cfca3f400
sysTid=9342 nice=10 cgrp=default sched=0/0 handle=0x7cf6efa4f0
state=S schedstat=( 490875615 328350733 1413 ) utm=29 stm=20 core=5 HZ=100
stack=0x7cf6df8000-0x7cf6dfa000 stackSize=1037KB
held mutexes=
at com.android.settingslib.bluetooth.CachedBluetoothDevice.dispatchAttributesChanged(CachedBluetoothDevice.java:642)
waiting to lock <0x0dfd4baf> (a java.util.ArrayList) held by thread 1
at com.android.settingslib.bluetooth.CachedBluetoothDevice.onUuidChanged(CachedBluetoothDevice.java:575)
at com.android.settingslib.bluetooth.CachedBluetoothDeviceManager.onUuidChanged(CachedBluetoothDeviceManager.java:155)
locked <0x0957278e> (a com.android.settingslib.bluetooth.CachedBluetoothDeviceManager)
at com.android.settingslib.bluetooth.BluetoothEventManager$UuidChangedHandler.onReceive(BluetoothEventManager.java:366)
at com.android.settingslib.bluetooth.BluetoothEventManager$1.onReceive(BluetoothEventManager.java:148)
at android.app.LoadedApk$ReceiverDispatcher$Args.lambda$-android_app_LoadedApk$ReceiverDispatcher$Args_53020(LoadedApk.java:1337)
at android.app.-$Lambda$aS31cHIhRx41653CMnd4gZqshIQ.$m$7(unavailable:-1)
at android.app.-$Lambda$aS31cHIhRx41653CMnd4gZqshIQ.run(unavailable:-1)
at android.os.Handler.handleCallback(Handler.java:794)
at android.os.Handler.dispatchMessage(Handler.java:103)
at android.os.Looper.TinnoLoop(Looper.java:164)
at android.os.Looper.loop(Looper.java:270)
at android.os.HandlerThread.run(HandlerThread.java:65)

可以看到A线程waiting to lock <0x0957278e> 同时 locked <0x0dfd4baf> (a java.util.ArrayList) B线程 waiting to lock <0x0dfd4baf> 同时 locked <0x0957278e> 是一个很明显的死锁问题

分析源码，简化代码逻辑如下： A线程: BluetoothEventManager$1.onReceive frameworks/base/packages/SettingsLib/src/com/android/settingslib/bluetooth/CachedBluetoothDeviceManager.java 获得CachedBluetoothDeviceManager锁

        CachedBluetoothDevice cachedDevice = findDevice(device);
        if (cachedDevice != null) {
            cachedDevice.onUuidChanged();
        }
    }

请求mCallbacks锁 frameworks/base/packages/SettingsLib/src/com/android/settingslib/bluetooth/CachedBluetoothDevice.java

    private void dispatchAttributesChanged() {
        synchronized (mCallbacks) {
            for (Callback callback : mCallbacks) {
                callback.onDeviceAttributesChanged();
            }
        }
    }

B线程: 先调用dispatchAttributesChanged() 获得mCallbacks锁 HidProfile$InputDeviceServiceListener.onServiceConnected->CachedBluetoothDevice.refresh->CachedBluetoothDevice.dispatchAttributesChanged->Systemui-> updateConnected() /home/android/codes/qc8937/LA.UM.6.6/LINUX/android/vendor/myos/packages/SystemUI/src/com/android/systemui/statusbar/policy/BluetoothControllerImpl.java

请求CachedBluetoothDeviceManager锁 CachedBluetoothDeviceManager.java

public synchronized Collection<CachedBluetoothDevice> getCachedDevicesCopy() {
    return new ArrayList<CachedBluetoothDevice>(mCachedDevices);
}

用最简单的DEMO抽象这个死锁问题

package com.fantasy.android.demo.java;

public class DeadObjectTest {

    private Object lockOne = new Object();
    private Object lockTwo = new Object();

    private void doA() {
        synchronized (lockOne) {
            try {
                System.out.println(Thread.currentThread().getName() + " doA begin");
                Thread.sleep(100);
                System.out.println(Thread.currentThread().getName() + " doA end");
                doSame();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    private void doSame() {
        synchronized (lockTwo) {
            try {
                System.out.println(Thread.currentThread().getName() + " doSame begin");
                Thread.sleep(200);
                System.out.println(Thread.currentThread().getName() + " doSame end ");
                doB();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    private void doB() {
        synchronized (lockOne) {
            try {
                System.out.println(Thread.currentThread().getName() + " doB begin ");
                Thread.sleep(100);
                System.out.println(Thread.currentThread().getName() + " doB end ");
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    public static void main(String args[]) {
        DeadObjectTest test = new DeadObjectTest();
        Thread a = new Thread(new Runnable() {
            @Override
            public void run() {
                test.doA();
            }
        });
        a.start();

        Thread b = new Thread(new Runnable() {
            @Override
            public void run() {
                test.doSame();
            }
        });
        b.start();
    }
}

执行查看运行结果：

Thread-0 doA begin
Thread-1 doSame begin
Thread-0 doA end
Thread-1 doSame end

没有“process finished” ，死锁了。

####解决方案一：使AB同步

    private void doA() {
        synchronized (lockOne) {
            try {
                // 出让锁
                lockOne.wait();
                System.out.println(Thread.currentThread().getName() + " doA begin");
                Thread.sleep(100);
                System.out.println(Thread.currentThread().getName() + " doA end");
                doSame();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

    private void doB() {
        synchronized (lockOne) {
            try {
                System.out.println(Thread.currentThread().getName() + " doB begin ");
                Thread.sleep(100);
                System.out.println(Thread.currentThread().getName() + " doB end ");
                // 使A进入阻塞队列
                lockOne.notifyAll();
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
        }
    }

看运行结果：

Thread-1 doSame begin
Thread-1 doSame end 
Thread-1 doB begin 
Thread-1 doB end 
Thread-0 doA begin
Thread-0 doA end
Thread-0 doSame begin
Thread-0 doSame end 
Thread-0 doB begin 
Thread-0 doB end 

Process finished with exit code 0

看起来很完美的解决了。 但是这个解决方案不适用于我们这个问题的情况 有很多方法用到CachedBluetoothDeviceManager这个锁，无法全部做到同步，业务逻辑上也不能全部同步。

####解决方案二：给dispatchAttributesChagned()方法用RenentrantLock的tryLock判断能否获得锁这个方案的问题是如果拿不到锁，有另一个线程的调用就被放弃了。

    private ReentrantLock mLock = new ReentrantLock();
    private void doSame() {
        if (mLock.tryLock()) {
            synchronized (lockTwo) {
                try {
                    System.out.println(Thread.currentThread().getName() + " doSame begin");
                    Thread.sleep(200);
                    System.out.println(Thread.currentThread().getName() + " doSame end ");
                    mLock.unlock();
                    doB();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }

运行结果可以看到Thread 0的 doSame调用没了，虽然解决了死锁。

Thread-0 doA begin
Thread-1 doSame begin
Thread-0 doA end
Thread-1 doSame end 
Thread-1 doB begin 
Thread-1 doB end 

Process finished with exit code 0

好吧，头大，目前还没找到最完美的适合这个问题场景的解决方案，有没有大神不吝赐教?