Android中的SparseArray与ArrayMap

SparseArray

• SparseArray是Android专门设计的一种key为int的key-Value型的map存储方式，使用int当key避免了HashMap中Integer当key而出现拆装箱的问题
• SparseArray比hashMap占用内存更小（底层数据结构决定的，无加载因子使得内存使用率更高，从而占用内存小），效率也更高
• SparseArray 还存在一些变体类，原理大同小异
  • SparseBooleanArray
  • SparseIntArray
  • SparseLongArray
  • LongSparseArray
• 实测比hashMap效率高很多

@Test
public void sparseArrayPut() {
    SparseArray<String> sparseArray = new SparseArray<>();
    long time = System.currentTimeMillis();
    for (int i = 0; i < 2000000; i++) {
        sparseArray.put(i, "数据" + i);
    }
    System.out.println("SparseArray的put消耗时间：" + (System.currentTimeMillis() - time) + "ms");
    long time3 = System.currentTimeMillis();
    for (int i = 0; i < 2000000; i++) {
        sparseArray.get(i);
    }
    System.out.println("SparseArray的get消耗时间：" + (System.currentTimeMillis() - time3) + "ms");

    HashMap<Integer, String> hashMap = new HashMap<>();
    long time2 = System.currentTimeMillis();
    for (int i = 2000000; i > 0; i--) {
        hashMap.put(i, "数据" + i);
    }
    System.out.println("hashMap的put消耗时间：" + (System.currentTimeMillis() - time2) + "ms");
    long time4 = System.currentTimeMillis();
    for (int i = 0; i < 2000000; i++) {
        hashMap.get(i);
    }
    System.out.println("hashMap的get消耗时间：" + (System.currentTimeMillis() - time4) + "ms");
}

运行结果：
SparseArray的put消耗时间：105ms
SparseArray的get消耗时间：4ms
hashMap的put消耗时间：357ms
hashMap的get消耗时间：45ms

构造器

• 利用一个key数组和一个Value数组来存放数据
• 初始默认容量为10

public SparseArray() {
    this(10);
}

/**
 * Creates a new SparseArray containing no mappings that will not
 * require any additional memory allocation to store the specified
 * number of mappings.  If you supply an initial capacity of 0, the
 * sparse array will be initialized with a light-weight representation
 * not requiring any additional array allocations.
 */
public SparseArray(int initialCapacity) {
    if (initialCapacity == 0) {
        // int[] mKeys int数组存放key
        mKeys = EmptyArray.INT;
        // Object[] mValues Object数组存放Value
        mValues = EmptyArray.OBJECT;
    } else {
        mValues = ArrayUtils.newUnpaddedObjectArray(initialCapacity);
        mKeys = new int[mValues.length];
    }
    mSize = 0;
}

put

• 使用二分查找法查找key在keys数组中的位置，查到了表示已存有值，直接覆盖value即可
• 没查到，直接在插入位置插入或者替换DELETED标签
• 当Values数组中有DELETED标签（mGarbage=true）并且keys数组已满，会调用gc方法去除双数组中的DELETED标志
• 确定插入位置后面的元素后移，如果keys数组元素已满则会触发扩容，扩容后再进行插入
• 扩容方式：currentSize <= 4 ? 8 : currentSize * 2

/**
 * Adds a mapping from the specified key to the specified value,
 * replacing the previous mapping from the specified key if there
 * was one.
 * 添加一个元素，如果可以已存在直接覆盖更新
 */
public void put(int key, E value) {
    // 通过二分查找算法查找有序数组keys中指定的key值，找到了返回该位置，
    // 未找到返回~i(-i-1)即返回一个相邻近的取反值（~i=-i-1）方便后续获取插入位置
    int i = ContainerHelpers.binarySearch(mKeys, mSize, key);

    if (i >= 0) {
       // 找到相同的key,直接覆盖更新即可
        mValues[i] = value;
    } else {
        i = ~i; // 获取插入位置
        // 这里的DELETED是一个标志，表示这个位置已被删除，在remove方法中赋值
        if (i < mSize && mValues[i] == DELETED) {
        // 插入的位置是一个DELETED直接覆盖即可
            mKeys[i] = key;
            mValues[i] = value;
            return;
        }

        // mGarbage为true(remove方法中赋值)并且容器元素已满调用gc方法去除values数组中
        // 的DELETE标识以及keys数组中对应的index
        if (mGarbage && mSize >= mKeys.length) {
            gc();

            // Search again because indices may have changed. 
            // 由于删除了DELETED数组改变,需重新获取插入位置i
            i = ~ContainerHelpers.binarySearch(mKeys, mSize, key);
        }

        // GrowingArrayUtils的insert方法将会将插入位置之后的所有数据向后移动一位，
        // 然后将key和value分别插入到mKeys和mValue对应的第i个位置，
        // 如果数组空间不足还会开启扩容
        mKeys = GrowingArrayUtils.insert(mKeys, mSize, i, key);
        mValues = GrowingArrayUtils.insert(mValues, mSize, i, value);
        mSize++;
    }
}

• 使用二分查找算法，找到返回index，未找到返回最邻近位置的取反值（-index-1）

// 二分查找算法，找到返回index,未找到返回最邻近位置的取反值（-index-1）
static int binarySearch(int[] array, int size, int value) {
    int lo = 0;
    int hi = size - 1;

    while (lo <= hi) {
        final int mid = (lo + hi) >>> 1;
        final int midVal = array[mid];

        if (midVal < value) {
            lo = mid + 1;
        } else if (midVal > value) {
            hi = mid - 1;
        } else {
            return mid;  // value found
        }
    }
    return ~lo;  // value not present
}

• 插入以及扩容操作

// GrowingArrayUtils
public static <T> T[] insert(T[] array, int currentSize, int index, T element) {
    assert currentSize <= array.length;
    // 如果插入后数组size小于数组长度，能进行插入操作
    if (currentSize + 1 <= array.length) {
        // 将index之后的所有元素向后移动一位
        System.arraycopy(array, index, array, index + 1, currentSize - index);
        // 将key插入到index的位置
        array[index] = element;
        return array;
    }

    // 数组已满，需进行扩容操作。newArray即为扩容后的数组
    T[] newArray = ArrayUtils.newUnpaddedArray((Class<T>)array.getClass().getComponentType(),
            growSize(currentSize));
    System.arraycopy(array, 0, newArray, 0, index);
    newArray[index] = element;
    System.arraycopy(array, index, newArray, index + 1, array.length - index);
    return newArray;
}

// 返回扩容后的size
public static int growSize(int currentSize) {
    return currentSize <= 4 ? 8 : currentSize * 2;
}

• gc方法去除values中的DELETED标志以及相对应keys中的key的index，获取有用数据的总size，从而调用System.arraycopy复制进新数组中

private void gc() {
    // Log.e("SparseArray", "gc start with " + mSize);

    int n = mSize;
    int o = 0;
    int[] keys = mKeys;
    Object[] values = mValues;

    for (int i = 0; i < n; i++) {
        Object val = values[i];

        if (val != DELETED) {
            if (i != o) {
                keys[o] = keys[i];
                values[o] = val;
                values[i] = null;
            }

            o++;
        }
    }

    mGarbage = false;
    mSize = o;

    // Log.e("SparseArray", "gc end with " + mSize);
}

get

• 通过二分查找获取key在keys数组中所在的位置，如果获取不到或者获取到的位置所对应的Value为DELETED，则返回null，否则返回查找到的value值

/**
 * Gets the Object mapped from the specified key, or <code>null</code>
 * if no such mapping has been made.
 */
public E get(int key) {
    return get(key, null);
}

/**
 * Gets the Object mapped from the specified key, or the specified Object
 * if no such mapping has been made.
 */
@SuppressWarnings("unchecked")
public E get(int key, E valueIfKeyNotFound) {
    int i = ContainerHelpers.binarySearch(mKeys, mSize, key);
    // 未找到或找到指定位置对应的Value值为DELETED表明无此元素返回null
    if (i < 0 || mValues[i] == DELETED) {
        return valueIfKeyNotFound;
    } else {
        return (E) mValues[i];
    }
}

remove

• 寻找要删除的元素将其置为DELETED，而不是真正删除，比数组移位操作效率高得多
• 在置为DELETED标志的同时将mGarbage置为true,表示value中有DELETED元素，好在put时判断是否需要gc

/**
 * Alias for {@link #delete(int)}.
 */
public void remove(int key) {
    // 本质上调用的是delete方法
    delete(key);
}

/**
 * Removes the mapping from the specified key, if there was any.
 */
public void delete(int key) {
    int i = ContainerHelpers.binarySearch(mKeys, mSize, key);

    // 这是精髓之处，不真正删除，而是将要删除位置的元素置为DELETED，比数组移位效率高得多
    // 全局变量private static final Object DELETED = new Object();
    // 并且将mGarbage置为true,表示value中有DELETED元素好在put时判断是否需要gc
    if (i >= 0) {
        if (mValues[i] != DELETED) {
            mValues[i] = DELETED;
            mGarbage = true;
        }
    }
}

总结

• SparseArray 内部使用双数组，分别存储 Key 和 Value，Key 是 int[]，用于查找 Value 对应的 Index，来定位数据在 Value 中的位置。
• key的存储以及查找是利用二分查找法，找到指定位置，或当不存在时找到刚好比某值小的上一个位置，同时兼顾顺序插入以及查找
• remove中巧妙的利用了DELETE 标记避免数组移动带来的效率低的问题

ArrayMap

• ArrayMap是Android专门设计的一种key-Value型的map存储方式，主要是在数据量比较小时用来替代HashMap，比HashMap占用内存更少，空间换时间
  • HashMap存在加载因子，无法充分利用内存，而且每次扩容都为2倍；
  • ArrayMap可以充分利用内存，并且扩容为n=old < 4 ? 4 : (old < 8 ? 8: old*1.5)

@Test
public void arrayPut() {
    ArrayMap<String, Integer> arrayMap = new ArrayMap<>();
    long time = System.currentTimeMillis();
    for (int i = 0; i < 20000; i++) {
        arrayMap.put("数据" + i, i);
    }
    // 当倒叙时key存储的顺序为由大到小，而arrayMap存储的key是由小到大，速度会更慢，实测126ms
    // 所以当存储的key值正序率比较高时效率更高
    // for (int i = 20000; i > 0; i--) {
    //    arrayMap.put("数据" + i, i);
    // }
    System.out.println("arrayMap的put消耗时间：" + (System.currentTimeMillis() - time) + "ms");

    HashMap<String, Integer> hashMap = new HashMap<>();
    long time2 = System.currentTimeMillis();
    for (int i = 20000; i > 0; i--) {
        hashMap.put("数据" + i, i);
    }
    System.out.println("hashMap的put消耗时间：" + (System.currentTimeMillis() - time2) + "ms");
}

运行结果：
arrayMap的put消耗时间：24ms
hashMap的put消耗时间：13ms

• ArrayMap存储的数据结构，array数组是hash数组的两倍

// 保存每个key的hashCode的数组
int[] mHashes;
// 存储key、value的数组
Object[] mArray;

构造器

• 默认初始容量为0，当调用put时才会进行数组初始化

/**
 * Create a new empty ArrayMap.  The default capacity of an array map is 0, and
 * will grow once items are added to it.
 * 默认创建一个空的ArrayMap，只有在添加元素时才会进行扩容
 */
public ArrayMap() {
    this(0, false);
}

/**
 * Create a new ArrayMap with a given initial capacity.
 */
public ArrayMap(int capacity) {
    this(capacity, false);
}

/** {@hide} 此构造对外不可见 */
public ArrayMap(int capacity, boolean identityHashCode) {
    mIdentityHashCode = identityHashCode;

    // If this is immutable, use the sentinal EMPTY_IMMUTABLE_INTS
    // instance instead of the usual EmptyArray.INT. The reference
    // is checked later to see if the array is allowed to grow.
    if (capacity < 0) {
        mHashes = EMPTY_IMMUTABLE_INTS;
        mArray = EmptyArray.OBJECT;
    } else if (capacity == 0) {
        mHashes = EmptyArray.INT;
        mArray = EmptyArray.OBJECT;
    } else {
        allocArrays(capacity);
    }
    mSize = 0;
}

• allocArrays(capacity)扩容分配内存

private void allocArrays(final int size) {
    if (mHashes == EMPTY_IMMUTABLE_INTS) {
        throw new UnsupportedOperationException("ArrayMap is immutable");
    }
    // 容量为8
    if (size == (BASE_SIZE*2)) {
        synchronized (sTwiceBaseCacheLock) {
            if (mTwiceBaseCache != null) {
                // 缓存相关...
                final Object[] array = mTwiceBaseCache;
                mArray = array;
                try {
                    mTwiceBaseCache = (Object[]) array[0];
                    mHashes = (int[]) array[1];
                    if (mHashes != null) {
                        array[0] = array[1] = null;
                        mTwiceBaseCacheSize--;
                        if (DEBUG) {
                            Log.d(TAG, "Retrieving 2x cache " + mHashes
                                    + " now have " + mTwiceBaseCacheSize + " entries");
                        }
                        return;
                    }
                } catch (ClassCastException e) {
                }
                // Whoops!  Someone trampled the array (probably due to not protecting
                // their access with a lock).  Our cache is corrupt; report and give up.
                Slog.wtf(TAG, "Found corrupt ArrayMap cache: [0]=" + array[0]
                        + " [1]=" + array[1]);
                mTwiceBaseCache = null;
                mTwiceBaseCacheSize = 0;
            }
        }
    } else if (size == BASE_SIZE) { // 容量为4
        synchronized (sBaseCacheLock) {
            if (mBaseCache != null) {
                final Object[] array = mBaseCache;
                mArray = array;
                try {
                    mBaseCache = (Object[]) array[0];
                    mHashes = (int[]) array[1];
                    if (mHashes != null) {
                        array[0] = array[1] = null;
                        mBaseCacheSize--;
                        if (DEBUG) {
                            Log.d(TAG, "Retrieving 1x cache " + mHashes
                                    + " now have " + mBaseCacheSize + " entries");
                        }
                        return;
                    }
                } catch (ClassCastException e) {
                }
                // Whoops!  Someone trampled the array (probably due to not protecting
                // their access with a lock).  Our cache is corrupt; report and give up.
                Slog.wtf(TAG, "Found corrupt ArrayMap cache: [0]=" + array[0]
                        + " [1]=" + array[1]);
                mBaseCache = null;
                mBaseCacheSize = 0;
            }
        }
    }
    
   // Array存储了key和value所以是hash数组的2倍
  mHashes = new int[size];
  mArray = new Object[size<<1];
}

put

/**
 * Add a new value to the array map.
 * @param key The key under which to store the value.  If
 * this key already exists in the array, its value will be replaced.
 * @param value The value to store for the given key.
 * @return Returns the old value that was stored for the given key, or null if there
 * was no such key.
 */
@Override
public V put(K key, V value) {
    final int osize = mSize;
    final int hash;
    int index;
    // key为null时hash值为0
    if (key == null) {
        hash = 0;
        // 获取null的下标
        index = indexOfNull();
    } else {
        // 根据key获取hash值
        hash = mIdentityHashCode ? System.identityHashCode(key) : key.hashCode();
        // 通过key和hash获取key存放的具体位置
        index = indexOf(key, hash);
    }
    // index>=0表示找到了相同的位置，可以直接替换旧值
    if (index >= 0) {
        // 这里由于array数组对应的key位置为index*2+1
        index = (index<<1) + 1;
        final V old = (V)mArray[index];
        // 更新value值
        mArray[index] = value;
        return old;
    }

    // 没找到相同的key,进行添加操作
    index = ~index;
    // 如果需要扩容便进行扩容操作
    if (osize >= mHashes.length) {
        // 扩容逻辑：n=old < 4 ? 4 : (old < 8 ? 8: old*1.5)
        final int n = osize >= (BASE_SIZE*2) ? (osize+(osize>>1))
                : (osize >= BASE_SIZE ? (BASE_SIZE*2) : BASE_SIZE);

        if (DEBUG) Log.d(TAG, "put: grow from " + mHashes.length + " to " + n);

        final int[] ohashes = mHashes;
        final Object[] oarray = mArray;
        allocArrays(n);

        if (CONCURRENT_MODIFICATION_EXCEPTIONS && osize != mSize) {
            throw new ConcurrentModificationException();
        }

        if (mHashes.length > 0) {
            if (DEBUG) Log.d(TAG, "put: copy 0-" + osize + " to 0");
            // hash数组以及array数组的复制操作
            System.arraycopy(ohashes, 0, mHashes, 0, ohashes.length);
            System.arraycopy(oarray, 0, mArray, 0, oarray.length);
        }
        
        // 释放临时数组空间，以及数组的缓存操作
        freeArrays(ohashes, oarray, osize);
    }

    // index在中间需要进行数组移位操作
    if (index < osize) {
        if (DEBUG) Log.d(TAG, "put: move " + index + "-" + (osize-index)
                + " to " + (index+1));
        System.arraycopy(mHashes, index, mHashes, index + 1, osize - index);
        System.arraycopy(mArray, index << 1, mArray, (index + 1) << 1, (mSize - index) << 1);
    }

    if (CONCURRENT_MODIFICATION_EXCEPTIONS) {
        if (osize != mSize || index >= mHashes.length) {
            throw new ConcurrentModificationException();
        }
    }
    
    // 分别对hash数组以及array数组赋值
    mHashes[index] = hash;
    // array数组，根据下标*2存key，*2+1存value
    mArray[index<<1] = key;
    mArray[(index<<1)+1] = value;
    mSize++;
    return null;
}

• indexOfNull()和indexOf(key, hash),里面主要用到了二分查找的算法

remove

• 主要调用了removeAt方法

/**
 * Remove an existing key from the array map.
 * @param key The key of the mapping to remove.
 * @return Returns the value that was stored under the key, or null if there
 * was no such key.
 */
@Override
public V remove(Object key) {
    final int index = indexOfKey(key);
    if (index >= 0) {
        return removeAt(index);
    }

    return null;
}

/**
 * Remove the key/value mapping at the given index.
 *
 * <p>For indices outside of the range <code>0...size()-1</code>, the behavior is undefined for
 * apps targeting {@link android.os.Build.VERSION_CODES#P} and earlier, and an
 * {@link ArrayIndexOutOfBoundsException} is thrown for apps targeting
 * {@link android.os.Build.VERSION_CODES#Q} and later.</p>
 *
 * @param index The desired index, must be between 0 and {@link #size()}-1.
 * @return Returns the value that was stored at this index.
 */
public V removeAt(int index) {
    if (index >= mSize && UtilConfig.sThrowExceptionForUpperArrayOutOfBounds) {
        // The array might be slightly bigger than mSize, in which case, indexing won't fail.
        // Check if exception should be thrown outside of the critical path.
        throw new ArrayIndexOutOfBoundsException(index);
    }

    final Object old = mArray[(index << 1) + 1];
    final int osize = mSize;
    final int nsize;
    if (osize <= 1) {
        // 如果之前的集合长度小于等于1，则执行过删除操作后，集合现在就是空的
        // Now empty.
        if (DEBUG) Log.d(TAG, "remove: shrink from " + mHashes.length + " to 0");
        final int[] ohashes = mHashes;
        final Object[] oarray = mArray;
        mHashes = EmptyArray.INT;
        mArray = EmptyArray.OBJECT;
        freeArrays(ohashes, oarray, osize);
        nsize = 0;
    } else {
        nsize = osize - 1;
        if (mHashes.length > (BASE_SIZE*2) && mSize < mHashes.length/3) {
            // 根据元素数量和集合占用的空间情况，判断是否要执行收缩操作，如果mHashes长度大于8，且
            // 集合长度小于当前空间的 1/3,则执行一个shrunk，收缩操作，避免空间的浪费
            // Shrunk enough to reduce size of arrays.  We don't allow it to
            // shrink smaller than (BASE_SIZE*2) to avoid flapping between
            // that and BASE_SIZE.
            final int n = osize > (BASE_SIZE*2) ? (osize + (osize>>1)) : (BASE_SIZE*2);

            if (DEBUG) Log.d(TAG, "remove: shrink from " + mHashes.length + " to " + n);
            // 分配新的更小的空间（收缩操作）
            final int[] ohashes = mHashes;
            final Object[] oarray = mArray;
            allocArrays(n);

            if (CONCURRENT_MODIFICATION_EXCEPTIONS && osize != mSize) {
                throw new ConcurrentModificationException();
            }

            if (index > 0) {
                if (DEBUG) Log.d(TAG, "remove: copy from 0-" + index + " to 0");
                System.arraycopy(ohashes, 0, mHashes, 0, index);
                System.arraycopy(oarray, 0, mArray, 0, index << 1);
            }
            if (index < nsize) {
                if (DEBUG) Log.d(TAG, "remove: copy from " + (index+1) + "-" + nsize
                        + " to " + index);
                System.arraycopy(ohashes, index + 1, mHashes, index, nsize - index);
                System.arraycopy(oarray, (index + 1) << 1, mArray, index << 1,
                        (nsize - index) << 1);
            }
        } else {
            if (index < nsize) {
                if (DEBUG) Log.d(TAG, "remove: move " + (index+1) + "-" + nsize
                        + " to " + index);
                // 用复制操作去覆盖元素达到删除的目的        
                System.arraycopy(mHashes, index + 1, mHashes, index, nsize - index);
                System.arraycopy(mArray, (index + 1) << 1, mArray, index << 1,
                        (nsize - index) << 1);
            }
            // 只清空array数组复用hash数组
            mArray[nsize << 1] = null;
            mArray[(nsize << 1) + 1] = null;
        }
    }
    if (CONCURRENT_MODIFICATION_EXCEPTIONS && osize != mSize) {
        throw new ConcurrentModificationException();
    }
    mSize = nsize;
    return (V)old;
}

get

@Override
public V get(Object key) {
    final int index = indexOfKey(key);
    // 直接查找到hash数组中key相对应的hash值的位置然后去Array数组中获取value值
    return index >= 0 ? (V)mArray[(index<<1)+1] : null;
}

/**
 * Returns the index of a key in the set.
 *
 * @param key The key to search for.
 * @return Returns the index of the key if it exists, else a negative integer.
 */
public int indexOfKey(Object key) {
    return key == null ? indexOfNull()
            : indexOf(key, mIdentityHashCode ? System.identityHashCode(key) : key.hashCode());
}

总结

• 默认容量为0，put时才会初始化容量
• 扩容规则：n=old < 4 ? 4 : (old < 8 ? 8: old*1.5)
• 扩容时，会查看之前是否有缓存的int[]数组和object[]数组，有的话赋值给mHashes和mArray
• 每次插入时，根据key的哈希值利用二分查找，去寻找key在mHashes数组中的下标位置
• 根据key的hash值在mHashs中的index，可以得到在mArray中key的位置是index*2，value的位置是index*2+1，也就是说mArray是利用连续的两位空间去存放key、value
• remove操作是利用复制操作去让数组移位，并根据元素数量和集合占用的空间情况，判断是否要执行收缩操作
• 如果mHashes长度大于8，且集合长度小于当前空间的1/3,则执行一个shrunk收缩操作，避免空间的浪费