Weak、Soft 及 Phantom 引用
另一类影响 GC 的问题是程序中的 non-strong 引用。虽然这类引用在很多情况下可以避免出现 OutOfMemoryError,但过量使用也会对 GC 造成严重的影响,反而降低系统性能。
弱引用的缺点
首先,弱引用(weak reference)是可以被 GC 强制回收的。当垃圾收集器发现一个弱可达对象(weakly reachable,即指向该对象的引用只剩下弱引用)时,就会将其置入相应的 ReferenceQueue 中,变成可终结的对象。之后可能会遍历这个 reference queue,并执行相应的清理。典型的示例是清除缓存中不再引用的 KEY。
当然,在这个时候我们还可以将该对象赋值给新的强引用,在最后终结和回收前,GC 会再次确认该对象是否可以安全回收。因此,弱引用对象的回收过程是横跨多个 GC 周期的。
实际上弱引用使用的很多。大部分缓存框架都是基于弱引用实现的,所以虽然业务代码中没有直接使用弱引用,但程序中依然会大量存在。
软引用的缺点
其次,软引用(soft reference)比弱引用更难被垃圾收集器回收。回收软引用没有确切的时间点,由 JVM 自己决定。一般只会在即将耗尽可用内存时,才会回收软引用,以作最后手段。这意味着可能会有更频繁的 Full GC,暂停时间也比预期更长,因为老年代中的存活对象会很多。
最后,使用虚引用(phantom reference)时,必须手动进行内存管理,以标识这些对象是否可以安全地回收。表面上看起来很正常,但实际上并不是这样。javadoc 中写道:
In order to ensure that a reclaimable object remains so, the referent of a phantom reference may not be retrieved: The get method of a phantom reference always returns null.
为了防止可回收对象的残留,虚引用对象不应该被获取:phantom reference 的 get 方法返回值永远是 null。
Unlike soft and weak references,phantom references are not automatically cleared by the garbage collector as they are enqueued. An object that is reachable via phantom references will remain so until all such references are cleared or themselves become unreachable.
与软引用和弱引用不同,虚引用不会被 GC 自动清除,因为他们被存放到队列中。通过虚引用可达的对象会继续留在内存中,直到调用此引用的 clear 方法,或者引用自身变为不可达。
也就是说,我们必须手动调用 clear() 来清除虚引用,否则可能会造成 OutOfMemoryError而导致 JVM 挂掉。使用虚引用的理由是,对于用编程手段来跟踪某个对象何时变为不可达对象,这是唯一的常规手段。和软引用/弱引用不同的是,我们不能“复活”虚可达(phantom-reachable)对象。
示例
让我们看一个弱引用示例,
package com.core.gc;
import java.lang.ref.WeakReference;
import java.util.Arrays;
public class WeakReferences {
/**
* This example shows how having weak references pointing to objects
* May result in more frequent Full GC pauses
* 这个demo 展示的是如何使用弱引用指向对象而导致频繁的full gc
* <p>
* There are two modes (controlled by weak.refs)
* 1. A lot of objects are created
* 2. A lot of objects are created, and weak references are created
* for them. These references are held in a buffer until it's full
* <p>
* 有两种模式
* 1.创建了许多对象
* 2.创建了许多对象,并为它们创建了弱引用。这些引用被保存在缓冲区中,直到缓冲区满为止
* <p>
* The allocations made in both cases need to be exactly the same,
* so in (1) weak references will be also created, but all of them
* will be pointing to the same object
* <p>
* 在这两种情况下进行的分配需要完全相同,因此在(1)中也将创建弱引用,但所有引用都指向同一对象
*
* <p>
* 1. Run with: -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy -XX:+PrintGCTimeStamps
* <p>
* Observe that there are mostly young GCs
* 观察可以发现大量的young gc
* <p>
* 2. Run with: -Dweak.refs=true -verbose:gc -Xmx24m -XX:NewSize=16m-XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
* <p>
* Observe that there are mostly full GCs
* 观察可以发现大量的full gc
* <p>
* 3. Run with: -Dweak.refs=true -verbose:gc -Xmx64m -XX:NewSize=32m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
* <p>
* Observe that there are mostly young GCs
* 观察可以发现大量的young gc
*/
private static final int OBJECT_SIZE = Integer.getInteger("obj.size", 192);
private static final int BUFFER_SIZE = Integer.getInteger("buf.size", 64 * 1024);
private static final boolean WEAK_REFS_FOR_ALL = Boolean.getBoolean("weak.refs");
private static Object makeObject() {
return new byte[OBJECT_SIZE];
}
public static volatile Object sink;
public static void main(String[] args) throws InterruptedException {
System.out.printf("Buffer size: %d; Object size: %d; Weak refs for all: %s%n", BUFFER_SIZE, OBJECT_SIZE, WEAK_REFS_FOR_ALL);
final Object substitute = makeObject(); // We want to create it in both scenarios so the footprint matches
final Object[] refs = new Object[BUFFER_SIZE];
System.gc(); // Clean up young gen
for (int index = 0; ; ) {
Object object = makeObject();
sink = object; // Prevent Escape Analysis from optimizing the allocation away
if (!WEAK_REFS_FOR_ALL) {
object = substitute;
}
refs[index++] = new WeakReference<>(object);
if (index == BUFFER_SIZE) {
Arrays.fill(refs, null);
index = 0;
}
}
}
}
其中创建了大量的对象,并在 Minor GC 中完成回收。修改提升阀值。
Run with: -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy -XX:+PrintGCTimeStamps
-Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1
此时 GC 日志如下所示:
0.237: [GC (Allocation Failure) 15032K->2328K(22528K), 0.0039775 secs]
0.243: [GC (Allocation Failure) 14616K->1880K(22528K), 0.0022431 secs]
0.246: [GC (Allocation Failure) 14168K->3656K(22528K), 0.0095724 secs]
0.257: [GC (Allocation Failure) 15944K->3656K(22528K), 0.0074434 secs]
0.266: [GC (Allocation Failure) 15944K->3176K(22528K), 0.0056915 secs]
0.273: [GC (Allocation Failure) 15464K->2760K(22528K), 0.0039805 secs]
0.279: [GC (Allocation Failure) 15048K->2344K(22528K), 0.0019539 secs]
0.282: [GC (Allocation Failure) 14632K->4064K(22528K), 0.0086142 secs]
0.293: [GC (Allocation Failure) 16352K->4096K(22528K), 0.0076310 secs]
0.303: [GC (Allocation Failure) 16384K->3648K(22528K), 0.0056940 secs]
0.761: [Full GC (Ergonomics) 9448K->2819K(22528K), 0.0183875 secs]
可以看到,Full GC 的次数很少。但如果使用弱引用来指向创建的对象,使用 JVM 参数 -Dweak.refs=true,则情况会发生明显变化。使用弱引用的原因很多,比如在 weak hash map 中将对象作为 Key 的情况。在任何情况下,使用弱引用都可能会导致以下情形:
2.059: [Full GC (Ergonomics) 20365K->19611K(22528K), 0.0654090 secs]
2.125: [Full GC (Ergonomics) 20365K->19711K(22528K), 0.0707499 secs]
2.196: [Full GC (Ergonomics) 20365K->19798K(22528K), 0.0717052 secs]
2.268: [Full GC (Ergonomics) 20365K->19873K(22528K), 0.0686290 secs]
2.337: [Full GC (Ergonomics) 20365K->19939K(22528K), 0.0702009 secs]
2.407: [Full GC (Ergonomics) 20365K->19995K(22528K), 0.0694095 secs]
可以看到,发生了多次 Full GC
这是过早提升的另一个例子,但这次情况更加棘手:问题的根源在于弱引用。这些临死的对象,在添加弱引用之后,被提升到了老年代。但是,他们现在陷入另一次 GC 循环之中,所以需要对其做一些适当的清理。
像之前一样,最简单的办法是增加年轻代的大小,例如指定 JVM 参数 -Xmx64m -XX:NewSize=32m:
2.328: [GC (Allocation Failure) 38940K->13596K(61440K),0.0012818 secs]
2.332: [GC (Allocation Failure) 38172K->14812K(61440K),0.0060333 secs]
2.341: [GC (Allocation Failure) 39388K->13948K(61440K),0.0029427 secs]
2.347: [GC (Allocation Failure) 38524K->15228K(61440K),0.0101199 secs]
2.361: [GC (Allocation Failure) 39804K->14428K(61440K),0.0040940 secs]
2.368: [GC (Allocation Failure) 39004K->13532K(61440K),0.0012451 secs]
这时候,对象在 Minor GC 中就被回收了。
更坏的情况是使用软引用,例如
package com.core.gc;
import java.lang.ref.SoftReference;
import java.util.Arrays;
public class SoftReferences {
// This example shows how having soft references pointing to objects
// May result in more frequent Full GC pauses
//
// There are two modes (controlled by soft.refs)
//
// 1. A lot of objects are created
// 2. A lot of objects are created, and soft references are created
// for them. These references are held in a buffer until it's full
//
// The allocations made in both cases need to be exactly the same,
// so in (1) soft references will be also created, but all of them
// will be pointing to the same object
// 1. Run with: -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
//
// Observe that there are mostly young GCs
//
// 2. Run with: -Dsoft.refs=true -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
//
// Observe that there are lots of full GCs
//
// 3. Run with: -Dsoft.refs=true -verbose:gc -Xmx64m -XX:NewSize=32m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
//
// Observe that there are still many full GCs
private static final int OBJECT_SIZE = Integer.getInteger("obj.size", 192);
private static final int BUFFER_SIZE = Integer.getInteger("buf.size", 64 * 1024);
private static final boolean SOFT_REFS_FOR_ALL = Boolean.getBoolean("soft.refs");
private static Object makeObject() {
return new byte[OBJECT_SIZE];
}
public static volatile Object sink;
public static void main(String[] args) throws InterruptedException {
System.out.printf("Buffer size: %d; Object size: %d; Soft refs for all: %s%n", BUFFER_SIZE, OBJECT_SIZE, SOFT_REFS_FOR_ALL);
final Object substitute = makeObject(); // We want to create it in both scenarios so the footprint matches
final Object[] refs = new Object[BUFFER_SIZE];
System.gc(); // Clean up young gen
for (int index = 0; ; ) {
Object object = makeObject();
sink = object; // Prevent Escape Analysis from optimizing the allocation away
if (!SOFT_REFS_FOR_ALL) {
object = substitute;
}
refs[index++] = new SoftReference<>(object);
if (index == BUFFER_SIZE) {
Arrays.fill(refs, null);
index = 0;
}
}
}
}
如果程序不是即将发生 OutOfMemoryError,软引用对象就不会被回收。在示例程序中,用软引用替代弱引用,立即出现了更多的 Full GC 事件:
2.162: [Full GC (Ergonomics) 31561K->12865K(61440K),0.0181392 secs]
2.184: [GC (Allocation Failure) 37441K->17585K(61440K),0.0024479 secs]
2.189: [GC (Allocation Failure) 42161K->27033K(61440K),0.0061485 secs]
2.195: [Full GC (Ergonomics) 27033K->14385K(61440K),0.0228773 secs]
2.221: [GC (Allocation Failure) 38961K->20633K(61440K),0.0030729 secs]
2.227: [GC (Allocation Failure) 45209K->31609K(61440K),0.0069772 secs]
2.234: [Full GC (Ergonomics) 31609K->15905K(61440K),0.0257689 secs]
最有趣的是虚引用,
package com.core.gc;
import sun.misc.GC;
import java.util.ArrayList;
import java.util.Collection;
public class PrematurePromotion {
/**
* This example shows how objects lingering in the heap for too long
* may result in many more Full GC pauses than there could be.
* <p>
* 1. Run with: -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
* <p>
* Observe that there are many Full GCs
* 可以发现有很多full GC
* <p>
* 2. Run with: -verbose:gc -Xmx64m -XX:NewSize=32m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
* <p>
* Observe that most of GCs are minor
* <p>
* 3. Run with: -Dmax.chunks=1000 -verbose:gc -Xmx24m -XX:NewSize=16m -XX:MaxTenuringThreshold=1 -XX:-UseAdaptiveSizePolicy
* <p>
* Observe that most of GCs are minor
*/
private static final int MAX_CHUNKS = Integer.getInteger("max.chunks", 10_000);
private static final Collection<byte[]> accumulatedChunks = new ArrayList<>();
private static void onNewChunk(byte[] bytes) {
accumulatedChunks.add(bytes);
if (accumulatedChunks.size() > MAX_CHUNKS) {
processBatch(accumulatedChunks);
accumulatedChunks.clear();
}
}
public static void main(String[] args) {
while (true) {
onNewChunk(produceChunk());
}
}
private static byte[] produceChunk() {
byte[] bytes = new byte[1024];
for (int i = 0; i < bytes.length; i++) {
bytes[i] = (byte) (Math.random() * Byte.MAX_VALUE);
}
return bytes;
}
public static volatile byte sink;
public static void processBatch(Collection<byte[]> bytes) {
byte result = 0;
for (byte[] chunk : bytes) {
for (byte b : chunk) {
result ^= b;
}
}
sink = result;
}
}
使用同样的 JVM 参数启动,其结果和弱引用示例非常相似。实际上,Full GC 暂停的次数会小得多,原因前面说过,他们有不同的终结方式。
如果禁用虚引用清理,增加 JVM 启动参数(-Dno.ref.clearing=true),则可以看到:
4.180: [Full GC (Ergonomics) 57343K->57087K(61440K),0.0879851 secs]
4.269: [Full GC (Ergonomics) 57089K->57088K(61440K),0.0973912 secs]
4.366: [Full GC (Ergonomics) 57091K->57089K(61440K),0.0948099 secs]
主线程中很快抛出异常:
java.lang.OutOfMemoryError: Java heap space
使用虚引用时要小心谨慎,并及时清理虚可达对象。如果不清理,很可能会发生 OutOfMemoryError。
请相信我们的经验教训:处理 reference queue 的线程中如果没 catch 住异常,系统很快就会被整挂了。
使用非强引用的影响
使用 JVM 参数 -XX:+PrintReferenceGC 来看看各种引用对 GC 的影响。如果将此参数用于启动弱引用示例,将会看到:
2.173: [Full GC (Ergonomics)
2.234: [SoftReference,0 refs,0.0000151 secs]
2.234: [WeakReference,2648 refs,0.0001714 secs]
2.234: [FinalReference,1 refs,0.0000037 secs]
2.234: [PhantomReference,0 refs,0 refs,0.0000039 secs]
2.234: [JNI Weak Reference,0.0000027 secs]
[PSYoungGen: 9216K->8676K(10752K)]
[ParOldGen: 12115K->12115K(12288K)]
21331K->20792K(23040K),
[Metaspace: 3725K->3725K(1056768K)],
0.0766685 secs]
[Times: user=0.49 sys=0.01,real=0.08 secs]
2.250: [Full GC (Ergonomics)
2.307: [SoftReference,0 refs,0.0000173 secs]
2.307: [WeakReference,2298 refs,0.0001535 secs]
2.307: [FinalReference,3 refs,0.0000043 secs]
2.307: [PhantomReference,0 refs,0 refs,0.0000042 secs]
2.307: [JNI Weak Reference,0.0000029 secs]
[PSYoungGen: 9215K->8747K(10752K)]
[ParOldGen: 12115K->12115K(12288K)]
21331K->20863K(23040K),
[Metaspace: 3725K->3725K(1056768K)],
0.0734832 secs]
[Times: user=0.52 sys=0.01,real=0.07 secs]
2.323: [Full GC (Ergonomics)
2.383: [SoftReference,0 refs,0.0000161 secs]
2.383: [WeakReference,1981 refs,0.0001292 secs]
2.383: [FinalReference,16 refs,0.0000049 secs]
2.383: [PhantomReference,0 refs,0 refs,0.0000040 secs]
2.383: [JNI Weak Reference,0.0000027 secs]
[PSYoungGen: 9216K->8809K(10752K)]
[ParOldGen: 12115K->12115K(12288K)]
21331K->20925K(23040K),
[Metaspace: 3725K->3725K(1056768K)],
0.0738414 secs]
[Times: user=0.52 sys=0.01,real=0.08 secs]
只有确定 GC 对应用的吞吐量和延迟造成影响之后,才应该花心思来分析这些信息,审查这部分日志。通常情况下,每次 GC 清理的引用数量都是很少的,大部分情况下为 0。
如果 GC 花了较多时间来清理这类引用,或者清除了很多的此类引用,就需要进一步观察和分析了。
解决方案
如果程序确实碰到了 mis-、ab- 等问题或者滥用 weak/soft/phantom 引用,一般都要修改程序的实现逻辑。每个系统不一样,因此很难提供通用的指导建议,但有一些常用的经验办法:
- 弱引用(Weak references):如果某个内存池的使用量增大,造成了性能问题,那么增加这个内存池的大小(可能也要增加堆内存的最大容量)。如同示例中所看到的,增加堆内存的大小,以及年轻代的大小,可以减轻症状。
- 软引用(Soft references):如果确定问题的根源是软引用,唯一的解决办法是修改程序源码,改变内部实现逻辑。
- 虚引用(Phantom references):请确保在程序中调用了虚引用的 clear 方法。编程中很容易忽略某些虚引用,或者清理的速度跟不上生产的速度,又或者清除引用队列的线程挂了,就会对 GC 造成很大压力,最终可能引起 OutOfMemoryError。