AutoCloseable引发的事故(Too many open files)

558 阅读2分钟

JDK1.8终于有了StreamAPI,用起来也很顺手,而且根据 public interface BaseStream<T, S extends BaseStream<T, S>> extends AutoCloseable 类签名来看我们是不需要手工关闭资源的,于是结合我在生产中的需求,就有了如下代码:

for (String path : paths) {
    Files.list(Paths.get(path)).forEach(p -> {
        final File file = p.toFile();
        if (file.isDirectory()) {
            try {
                Files.list(file.toPath()).forEach(sub -> {
                    final File subFile = sub.toFile();
                    if (subFile.isDirectory()) {
                        fileMap.put(subFile.getName(), 
                                    subFile.getParentFile().getName());
                    }
                });
            } catch (IOException e) {
                log.error("List file error:", e);
            }
        }
    });
}

先说一下需求,线上需要扫描多个文件夹,需要通过子文件夹找到父文件夹的路径,这段代码就是建立了一个 子文件夹名-父文件夹名 的Map,本地测试没有问题,部署上线却报异常:

Too many open files

这就很奇怪了,多个文件夹内的所有文件也不会超过系统打开文件上线,为什么这里会报这个错误呢,于是Google一番,发现了stackoverflow.com/questions/3…,自己又去看了看AutoCloseable源码,发现如下注释:

/**
 * An object that may hold resources (such as file or socket handles)
 * until it is closed. The {@link #close()} method of an {@code AutoCloseable}
 * object is called automatically when exiting a {@code
 * try}-with-resources block for which the object has been declared in
 * the resource specification header. This construction ensures prompt
 * release, avoiding resource exhaustion exceptions and errors that
 * may otherwise occur.
 *
 * @apiNote
 * <p>It is possible, and in fact common, for a base class to
 * implement AutoCloseable even though not all of its subclasses or
 * instances will hold releasable resources.  For code that must operate
 * in complete generality, or when it is known that the {@code AutoCloseable}
 * instance requires resource release, it is recommended to use {@code
 * try}-with-resources constructions. However, when using facilities such as
 * {@link java.util.stream.Stream} that support both I/O-based and
 * non-I/O-based forms, {@code try}-with-resources blocks are in
 * general unnecessary when using non-I/O-based forms.
 *
 * @author Josh Bloch
 * @since 1.7
 */

其中关键点在于However, when using facilities such as {@link java.util.stream.Stream} that support both I/O-based and non-I/O-based forms, {@code try}-with-resources blocks are in general unnecessary when using non-I/O-based forms.,这段内容很浅显,就是说,非IO操作不需要手工关闭AutoCloseable,IO操作就需要关了,于是代码被改成如下:

for (String path : paths) {
    try (Stream<Path> list = Files.list(Paths.get(path))) {
        list.forEach(p -> {
            final File file = p.toFile();
            if (file.isDirectory()) {
                try (Stream<Path> subStream = Files.list(file.toPath())) {
                    subStream.forEach(sub -> {
                        final File subFile = sub.toFile();
                        if (subFile.isDirectory()) {
                            fileMap.put(subFile.getName(), 
                                subFile.getParentFile().getName());
                        }
                    });
                } catch (IOException e) {
                    log.error("List file error:", e);
                }
            }
        });
    } catch (Exception e) {
        log.error("List file error:", e);
    }
}

这样一来,线上错误就消失了。