刨解OkHttp之缓存机制

1,186 阅读8分钟

时间一晃而过,今天想给大家带来OkHttp的zuihou最后一篇文章,主要讲一下OkHttp的缓存机制。OkHttp的责任链中有一个拦截器就是专门应对OkHttp的缓存的,那就是CacheInterceptor拦截器。

CacheInterceptor

其对应的方法如下,我们就从这个方法讲起:

public Response intercept(Chain chain) throws IOException {
    
    //假如有缓存,会得到拿到缓存,否则为null
    Response cacheCandidate = cache != null
        ? cache.get(chain.request())
        : null;

    long now = System.currentTimeMillis();

    //获取缓存策略
    CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();

    //缓存策略请求
    Request networkRequest = strategy.networkRequest;
    //缓存策略响应
    Response cacheResponse = strategy.cacheResponse;

    //缓存非空判断
    if (cache != null) {
      cache.trackResponse(strategy);
    }

    //本地缓存不为空并且缓存策略响应为空
    if (cacheCandidate != null && cacheResponse == null) {
      closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
    }

    //缓存策略请求和缓存策略响应为空,禁止使用网络直接返回
    // If we're forbidden from using the network and the cache is insufficient, fail.
    if (networkRequest == null && cacheResponse == null) {
      return new Response.Builder()
          .request(chain.request())
          .protocol(Protocol.HTTP_1_1)
          .code(504)
          .message("Unsatisfiable Request (only-if-cached)")
          .body(Util.EMPTY_RESPONSE)
          .sentRequestAtMillis(-1L)
          .receivedResponseAtMillis(System.currentTimeMillis())
          .build();
    }

    //缓存策略请求为空,即缓存有效则直接使用缓存不使用网络
    // If we don't need the network, we're done.
    if (networkRequest == null) {
      return cacheResponse.newBuilder()
          .cacheResponse(stripBody(cacheResponse))
          .build();
    }

    //缓存无效,则执行下一个拦截器以获取请求
    Response networkResponse = null;
    try {
      networkResponse = chain.proceed(networkRequest);
    } finally {
      // If we're crashing on I/O or otherwise, don't leak the cache body.
      if (networkResponse == null && cacheCandidate != null) {
        closeQuietly(cacheCandidate.body());
      }
    }

    //假如本地也有缓存,则根据条件选择使用哪个响应
    // If we have a cache response too, then we're doing a conditional get.
    if (cacheResponse != null) {
      if (networkResponse.code() == HTTP_NOT_MODIFIED) {
        Response response = cacheResponse.newBuilder()
            .headers(combine(cacheResponse.headers(), networkResponse.headers()))
            .sentRequestAtMillis(networkResponse.sentRequestAtMillis())
            .receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
            .cacheResponse(stripBody(cacheResponse))
            .networkResponse(stripBody(networkResponse))
            .build();
        networkResponse.body().close();

        // Update the cache after combining headers but before stripping the
        // Content-Encoding header (as performed by initContentStream()).
        cache.trackConditionalCacheHit();
        //更新缓存
        cache.update(cacheResponse, response);
        return response;
      } else {
        closeQuietly(cacheResponse.body());
      }
    }

    //没有缓存,则直接使用网络响应
    Response response = networkResponse.newBuilder()
        .cacheResponse(stripBody(cacheResponse))
        .networkResponse(stripBody(networkResponse))
        .build();

    if (cache != null) {
      //缓存到本地
      if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
        // Offer this request to the cache.
        CacheRequest cacheRequest = cache.put(response);
        return cacheWritingResponse(cacheRequest, response);
      }

      if (HttpMethod.invalidatesCache(networkRequest.method())) {
        try {
          cache.remove(networkRequest);
        } catch (IOException ignored) {
          // The cache cannot be written.
        }
      }
    }

    return response;
  }

这就是整个缓存拦截器的主要方法,首先会从cache去拿缓存,没有则返回null,然后通过CacheStrategy来获取缓存策略,CacheStrategy根据之前缓存的结果与当前将要发送Request的header进行策略,并得出是否进行请求的结果。由于篇幅关系,这一块不细讲因为涉及网络协议,最终他的得出的规则如下如:


image.png

因为我把注释流程都写在代码了,大家可以看上面方法代码理解,其整体缓存流程如下:

  1. 如果有缓存,则取出缓存否则为null
  2. 根据CacheStrategy拿到它的缓存策略请求和响应
  3. 缓存策略请求和缓存策略响应为空,禁止使用网络直接返回
  4. 缓存策略请求为空,即缓存有效则直接使用缓存不使用网络
  5. 缓存无效,则执行下一个拦截器以获取请求
  6. 假如本地也有缓存,则根据条件选择使用哪个响应,更新缓存
  7. 没有缓存,则直接使用网络响应
  8. 添加缓存

到这里我们可以看到,缓存的“增删改查”都是cache(Cache)类来进行操作的。下面让我们来看一下这个类吧。

Cache

Cache的“增删改查”其实都是基于DiskLruCache,下面我们会继续讲,先来看一下“增删改查”的各个方法吧

  • 添加缓存
CacheRequest put(Response response) {
    String requestMethod = response.request().method();
    //如果请求是"POST","PUT","PATCH","PROPPATCH","REPORT"则移除这些缓存  
    if (HttpMethod.invalidatesCache(response.request().method())) {
      try {
        remove(response.request());
      } catch (IOException ignored) {
      }
      return null;
    }
    //仅支持GET的请求缓存,其他请求不缓存
    if (!requestMethod.equals("GET")) {
       return null;
    }
    //判断请求中的http数据包中headers是否有符号"*"的通配符,有则不缓存  
    if (HttpHeaders.hasVaryAll(response)) {
      return null;
    }
    //把response构建成一个Entry对象
    Entry entry = new Entry(response);
    DiskLruCache.Editor editor = null;
    try {
      //生成DiskLruCache.Editor对象
      editor = cache.edit(key(response.request().url()));
      if (editor == null) {
        return null;
      }
      //对缓存进行写入
      entry.writeTo(editor);
      //构建一个CacheRequestImpl类,包含Ok.io的Sink对象
      return new CacheRequestImpl(editor);
    } catch (IOException e) {
      abortQuietly(editor);
      return null;
    }
  }
  • 得到缓存
Response get(Request request) {
    //获取url转换过来的key
    String key = key(request.url());
    DiskLruCache.Snapshot snapshot;
    Entry entry;
    try {
         //根据key获取对应的snapshot 
         snapshot = cache.get(key);
         if (snapshot == null) {
             return null;
         }
    } catch (IOException e) {
      return null;
    }
    try {
     //创建一个Entry对象,并由snapshot.getSource()获取Sink
      entry = new Entry(snapshot.getSource(ENTRY_METADATA));
    } catch (IOException e) {
      Util.closeQuietly(snapshot);
      return null;
    }
    //通过entry和response生成respson,通过Okio.buffer获取请求体,然后封装各种请求信息
    Response response = entry.response(snapshot);
    if (!entry.matches(request, response)) {
      //对request和Response进行比配检查,成功则返回该Response。
      Util.closeQuietly(response.body());
      return null;
    }
    return response;
  }
  • 更新缓存
void update(Response cached, Response network) {
    //用Respon构建一个Entry
    Entry entry = new Entry(network);
    //从缓存中获取DiskLruCache.Snapshot
    DiskLruCache.Snapshot snapshot = ((CacheResponseBody) cached.body()).snapshot;
    DiskLruCache.Editor editor = null;
    try {
      //获取DiskLruCache.Snapshot.edit对象
      editor = snapshot.edit(); // Returns null if snapshot is not current.
      if (editor != null) {
        //将entry写入editor中
        entry.writeTo(editor);
        editor.commit();
      }
    } catch (IOException e) {
      abortQuietly(editor);
    }
  }
  • 删除缓存
void remove(Request request) throws IOException {
    //通过url转化成的key去删除缓存
    cache.remove(key(request.url()));
  }

Cache的"增删改查"大体通过注释代码的方式给出,Cache还有一个更重要的缓存处理类就是DiskLruCache。

DiskLruCache

不仔细看还以为这个类和JakeWharton写的DiskLruCache:[link.jianshu.com/t=https://g…是一样的,其实主体架构差不多,只不过OkHttp的DiskLruCache结合了 Ok.io,用Ok.io处理数据文件的储存.
我们可以看到上面的DiskLruCache有shang三个内部类,分别是Entry,Snapshot,Editor。

Entry

final String key;

    /** Lengths of this entry's files. */
    final long[] lengths;
    final File[] cleanFiles;
    final File[] dirtyFiles;

    /** True if this entry has ever been published. */
    boolean readable;

    /** The ongoing edit or null if this entry is not being edited. */
    Editor currentEditor;

    /** The sequence number of the most recently committed edit to this entry. */
    long sequenceNumber;

    Entry(String key) {
      this.key = key;

      lengths = new long[valueCount];
      cleanFiles = new File[valueCount];
      dirtyFiles = new File[valueCount];

      // The names are repetitive so re-use the same builder to avoid allocations.
      StringBuilder fileBuilder = new StringBuilder(key).append('.');
      int truncateTo = fileBuilder.length();
      for (int i = 0; i < valueCount; i++) {
        fileBuilder.append(i);
        cleanFiles[i] = new File(directory, fileBuilder.toString());
        fileBuilder.append(".tmp");
        dirtyFiles[i] = new File(directory, fileBuilder.toString());
        fileBuilder.setLength(truncateTo);
      }
    }
    
    //省略
    ......

实际上只是用于存储缓存数据的实体类,一个url对应一个实体,在Entry还有Snapshot对象,代码如下:

Snapshot snapshot() {
      if (!Thread.holdsLock(DiskLruCache.this)) throw new AssertionError();

      Source[] sources = new Source[valueCount];
      long[] lengths = this.lengths.clone(); // Defensive copy since these can be zeroed out.
      try {
        for (int i = 0; i < valueCount; i++) {
          sources[i] = fileSystem.source(cleanFiles[i]);
        }
        return new Snapshot(key, sequenceNumber, sources, lengths);
      } catch (FileNotFoundException e) {
        // A file must have been deleted manually!
        for (int i = 0; i < valueCount; i++) {
          if (sources[i] != null) {
            Util.closeQuietly(sources[i]);
          } else {
            break;
          }
        }
        // Since the entry is no longer valid, remove it so the metadata is accurate (i.e. the cache
        // size.)
        try {
          removeEntry(this);
        } catch (IOException ignored) {
        }
        return null;
      }
    }

即一个Entry对应着一个Snapshot对象,在看一下Snapshot的内部代码:

public final class Snapshot implements Closeable {
    private final String key;
    private final long sequenceNumber;
    private final Source[] sources;
    private final long[] lengths;

    Snapshot(String key, long sequenceNumber, Source[] sources, long[] lengths) {
      this.key = key;
      this.sequenceNumber = sequenceNumber;
      this.sources = sources;
      this.lengths = lengths;
    }

    public String key() {
      return key;
    }

    /**
     * Returns an editor for this snapshot's entry, or null if either the entry has changed since
     * this snapshot was created or if another edit is in progress.
     */
    public @Nullable Editor edit() throws IOException {
      return DiskLruCache.this.edit(key, sequenceNumber);
    }

    /** Returns the unbuffered stream with the value for {@code index}. */
    public Source getSource(int index) {
      return sources[index];
    }

    /** Returns the byte length of the value for {@code index}. */
    public long getLength(int index) {
      return lengths[index];
    }

    public void close() {
      for (Source in : sources) {
        Util.closeQuietly(in);
      }
    }
  }

初始化的Snapshot仅仅只是存储了一些变量而已。

Editor

在Editor的初始化中要传入Editor,其实Editor就是编辑entry的类。源码如下:

public final class Editor {
    final Entry entry;
    final boolean[] written;
    private boolean done;

    Editor(Entry entry) {
      this.entry = entry;
      this.written = (entry.readable) ? null : new boolean[valueCount];
    }
  
    void detach() {
      if (entry.currentEditor == this) {
        for (int i = 0; i < valueCount; i++) {
          try {
            fileSystem.delete(entry.dirtyFiles[i]);
          } catch (IOException e) {
            // This file is potentially leaked. Not much we can do about that.
          }
        }
        entry.currentEditor = null;
      }
    }

    //返回指定index的cleanFile的读入流
    public Source newSource(int index) {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (!entry.readable || entry.currentEditor != this) {
          return null;
        }
        try {
          return fileSystem.source(entry.cleanFiles[index]);
        } catch (FileNotFoundException e) {
          return null;
        }
      }
    }
    
    //向指定index的dirtyFiles文件写入数据
    public Sink newSink(int index) {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor != this) {
          return Okio.blackhole();
        }
        if (!entry.readable) {
          written[index] = true;
        }
        File dirtyFile = entry.dirtyFiles[index];
        Sink sink;
        try {
          sink = fileSystem.sink(dirtyFile);
        } catch (FileNotFoundException e) {
          return Okio.blackhole();
        }
        return new FaultHidingSink(sink) {
          @Override protected void onException(IOException e) {
            synchronized (DiskLruCache.this) {
              detach();
            }
          }
        };
      }
    }

    //这里执行的工作是提交数据,并释放锁,最后通知DiskLruCache刷新相关数据
    public void commit() throws IOException {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor == this) {
          completeEdit(this, true);
        }
        done = true;
      }
    }

    //终止编辑,并释放锁
    public void abort() throws IOException {
      synchronized (DiskLruCache.this) {
        if (done) {
          throw new IllegalStateException();
        }
        if (entry.currentEditor == this) {
          completeEdit(this, false);
        }
        done = true;
      }
    }

    //除非正在编辑,否则终止
    public void abortUnlessCommitted() {
      synchronized (DiskLruCache.this) {
        if (!done && entry.currentEditor == this) {
          try {
            completeEdit(this, false);
          } catch (IOException ignored) {
          }
        }
      }
    }
  }

各个方法对应作用如下:

  • Source newSource(int index):返回指定index的cleanFile的读入流
  • Sink newSink(int index):向指定index的dirtyFiles文件写入数据
  • commit():这里执行的工作是提交数据,并释放锁,最后通知DiskLruCache刷新相关数据
  • abort():终止编辑,并释放锁
  • abortUnlessCommitted():除非正在编辑,否则终止

剩下关键来了,还记得上面我们讲Cache添加有一行代码entry.writeTo(editor);,里面操作如下:

 public void writeTo(DiskLruCache.Editor editor) throws IOException {
      BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));

      sink.writeUtf8(url)
          .writeByte('\n');
      sink.writeUtf8(requestMethod)
          .writeByte('\n');
      sink.writeDecimalLong(varyHeaders.size())
          .writeByte('\n');
      for (int i = 0, size = varyHeaders.size(); i < size; i++) {
        sink.writeUtf8(varyHeaders.name(i))
            .writeUtf8(": ")
            .writeUtf8(varyHeaders.value(i))
            .writeByte('\n');
      }

      sink.writeUtf8(new StatusLine(protocol, code, message).toString())
          .writeByte('\n');
      sink.writeDecimalLong(responseHeaders.size() + 2)
          .writeByte('\n');
      for (int i = 0, size = responseHeaders.size(); i < size; i++) {
        sink.writeUtf8(responseHeaders.name(i))
            .writeUtf8(": ")
            .writeUtf8(responseHeaders.value(i))
            .writeByte('\n');
      }
      sink.writeUtf8(SENT_MILLIS)
          .writeUtf8(": ")
          .writeDecimalLong(sentRequestMillis)
          .writeByte('\n');
      sink.writeUtf8(RECEIVED_MILLIS)
          .writeUtf8(": ")
          .writeDecimalLong(receivedResponseMillis)
          .writeByte('\n');

      if (isHttps()) {
        sink.writeByte('\n');
        sink.writeUtf8(handshake.cipherSuite().javaName())
            .writeByte('\n');
        writeCertList(sink, handshake.peerCertificates());
        writeCertList(sink, handshake.localCertificates());
        sink.writeUtf8(handshake.tlsVersion().javaName()).writeByte('\n');
      }
      sink.close();
    }

上面的都是Ok.io的操作了,不懂OK.io的可以去看一下相关知识。BufferedSink sink = Okio.buffer(editor.newSink(ENTRY_METADATA));editor.newSink拿到ok.io版的OutputStream(Sink)生成Ok.io的输入类,剩下的就是把数据用ok.io写入文件,然后关闭输出类。

同理我看们可以一下上面Cache获取缓存的代码 Response response = entry.response(snapshot);,在response方法里又有一个方法:CacheResponseBody()就是获取缓存的方法,代码如下:

 CacheResponseBody(final DiskLruCache.Snapshot snapshot,String contentType, String contentLength) {
      this.snapshot = snapshot;
      this.contentType = contentType;
      this.contentLength = contentLength;

      Source source = snapshot.getSource(ENTRY_BODY);
      bodySource = Okio.buffer(new ForwardingSource(source) {
        @Override public void close() throws IOException {
          snapshot.close();
          super.close();
        }
      });
    }

new ForwardingSource(source)相当于传入ok.io版的InputStream(Source)生成Ok.io的读取类,剩下的都是读取缓存数据然后生成Response.

而上面Cache的Update()方法,其写入过程也和上面的添加是一样的,不同的只不过先构造成一个就得Entry然后再把新的缓存写上去更新而已,因为涉及我重要的Ok.io是一样的,所以不细讲。

剩下就是删除了,在Cache的delete方法里,在removeEntry就是执行删除操作,代码如下:

 boolean removeEntry(Entry entry) throws IOException {
  
    //省略

    journalWriter.writeUtf8(REMOVE).writeByte(' ').writeUtf8(entry.key).writeByte('\n');
    lruEntries.remove(entry.key);

    //省略
    return true;
  }

上面这两句代码就是删除的关键, journalWriter.writeUtf8表示在DiskLruCache的本地缓存清单列表里删除,lruEntries.remove表示在缓存内存里删除。

到此增删给查的流程基本结束,其实DiskLruCache还有很多可以讲,但是我的重心是OKhttp的缓存底层是用Ok.io,为此在这里点到为止。

内容有点多,如有错误请多多指出