restTemplate文件上传与下载

前言

上一章写了个文件上传的form表单解析器，但是有时候需要文件上传透传，当然也可以使用分布式文件系统解决这个问题，只是很多时候文件上传只是一个小功能，但是又不可或缺。其实文件上传下载可以通过restTemplate来实现，可以通过文件流的方式或者临时文件转发，推荐文件流，避免写文件清理的过程。

1. restTemplate

restTemplate实际上是使用execute方法，随意看一个方法，最终调用execute方法，其他类似

跟踪execute方法，doExecute

	protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
			@Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {

		Assert.notNull(url, "URI is required");
		Assert.notNull(method, "HttpMethod is required");
		ClientHttpResponse response = null;
		try {
			ClientHttpRequest request = createRequest(url, method);
			if (requestCallback != null) {
				requestCallback.doWithRequest(request);
			}
			response = request.execute();
			handleResponse(url, method, response);
			return (responseExtractor != null ? responseExtractor.extractData(response) : null);
		}
		catch (IOException ex) {
			String resource = url.toString();
			String query = url.getRawQuery();
			resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
			throw new ResourceAccessException("I/O error on " + method.name() +
					" request for \"" + resource + "\": " + ex.getMessage(), ex);
		}
		finally {
            //这里要注意，如果是流，必须重写这个方法，或者在callback里面处理流
            //下载这里就不能直接输出
			if (response != null) {
				response.close();
			}
		}
	}

可以看到restTemplate的源码很简单。

2. 文件上传

2.1 临时文件方式

临时文件会在应用的磁盘写一个临时文件，需要在上传结束后删除，不然文件会越来越多，占用磁盘空间影响应用。

    @ResponseBody
    @RequestMapping("/upload")
    public String upload(String arg, MultipartFile file1) throws IOException {
        restTemplateService.postForEntity(arg, file1);
        return "{\"upload\":\"success\"}";
    }


    public String postForEntity(String arg, MultipartFile file) throws IOException {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.MULTIPART_FORM_DATA);
        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        body.add("arg", arg);

        File tempFile = new File("/Users/huahua/logs/"+file.getOriginalFilename());
        if (tempFile.exists()) {
            tempFile.delete();
        }
        tempFile.createNewFile();

        try (FileOutputStream outputStream = new FileOutputStream(tempFile);
             InputStream inputStream = file.getInputStream()) {
            StreamUtils.copy(inputStream, outputStream);
        }

        FileSystemResource fileSystemResource = new FileSystemResource(tempFile);
        body.add("file1", fileSystemResource);
        HttpEntity httpEntity = new HttpEntity(body, headers);
        return restTemplate.postForEntity("http://localhost:8383/upload", httpEntity, String.class).getBody();
        //finally 代码删除临时文件，省略
    }

执行上传调用，HttpServer使用上一章的自己写的server 

 临时文件，上传目录文件

2.2 去除临时文件

每次上传写一个临时文件，明显不合理，磁盘占用，读写流，写磁盘的开销还是很大的，那么如何节省开销，使用Spring的

InputStreamResource

但是InputStreamResource只能传输流，而文件名与文件大小只能读取流获取，流显然只能读取一次比较好,InputStreamResource的getfilename与contentLength方法继承了

AbstractResource

见源码，尤其是contentLength是必须的，不然怎么知道流的那部分是文件的结尾呢，然后getInputStream();读取完就关闭流了

    public long contentLength() throws IOException {
		InputStream is = getInputStream();
		try {
			long size = 0;
			byte[] buf = new byte[256];
			int read;
			while ((read = is.read(buf)) != -1) {
				size += read;
			}
			return size;
		}
		finally {
			try {
				is.close();
			}
			catch (IOException ex) {
				Log logger = LogFactory.getLog(getClass());
				if (logger.isDebugEnabled()) {
					logger.debug("Could not close content-length InputStream for " + getDescription(), ex);
				}
			}
		}
	}

而且没有文件名，那个文件是什么文件，多个文件怎么区分

    public String getFilename() {
		return null;
	}

InputStreamResource的设计就是一个单次读取的，如果要实现这些就需要定制

而其他实现确处理了，比如

FileSystemResource

	public long contentLength() throws IOException {
		if (this.file != null) {
			long length = this.file.length();
			if (length == 0L && !this.file.exists()) {
				throw new FileNotFoundException(getDescription() +
						" cannot be resolved in the file system for checking its content length");
			}
			return length;
		}
		else {
			try {
				return Files.size(this.filePath);
			}
			catch (NoSuchFileException ex) {
				throw new FileNotFoundException(ex.getMessage());
			}
		}
	}


	public String getFilename() {
		return (this.file != null ? this.file.getName() : this.filePath.getFileName().toString());
	}

仿照这个，自己实现一个不就完美解决问题了，😄，来开干

public class UploadResource extends InputStreamResource {
    
    private long contentLength;
    private String filename;
    
    public UploadResource(InputStream inputStream) {
        super(inputStream);
    }
    
    public UploadResource(InputStream inputStream, long contentLength, String filename) {
        super(inputStream);
        this.contentLength = contentLength;
        this.filename = filename;
    }

    @Override
    public long contentLength() throws IOException {
        return this.contentLength;
    }

    @Override
    public String getFilename() {
        return this.filename;
    }
}

在修改

    public String postForEntity(String arg, MultipartFile file) throws IOException {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.MULTIPART_FORM_DATA);
        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        body.add("arg", arg);

        UploadResource inputStreamResource = new UploadResource(file.getInputStream(), file.getSize(), file.getOriginalFilename());
        body.add("file1", inputStreamResource);
        HttpEntity httpEntity = new HttpEntity(body, headers);
        return restTemplate.postForEntity("http://localhost:8383/upload", httpEntity, String.class).getBody();
    }

上传成功，临时文件没有写，所以节省磁盘，节省时间。

 3. 文件下载

3.1 一次性读取下载

    @RequestMapping("/download")
    public ResponseEntity download(String path) throws IOException {
        byte[] body = restTemplateService.getForEntity(path);
        ResponseEntity responseEntity = ResponseEntity.ok()
                .header("Content-Disposition", "attachment;filename=closeSmps.jpg")//path可以获取文件名
                .contentType(MediaType.APPLICATION_OCTET_STREAM)
                .contentLength(body.length)
                .body(body);
        return responseEntity;
    }

    public byte[] getForEntity(String path) throws IOException {
        return restTemplate.getForEntity("http://localhost:8383/download", byte[].class).getBody();
    }

一次性将文件读到内存，然后返回，适合小文件传输，大文件内存绝对hold不住 

这里使用

ResourceHttpMessageConverter

解析返回，Spring boot 默认加载，如果没有需要在restTemplate加入解析对象

ResourceHttpMessageConverter fastConverter = new ResourceHttpMessageConverter();
restTemplate.getMessageConverters().add(fastConverter);

3.2 流下载

    @RequestMapping("/download")
    public void download(String path, HttpServletResponse response) throws IOException, URISyntaxException {
        restTemplateService.getForEntity(path, response);
    }

    public void getForEntity(String path, HttpServletResponse response) throws IOException, URISyntaxException {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        HttpEntity httpEntity = new HttpEntity(body, headers);
        RequestCallback requestCallback = restTemplate.httpEntityCallback(httpEntity);
        restTemplate.execute(new URI("http://localhost:8383/download"), HttpMethod.GET, requestCallback, (clientResponse) -> {
            InputStream in = clientResponse.getBody();
            response.setHeader("Content-Disposition", "attachment;filename=closeSmps.jpg");//path可以获取文件名
//            httpExchange.sendResponseHeaders(200, file.length());

            OutputStream out = response.getOutputStream();
            StreamUtils.copy(in, out);

            out.flush();
            out.close();
            in.close();
            return null;
        });
    }

其实就是回调的时候像外写入文件，浏览器就能下载了。 

3.3 流直接返回处理

为什么没有直接返回ResponseEntity，是由于前面分析的template在返回时，关闭response流，download需要特别定制

@Configuration
public class RestConfig {

    @Bean
    @Primary
    public RestTemplate initRestTemplate(){
        SimpleClientHttpRequestFactory factory=new SimpleClientHttpRequestFactory();
        factory.setConnectTimeout(5000);
        factory.setReadTimeout(30000);
        RestTemplate restTemplate = new RestTemplate(factory);
        //可以自定义解析器，处理特定的数据
        FormHttpMessageConverter fastConverter = new FormHttpMessageConverter();
        restTemplate.getMessageConverters().add(fastConverter);
        return restTemplate;
    }
    @Bean
    @Qualifier("downloadRestTemplate")
    public DownloadRestTemplate initRestTemplateDownload(){
        SimpleClientHttpRequestFactory factory=new SimpleClientHttpRequestFactory();
        factory.setConnectTimeout(5000);
        factory.setReadTimeout(30000);
        DownloadRestTemplate restTemplate = new DownloadRestTemplate(factory);
        //可以自定义解析器，处理特定的数据
        FormHttpMessageConverter fastConverter = new FormHttpMessageConverter();
        restTemplate.getMessageConverters().add(fastConverter);
        return restTemplate;
    }
}

Qualifier表示对象的定制，@Value表示基本数据类型和String

public class DownloadRestTemplate extends RestTemplate {
    public DownloadRestTemplate(ClientHttpRequestFactory requestFactory) {
        super(requestFactory);
    }

    protected <T> T doExecute(URI url, @Nullable HttpMethod method, @Nullable RequestCallback requestCallback,
                              @Nullable ResponseExtractor<T> responseExtractor) throws RestClientException {

        Assert.notNull(url, "URI is required");
        Assert.notNull(method, "HttpMethod is required");
        ClientHttpResponse response = null;
        try {
            ClientHttpRequest request = createRequest(url, method);
            if (requestCallback != null) {
                requestCallback.doWithRequest(request);
            }
            response = request.execute();
            handleResponse(url, method, response);
            return (responseExtractor != null ? responseExtractor.extractData(response) : null);
        }
        catch (IOException ex) {
            String resource = url.toString();
            String query = url.getRawQuery();
            resource = (query != null ? resource.substring(0, resource.indexOf('?')) : resource);
            throw new ResourceAccessException("I/O error on " + method.name() +
                    " request for \"" + resource + "\": " + ex.getMessage(), ex);
        }
        /*finally {
            if (response != null) {
                response.close();
            }
        }*/
    }
}

仅用于download，直接处理好结果

    @Autowired
    @Qualifier("downloadRestTemplate")
    private DownloadRestTemplate downloadRestTemplate;
    public ResponseEntity getForEntityForReturn(String path) throws IOException, URISyntaxException {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_FORM_URLENCODED);
        MultiValueMap<String, Object> body = new LinkedMultiValueMap<>();
        HttpEntity httpEntity = new HttpEntity(body, headers);
        RequestCallback requestCallback = restTemplate.httpEntityCallback(httpEntity);
        return downloadRestTemplate.execute(new URI("http://localhost:8383/download"), HttpMethod.GET, requestCallback, (clientResponse) -> {
            InputStream in = clientResponse.getBody();
            long length = clientResponse.getHeaders().getContentLength();
            return ResponseEntity.ok()
                    .header("Content-Disposition", "attachment;filename=closeSmps.jpg")//path可以获取文件名
                    .contentType(MediaType.APPLICATION_OCTET_STREAM)
                    .contentLength(length)
                    .body(new UploadResource(in, length, "closeSmps.jpg"));
        });
    }

关闭流交给Spring框架吧 ，有点类似C++编程，前面定义，后面回收资源

    @RequestMapping("/download")
    public ResponseEntity download(String path) throws IOException, URISyntaxException {
        return restTemplateService.getForEntityForReturn(path);
    }

下载成功😄 

总结

文件上传一般是直接上传即可，上一章也说了解析form表单的信息，Spring解析也差不多，估计更高效，或者文件直接上传文件服务器，但是有时候需要透传文件，这个时候就需要了解文件的传输原理，协议的传递。 这里是http协议form传输，也可以使用sftp协议，那就是另外的做法了。