- 由于业务需求扩容,现在图片量很大,分布式搭建fastDFS,那么和大家一起总结一下踩的坑。
下载fastDFS
开启华为云远程端口(如图:)
- 在这里顺便把后面tracker和storage ,网络访问端口一起放行吧(如图:)
# cd /usr
# mkdir download
# cd download
# wget https://github.com/happyfish100/libfastcommon/archive/V1.0.7.tar.gz
创建好目录下载fastDFS,安装下面的命令解压
# tar -zxvf V1.0.7.tar.gz
# cd libfastcommon-1.0.7
cd进入libfastcommon-1.0.7目录,然后对其进行编译和安装:
# ./make.sh
# ./make.sh install
安装好了后可以看到 libfastcommon.so ,是在/usr/lib64/libfastcommon.so下面(如图,我这是用宝塔面板看得哈ps可能更直观些吧)
# ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
# ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so
接下来就是见证奇迹的时候了安装fastDFS
# cd /usr/download/
# wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz
接下来解压安装
# tar -zxvf V5.05.tar.gz
# cd fastdfs-5.05
看到有 make.sh这是执行文件
# ./make.sh
# ./make.sh install
使用默认安装后可以看到有几个相应的文件:
# cd /etc/init.d
# ll
-rwxr-xr-x 1 root root 1186 Oct 30 10:18 fdfs_storaged
-rwxr-xr-x 1 root root 1186 Oct 30 10:18 fdfs_trackerd
服务脚本文件:
/etc/init.d/fdfs_storaged
/etc/init.d/fdfs_tracker
配置文件:
# cd /etc/fdfs
# ls
client.conf.sample storage.conf.sample tracker.conf.sample
修改命令路径,由于FastDFS 服务脚本设置的 bin 目录是 /usr/local/bin, 但实际命令安装在 /usr/bin/ 下,所以,创建个软连接吧。
# ln -s /usr/bin/fdfs_trackerd /usr/local/bin
# ln -s /usr/bin/fdfs_storaged /usr/local/bin
# ln -s /usr/bin/stop.sh /usr/local/bin
# ln -s /usr/bin/restart.sh /usr/local/bin
当然,也可以把 /etc/init.d/fdfs_storaged 和 /etc/init.d/fdfs_tracker 两个脚本中的 /usr/local/bin 修改成 /usr/bin。
# cd /etc/fdfs
# cp tracker.conf.sample tracker.conf
# vim tracker.conf
编辑tracker.conf,这里比较重要了。
# vim tracker.conf
is this config file disabled
# false for enabled
# true for disabled
# 配置文件是否不生效,false 为生效
disabled=false
# the tracker server port
# 提供服务的端口
port=22122
# the base path to store data and log files
# Tracker 数据和日志目录地址
base_path=/fastdfs/tracker
# HTTP port on this tracker server
# HTTP 服务端口
http.server_port=80
# cd /
# mkdir fastdfs
# cd fastdfs
# mkdir tracker
# mkdir storage
# mkdir file
# mkdir client
# ls
client file storage tracker
启动Tracker:
# service fdfs_trackerd start
查看 FastDFS Tracker 是否已成功启动 ,22122端口正在被监听,则算是Tracker服务安装成功:
# netstat -unltp|grep fdfs
看到如下结果
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 20435/fdfs_storaged
tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 19190/fdfs_trackerd
# service fdfs_trackerd stop
我只是说说关闭的命令,别在此真的关了!
tracker server 启动成功后,会在你创建的base_path目录下创建data和log两个目录 ,会有如下结果
然后配置 FastDFS 存储 (Storage):
进入 /etc/fdfs 目录,复制 FastDFS 存储器样例配置文件 storage.conf.sample,并重命名为 storage.conf。
# cd /etc/fdfs
# cp storage.conf.sample storage.conf
# vim storage.conf
编辑storage.conf:
# 配置文件是否不生效,false 为生效
disabled=false
# 指定此 storage server 所在 组(卷)
group_name=group1
# storage server 服务端口
port=23000
# Storage 数据和日志目录地址(根目录必须存在,子目录会自动生成)
base_path=/fastdfs/storage
# 逐一配置 store_path_count 个路径,索引号基于 0。
# 如果不配置 store_path0,那它就和 base_path 对应的路径一样。
store_path0=/fastdfs/file
# tracker_server 的列表 ,会主动连接 tracker_server
# 有多个 tracker server 时,每个 tracker server 写一行
tracker_server=120.79.178.190:22122
# 访问端口
http.server_port=80
base_path就是前面一起创建的storage目录拉,自己配置吧。base_path0对应的file目录。
重点来啦:tracker_server=120.79.178.190:22122
这里填写你的外网ip地址,确保阿里云服务器自己给自己开放了权限(同一台服务器上装tracker和storage)。
配置好了保存!
然后启动 Storage:
# service fdfs_storaged start
用命令查看是否监听成功:# netstat -unltp|grep fdfs
tcp 0 0 0.0.0.0:23000 0.0.0.0:* LISTEN 20435/fdfs_storaged
tcp 0 0 0.0.0.0:22122 0.0.0.0:* LISTEN 24200/fdfs_trackerd
如果出现了上面所示,则表明你成功了!如果没出现,肯定就是端口号没开放,或者ip地址配置配置错了,或者没给华为云服务器自己授权!
关闭Storage命令:
# service fdfs_storaged stop
不要关!
查看Storage和Tracker是否在通信:
#/usr/bin/fdfs_monitor /etc/fdfs/storage.conf
出现了下面就表示真的通信成功啦!
在 store_path0 =/fastdfs/file目录下,创建了N*N个子目录(如图):
修改 Tracker 服务器中的客户端配置文件:
# cd /etc/fdfs
# cp client.conf.sample client.conf
# vim client.conf
# Client 的数据和日志目录
base_path=/fastdfs/client
# Tracker端口(华为云服务器公网IP地址)
tracker_server=120.79.172.193:22122
然后安装nginx提供访问:
安装nginx需要前置环境,我们一步一步安装:
# yum install gcc-c++
# yum install -y pcre pcre-devel
# yum install -y zlib zlib-devel
# yum install -y openssl openssl-devel
然后安装nginx:
# cd /usr/download/
# wget -c https://nginx.org/download/nginx-1.12.1.tar.gz
把所有下载的东西都放在一个目录下,好统一管理,安装完就使用rm -rf *都给删了。免得占空间!
然后解压nginx:
# tar -zxvf nginx-1.12.1.tar.gz
# cd nginx-1.12.1
使用默认配置:
# ./configure
编译、安装:
# make
# make install
启动nginx:
# cd /usr/local/nginx/sbin/
# ./nginx
其它命令
# ./nginx -s stop
# ./nginx -s quit
# ./nginx -s reload
查看nginx的版本及模块:
# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.12.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC)
configure arguments:
修改nginx.conf:
# vim /usr/local/nginx/conf/nginx.conf
添加如下行,将 /group1/M00 映射到 /fastdfs/file/data
location /group1/M00 {
alias /fastdfs/file/data;
}
# 重启nginx
# /usr/local/nginx/sbin/nginx -s reload
SpringBoot java代码实现
pom加入依赖包
<!--fastDFS架包-->
<dependency>
<groupId>net.oschina.zcx7878</groupId>
<artifactId>fastdfs-client-java</artifactId>
<version>1.27.0.0</version>
</dependency>
application.properties 配置上传文件大小
#上传fastDFS上传图片限制大小
#最大支持文件大小
spring.servlet.multipart.maxFileSize=104857600
#最大支持请求大小
spring.servlet.multipart.maxRequestSize=104857600
PS 如果写成这样会报错
org.apache.tomcat.util.http.fileupload.FileUploadBase$FileSizeLimitExceededException: The field file exceeds its maximum permitted size of 1048576 bytes.
at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl$FileItemStreamImpl$1.raiseError(FileUploadBase.java:633)
at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.checkLimit(LimitedInputStream.java:76)
at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:135)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:98)
at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:294)
at org.apache.catalina.connector.Request.parseParts(Request.java:2868)
at org.apache.catalina.connector.Request.getParts(Request.java:2770)
at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1098)
at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:95)
at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.<init>(StandardMultipartHttpServletRequest.java:88)
at org.springframework.web.multipart.support.StandardServletMultipartResolver.resolveMultipart(StandardServletMultipartResolver.java:87)
at org.springframework.web.servlet.DispatcherServlet.checkMultipart(DispatcherServlet.java:1178)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1012)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:113)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at com.jjmy.qingmu.config.filter.JWTAuthorizationFilter.doFilterInternal(JWTAuthorizationFilter.java:48)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:745)
Controller层代码
//上传图片
@ApiOperation("后台用户上传图片,本次方法使用fastDFS")
@ApiImplicitParam(name = "file", value = "图片上传参数名称")
@RequestMapping(value = "/uploadImg", method = RequestMethod.POST)
public ResultResponse uploadImg(@RequestParam("file") MultipartFile file) throws IOException {
return ResultResponse.returnToken(backgroundManageService.uploadImg(file));
}
Service层代码
//图片上传
public ResultResponse uploadImg(MultipartFile file) throws IOException {
long startTime = System.currentTimeMillis();
try {
//加载fastdfs-client.properties配置文件
ClientGlobal.initByProperties("config/fastdfs-client.properties");
//定义TrackerClient,用于请求TrackerServer
TrackerClient trackerClient = new TrackerClient();
//得到storage服务器
TrackerServer trackerServer = trackerClient.getConnection();
StorageServer storeStorage = trackerClient.getStoreStorage(trackerServer);
//创建storageClient来上传文件
StorageClient1 storageClient1 = new StorageClient1(trackerServer, storeStorage);
//上传文件
//得到文件字节
byte[] bytes = file.getBytes();
//得到文件的原始名称
String originalFilename = file.getOriginalFilename();
//得到文件扩展名
String ext = originalFilename.substring(originalFilename.lastIndexOf(".") + 1);
String fileId = storageClient1.upload_file1(bytes, ext, null);
String uploadPath = "http://99.99.238.179/" + fileId;
logger.info("图片上传时间为[{}ms]", System.currentTimeMillis() - startTime);
return ResultResponse.success(uploadPath);
} catch (Exception e) {
e.printStackTrace();
logger.error("图片上传出现异常[{}ms]", System.currentTimeMillis() - startTime, e.getMessage());
return ResultResponse.fail("图片上传出现异常!");
}
}
- 有和前端交互只需要给前端返回url
fastdfs-client.properties 配置文件
fastdfs.connect_timeout_in_seconds = 5
fastdfs.network_timeout_in_seconds = 30
fastdfs.charset = UTF-8
fastdfs.tracker_servers = 99.99.238.179:22122
整个工程工程目录结构
用PostMan测试返回结构
SpringBoot+fastDFS搭建和测试就告一段落了,有问题大家可以一起交流
PS:如果大家在linux上搭建还有问题,可以参考这篇博客 blog.csdn.net/busishenren…