华为云Centos7搭建fastDFS图片上传

1,130 阅读7分钟
  • 由于业务需求扩容,现在图片量很大,分布式搭建fastDFS,那么和大家一起总结一下踩的坑。

下载fastDFS

开启华为云远程端口(如图:)

  • 在这里顺便把后面tracker和storage ,网络访问端口一起放行吧(如图:)

现在可以XSell远程连接服务器吧 进入命令行界面,输入pwd 看看所在的单前目录,在 cd /进入根目录下,我在usr的目录下创建了download来存放所下载的资料。

# cd /usr
 
# mkdir download
 
# cd download
 
# wget https://github.com/happyfish100/libfastcommon/archive/V1.0.7.tar.gz

创建好目录下载fastDFS,安装下面的命令解压

# tar -zxvf V1.0.7.tar.gz
# cd libfastcommon-1.0.7

cd进入libfastcommon-1.0.7目录,然后对其进行编译和安装:

# ./make.sh
# ./make.sh install

安装好了后可以看到 libfastcommon.so ,是在/usr/lib64/libfastcommon.so下面(如图,我这是用宝塔面板看得哈ps可能更直观些吧)

因为FastDFS主程序设置的lib目录是/usr/local/lib,要创建软连接

# ln -s /usr/lib64/libfastcommon.so /usr/local/lib/libfastcommon.so
# ln -s /usr/lib64/libfastcommon.so /usr/lib/libfastcommon.so
# ln -s /usr/lib64/libfdfsclient.so /usr/local/lib/libfdfsclient.so
# ln -s /usr/lib64/libfdfsclient.so /usr/lib/libfdfsclient.so 

接下来就是见证奇迹的时候了安装fastDFS

# cd /usr/download/
# wget https://github.com/happyfish100/fastdfs/archive/V5.05.tar.gz

接下来解压安装

# tar -zxvf V5.05.tar.gz
# cd fastdfs-5.05

看到有 make.sh这是执行文件

# ./make.sh
# ./make.sh install

使用默认安装后可以看到有几个相应的文件:

# cd /etc/init.d
# ll
-rwxr-xr-x  1 root root  1186 Oct 30 10:18 fdfs_storaged
-rwxr-xr-x  1 root root  1186 Oct 30 10:18 fdfs_trackerd

服务脚本文件:

/etc/init.d/fdfs_storaged
/etc/init.d/fdfs_tracker

配置文件:

# cd /etc/fdfs
# ls
client.conf.sample   storage.conf.sample   tracker.conf.sample

修改命令路径,由于FastDFS 服务脚本设置的 bin 目录是 /usr/local/bin, 但实际命令安装在 /usr/bin/ 下,所以,创建个软连接吧。

# ln -s /usr/bin/fdfs_trackerd   /usr/local/bin
# ln -s /usr/bin/fdfs_storaged   /usr/local/bin
# ln -s /usr/bin/stop.sh         /usr/local/bin
# ln -s /usr/bin/restart.sh      /usr/local/bin

当然,也可以把 /etc/init.d/fdfs_storaged 和 /etc/init.d/fdfs_tracker 两个脚本中的 /usr/local/bin 修改成 /usr/bin。

上面图片是用vim 编辑器进入的 然后开始配置tracker了,配置FastDFS跟踪器(Tracker):

# cd /etc/fdfs
# cp tracker.conf.sample tracker.conf
# vim tracker.conf

编辑tracker.conf,这里比较重要了。

# vim tracker.conf
 
 is this config file disabled
# false for enabled
# true for disabled
# 配置文件是否不生效,false 为生效
disabled=false
 
# the tracker server port
# 提供服务的端口
port=22122
 
# the base path to store data and log files
# Tracker 数据和日志目录地址
base_path=/fastdfs/tracker
 
# HTTP port on this tracker server
# HTTP 服务端口
http.server_port=80

要特别注意这里的端口要配置正确,不然后面用不了 上面只展示了4个比较重要的信息。base_path地址属于自己要放fast相关数据的目录文件地址,可自己自定义。 首先“cd /”进入根目录,然后“mkdir fastdfs”,然后“cd fastdfs”,再创建“mkdir tracker”。

# cd /
# mkdir fastdfs
# cd fastdfs
# mkdir tracker
# mkdir storage
# mkdir file  
# mkdir client  
# ls
client  file  storage  tracker

启动Tracker:

# service fdfs_trackerd start

查看 FastDFS Tracker 是否已成功启动 ,22122端口正在被监听,则算是Tracker服务安装成功:

# netstat -unltp|grep fdfs

看到如下结果

tcp        0      0 0.0.0.0:23000           0.0.0.0:*               LISTEN      20435/fdfs_storaged 
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      19190/fdfs_trackerd 

如果看到上面显示的,则表明你tracker安装成功啦,端口号也打开啦。 关闭Tracker命令:

# service fdfs_trackerd stop

我只是说说关闭的命令,别在此真的关了!

tracker server 启动成功后,会在你创建的base_path目录下创建data和log两个目录 ,会有如下结果

在logs里面有个trackerd.log文件,打开如下

然后配置 FastDFS 存储 (Storage):

进入 /etc/fdfs 目录,复制 FastDFS 存储器样例配置文件 storage.conf.sample,并重命名为 storage.conf。

# cd /etc/fdfs
# cp storage.conf.sample storage.conf
# vim storage.conf

编辑storage.conf:

# 配置文件是否不生效,false 为生效
disabled=false 
 
# 指定此 storage server 所在 组(卷)
group_name=group1
 
# storage server 服务端口
port=23000
 
# Storage 数据和日志目录地址(根目录必须存在,子目录会自动生成)
base_path=/fastdfs/storage
 
# 逐一配置 store_path_count 个路径,索引号基于 0。
# 如果不配置 store_path0,那它就和 base_path 对应的路径一样。
store_path0=/fastdfs/file
 
# tracker_server 的列表 ,会主动连接 tracker_server
# 有多个 tracker server 时,每个 tracker server 写一行
tracker_server=120.79.178.190:22122
 
# 访问端口
http.server_port=80

base_path就是前面一起创建的storage目录拉,自己配置吧。base_path0对应的file目录。

重点来啦:tracker_server=120.79.178.190:22122

这里填写你的外网ip地址,确保阿里云服务器自己给自己开放了权限(同一台服务器上装tracker和storage)。

配置好了保存!

然后启动 Storage:
# service fdfs_storaged start

用命令查看是否监听成功:# netstat -unltp|grep fdfs

tcp        0      0 0.0.0.0:23000           0.0.0.0:*               LISTEN      20435/fdfs_storaged 
tcp        0      0 0.0.0.0:22122           0.0.0.0:*               LISTEN      24200/fdfs_trackerd 

如果出现了上面所示,则表明你成功了!如果没出现,肯定就是端口号没开放,或者ip地址配置配置错了,或者没给华为云服务器自己授权!

关闭Storage命令:
# service fdfs_storaged stop

不要关!

查看Storage和Tracker是否在通信:
#/usr/bin/fdfs_monitor /etc/fdfs/storage.conf

出现了下面就表示真的通信成功啦!

Storage 启动成功后会自动在/fastdfs/storage目录下创建data和log两个目录。

在 store_path0 =/fastdfs/file目录下,创建了N*N个子目录(如图):

修改 Tracker 服务器中的客户端配置文件:
# cd /etc/fdfs
# cp client.conf.sample client.conf
# vim client.conf
# Client 的数据和日志目录
base_path=/fastdfs/client
 
# Tracker端口(华为云服务器公网IP地址)
tracker_server=120.79.172.193:22122
然后安装nginx提供访问:

安装nginx需要前置环境,我们一步一步安装:


# yum install gcc-c++
# yum install -y pcre pcre-devel
# yum install -y zlib zlib-devel
# yum install -y openssl openssl-devel

然后安装nginx:

# cd /usr/download/
# wget -c https://nginx.org/download/nginx-1.12.1.tar.gz

把所有下载的东西都放在一个目录下,好统一管理,安装完就使用rm -rf *都给删了。免得占空间!

然后解压nginx:

# tar -zxvf nginx-1.12.1.tar.gz
# cd nginx-1.12.1

使用默认配置:

# ./configure

编译、安装:

# make
# make install

启动nginx:

# cd /usr/local/nginx/sbin/
# ./nginx 
 
其它命令
# ./nginx -s stop
# ./nginx -s quit
# ./nginx -s reload

查看nginx的版本及模块:

# /usr/local/nginx/sbin/nginx -V
nginx version: nginx/1.12.1
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-39) (GCC) 
configure arguments:

修改nginx.conf:

# vim /usr/local/nginx/conf/nginx.conf
 
添加如下行,将 /group1/M00 映射到 /fastdfs/file/data
location /group1/M00 {
    alias /fastdfs/file/data;
}
 
# 重启nginx
# /usr/local/nginx/sbin/nginx -s reload

SpringBoot java代码实现

pom加入依赖包
    <!--fastDFS架包-->
        <dependency>
            <groupId>net.oschina.zcx7878</groupId>
            <artifactId>fastdfs-client-java</artifactId>
            <version>1.27.0.0</version>
        </dependency>
application.properties 配置上传文件大小
#上传fastDFS上传图片限制大小
#最大支持文件大小
spring.servlet.multipart.maxFileSize=104857600
#最大支持请求大小
spring.servlet.multipart.maxRequestSize=104857600

PS 如果写成这样会报错

org.apache.tomcat.util.http.fileupload.FileUploadBase$FileSizeLimitExceededException: The field file exceeds its maximum permitted size of 1048576 bytes.
	at org.apache.tomcat.util.http.fileupload.FileUploadBase$FileItemIteratorImpl$FileItemStreamImpl$1.raiseError(FileUploadBase.java:633)
	at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.checkLimit(LimitedInputStream.java:76)
	at org.apache.tomcat.util.http.fileupload.util.LimitedInputStream.read(LimitedInputStream.java:135)
	at java.io.FilterInputStream.read(FilterInputStream.java:107)
	at org.apache.tomcat.util.http.fileupload.util.Streams.copy(Streams.java:98)
	at org.apache.tomcat.util.http.fileupload.FileUploadBase.parseRequest(FileUploadBase.java:294)
	at org.apache.catalina.connector.Request.parseParts(Request.java:2868)
	at org.apache.catalina.connector.Request.getParts(Request.java:2770)
	at org.apache.catalina.connector.RequestFacade.getParts(RequestFacade.java:1098)
	at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
	at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
	at javax.servlet.http.HttpServletRequestWrapper.getParts(HttpServletRequestWrapper.java:356)
	at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.parseRequest(StandardMultipartHttpServletRequest.java:95)
	at org.springframework.web.multipart.support.StandardMultipartHttpServletRequest.<init>(StandardMultipartHttpServletRequest.java:88)
	at org.springframework.web.multipart.support.StandardServletMultipartResolver.resolveMultipart(StandardServletMultipartResolver.java:87)
	at org.springframework.web.servlet.DispatcherServlet.checkMultipart(DispatcherServlet.java:1178)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1012)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:909)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:113)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:126)
	at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:90)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:118)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:137)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:111)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter.doFilter(SecurityContextHolderAwareRequestFilter.java:158)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:63)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at com.jjmy.qingmu.config.filter.JWTAuthorizationFilter.doFilterInternal(JWTAuthorizationFilter.java:48)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:200)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:116)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.web.filter.CorsFilter.doFilterInternal(CorsFilter.java:92)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.header.HeaderWriterFilter.doHeadersAfter(HeaderWriterFilter.java:92)
	at org.springframework.security.web.header.HeaderWriterFilter.doFilterInternal(HeaderWriterFilter.java:77)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter.doFilterInternal(WebAsyncManagerIntegrationFilter.java:56)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
	at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
	at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
	at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
	at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:526)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:861)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1579)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
	at java.lang.Thread.run(Thread.java:745)
Controller层代码
 //上传图片
    @ApiOperation("后台用户上传图片,本次方法使用fastDFS")
    @ApiImplicitParam(name = "file", value = "图片上传参数名称")
    @RequestMapping(value = "/uploadImg", method = RequestMethod.POST)
    public ResultResponse uploadImg(@RequestParam("file") MultipartFile file) throws IOException {
        return ResultResponse.returnToken(backgroundManageService.uploadImg(file));
    }
Service层代码
  //图片上传
    public ResultResponse uploadImg(MultipartFile file) throws IOException {
        long startTime = System.currentTimeMillis();
        try {
            //加载fastdfs-client.properties配置文件
            ClientGlobal.initByProperties("config/fastdfs-client.properties");
            //定义TrackerClient,用于请求TrackerServer
            TrackerClient trackerClient = new TrackerClient();
            //得到storage服务器
            TrackerServer trackerServer = trackerClient.getConnection();

            StorageServer storeStorage = trackerClient.getStoreStorage(trackerServer);
            //创建storageClient来上传文件
            StorageClient1 storageClient1 = new StorageClient1(trackerServer, storeStorage);
            //上传文件
            //得到文件字节
            byte[] bytes = file.getBytes();
            //得到文件的原始名称
            String originalFilename = file.getOriginalFilename();
            //得到文件扩展名
            String ext = originalFilename.substring(originalFilename.lastIndexOf(".") + 1);
            String fileId = storageClient1.upload_file1(bytes, ext, null);
            String uploadPath = "http://99.99.238.179/" + fileId;
            logger.info("图片上传时间为[{}ms]", System.currentTimeMillis() - startTime);
            return ResultResponse.success(uploadPath);
        } catch (Exception e) {
            e.printStackTrace();
            logger.error("图片上传出现异常[{}ms]", System.currentTimeMillis() - startTime, e.getMessage());
            return ResultResponse.fail("图片上传出现异常!");
        }
    }
  • 有和前端交互只需要给前端返回url
fastdfs-client.properties 配置文件
fastdfs.connect_timeout_in_seconds = 5
fastdfs.network_timeout_in_seconds = 30
fastdfs.charset = UTF-8
fastdfs.tracker_servers = 99.99.238.179:22122 
整个工程工程目录结构

用PostMan测试返回结构

PS:以前用PostMan不会上传图片进行测试,现在终于可以用了。(微笑)

SpringBoot+fastDFS搭建和测试就告一段落了,有问题大家可以一起交流

PS:如果大家在linux上搭建还有问题,可以参考这篇博客 blog.csdn.net/busishenren…

生命不息,战斗不止!!!!!!!!