微服务从零开始之留言板

164 阅读6分钟

目录

  • 概述
  • 需求分析
  • 领域对象设计
  • API 设计
  • Create Message
  • Request
  • Response
  • Retrieve Message
  • Update Message
  • Delete Message
  • Query Message
  • TDD - Test Driven Development
  • 测试方法 Test methods
  • 测试矩阵 Test Matrix
  • 留言板实现细节
  • 搭建骨架
  • 构建工具和插件
  • 我们需要哪些库
  • 日志库
  • 测试库
  • 框架及工具库
  • 度量相关库
  • 基本骨架
  • web.xml
  • 数据对象
  • 配置类
  • 数据库创建
  • MessageDb *留言板的主要实现
  • MessageContoller
  • MessageService
  • MessageDao
  • 参考

概述

以一个最简单的留言本为例, 麻雀虽小, 五脏俱全, 它基本上牵涉到了一个微服务的各个方面, 让我们看看如何从零开始,从无到有构建一个微服务

需求分析

让我们先从用户故事开始, 用例比较多, 可以分优先级分步实施

留言本看下来简单, 其实牵涉到 Web 开发的各个方面, 类似一个小微博

User storyPriority
注册B
登录B
用户管理B
访客留言A
用户及访客评论B
留言管理C
评论管理C

优先级可以参考时间管理的任务重要性划分

  • A 重要且紧急
  • B 重要不紧急
  • C 紧急不重要
  • D 不紧急不重要

Paste_Image.png

用户故事图脚本

[Guest]-(Post Message), 
[Guest]-(Query Message), 
[Guest]-(Sign Up), 
[User]-(Sign In), 
[User]-(Add Comments), 
[User]-(Update Message), 
[User]-(Query Message), 
[User]-(Update Self), 
[Admin]-(User Manage), 
[Admin]-(Manage Message), 
(Manage Message)>(Delete Message), 
(Manage Message)>(Delete Comment), 
(Manage Message)>(Archive Message), 
(User Manage)>(Approve Sign Up), 
(User Manage)>(CRUDQ User), 
(CRUDQ User)>(Reset Password), 
(CRUDQ User)>(Lock User)

领域对象设计

对于 guestbook 的最高优先级的用户故事是留言, 也就是 Post Message

让我们先从领域对象设计开始, 留言本的核心对象是 Message 和 Guest

Paste_Image.png

类图脚本如下

%2F%2F Cool Class Diagram, , [Message|id:String;title:String;content:String;tags:String;author: Author; createTime: timestamp], [Message]<>-[Author|id:String;name:String;email: Email;phoneNumber: PhoneNumber;createTime: timestamp]

对应于领域对象如下

  • 消息 Message
attributetype
idString(UUID)
titleString
contentString
authorAuthor
tagsString
createTimeTimestamp
  • 留言者 Author
attributetype
idString(uuid)
nameString
emailString
phoneNumberString
createTimeTimestamp

API 设计

对象的基本操作是典型的 CRUDQ, 即创建 Create, 获取 Retrieve, 修改 Update, 删除 Delete 和查询 Query

OperationAPI
Create MessagePOST /messages
Retrieve MessageGET/messages/$id
Update MessagePUT /messages/$id
Delete MessageDELETE/messages/$id
Query MessageGET /messages?$parameters

Create Message

POST /api/v1/messages

Request

{
"title" : "String",
"content": "String",
"author": {
    "name" : "String",
    "email" : "Email",
    "phoneNumber": "PhoneNumber"
},
"tags" :  "String",
}

Response

{
"url": "http://guestbook.com/api/vi/messages/$id"
"title" : "String",
"content": "String",
"author": {
    "name" : "String",
    "email" : "Email",
    "phoneNumber": "PhoneNumber"
},
"tags" :  "String",
}

Retrieve Message

  • GET /api/v1/messages/:id

Update Message

  • PUT /api/v1/messages/:id

Delete Message

  • DELETE /api/v1/messages/:id

Query Message

  • GET /api/v1/messages

| Parameter | Type | Default | Comments | |:----------|------|-----------|---------|---------| | start | integer | 0 | start number | | limit | integer | 20 | message count | | order | string | asc | asc or desc | | sortby | string | title | id, title, author, email, createtime | | field | string | * | title, content, author name or email | | keyword | string | n/a | |

TDD - Test Driven Development

测试驱动开发已经深入人心, 从下到上, 从单元测试到集成测试, 这些是质量的保证 一般来说, 测试的行覆盖率起码要在 80% 以上

测试金字塔我们都有所耳闻,case多了,速度慢了,逻辑越复杂,测试越脆弱,测试集的归类,统计很重要

Paste_Image.png

测试方法 Test methods

  • 单元测试 Unit Test: TestNG, Mockito, SpringTesting
  • 接收测试 API Test: HttpClient
  • 端到端集成测试 E2E Test: Selenium

测试矩阵 Test Matrix

Test CaseCategoryComments
消息创建 Message:CreateUT,API
消息修改 Message:UpdateUT,API,E2E
消息删除 Message:DeleteUT, API
消息简单查询 Message:RetrieveUT
消息复杂查询 Message:QueryUT, API分页, 排序,根据关键字查询

好了, 到此为止, 咱们已经搞清楚需求和领域对象了, 可以动手开始编程了

且慢, 想想这是不是就够了, 做了就要上线, 上线之后我们最关心什么

关注点度量
功能是否正常完备Function Metrics
用量如何Usage Metrics
性能如何Performance Metrics
有无异常Exception Metrics
有无遭受攻击Fraud attack Metrics
出现问题的修复时间Fileover/Recover Metrics

让我们在编码实现的时候, 把这些记在心头.

留言板实现细节

以大家比较熟悉的 Java Web App 为例

技术选型如下

  • 前端框架: AngularJS
  • 后端框架: SpringMVC
  • 数据库: SQLite

搭建骨架

mvn archetype:generate -DgroupId=com.github.walterfan -DartifactId=guestbook  -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false

这样 maven 就为你创建了一个 Java Web App 的骨架

guestbook//pom.xml
guestbook//src
guestbook//src/main
guestbook//src/main/resources
guestbook//src/main/webapp
guestbook//src/main/webapp/index.jsp
guestbook//src/main/webapp/WEB-INF
guestbook//src/main/webapp/WEB-INF/web.xml

Java Web App 的开发框架汗牛充栋, 比如 Struts2, Spring MVC, 还有最近比较流行的 DropWizard 和 Spring Boot

这两个框架都是众多开源项目的集大成者, 先不用这么重的东西来做留言本 这里就用Spring Boot 的核心项目 Spring MVC 来快速实现

具体实现下节细说, 这里讲几句题外话, 很多Java 开发者都有"好读书, 不求甚解"的坏毛病, 包括我在内, 从 C/C++ 世界转过来, 发现 Java 太方便了, 开发效率极高, 各种库和框架让人目不暇接, 很容易就迷失了

有时间还是可以看一看 HTTP 协议 和 Servlet JSR(Java Specification Requests) 最近一版是 JSR 340: Java Servlet 3.1 Specification

归根到底, 它是一个网络应用程序, 程序启动时侦听 80 或其他端口, 接收 HTTP Request, 解包交应用逻辑进行一些处理后以 HTTP Response 的编码返回.

只不过, 现在通过Servlet 容器把这些底层的脏活累活干了, 交到 Application 手里已经是标准的 HttpRequest 和 HttpResponse

构建工具和插件

Maven 是Java世界的标配, 近年来gradel 异军突起, 有待时间的检验

参见详情: github.com/walterfan/g…

Maven 的插件也是林林总总, 不胜枚举, 这里只选用一些常用的

  • maven-surefire-plugin for uni test
  • Jacoco-maven-plugin for test coverage
  • maven-failsafe-plugin for integration test

我们需要哪些库

日志库

  • sl4j
  • logback

测试库

  • testng
  • mockito
  • spring test

框架及工具库

  • Spring MVC
  • Jackson
  • guava
  • commons lang3, io,

度量相关库

基本骨架

Spring MVC 原理回顾, DispatchServlet 是其核心

  • Controller
  • Service
  • Domain
  • Dao
  • Metrics: 度量相关代码

web.xml

<?xml version="1.0" ?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

<description>Micro Service</description>

<context-param>
  <param-name>contextClass</param-name>
  <param-value>org.springframework.web.context.support.AnnotationConfigWebApplicationContext</param-value>
</context-param>

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>com.github.walterfan.guestbook.MessageConfig</param-value>
</context-param>

<listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>

<servlet>
  <servlet-name>mvc-servlet</servlet-name>
  <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
  <init-param>
    <param-name>contextConfigLocation</param-name>
    <param-value></param-value>
  </init-param>
  <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
  <servlet-name>mvc-servlet</servlet-name>
  <url-pattern>/*</url-pattern>
</servlet-mapping>

</web-app>

数据对象

    1. class Message
package com.github.walterfan.guestbook.domain;


import org.hibernate.validator.constraints.NotBlank;

import javax.validation.constraints.NotNull;
import java.util.Date;


public class Message extends BaseObject {
    private String id;
    @NotBlank
    private String title;
    @NotBlank
    private String content;
    @NotNull
    private Author author;
    private String tags;
    private Date createTime;

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }

    public Author getAuthor() {
        return author;
    }

    public void setAuthor(Author author) {
        this.author = author;
    }

    public String getTags() {
        return tags;
    }

    public void setTags(String tags) {
        this.tags = tags;
    }

    public Date getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Date createTime) {
        this.createTime = createTime;
    }
}

配置类

  • 代替之前的spring bean xml 配置文件
    1. class MessageConfig
    
package com.github.walterfan.guestbook;

import com.github.walterfan.guestbook.controller.IndexController;
import com.github.walterfan.guestbook.controller.MessageController;
import com.github.walterfan.guestbook.dao.MessageDao;
import com.github.walterfan.guestbook.dao.MessageMapper;
import com.github.walterfan.guestbook.service.MessageService;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.core.env.Environment;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.SimpleDriverDataSource;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;

import javax.sql.DataSource;
import java.sql.Driver;

/**
 * Created by walter on 06/11/2016.
 */
@Configuration
@EnableWebMvc
@Import({
        IndexController.class,
        MessageController.class
})
public class MessageConfig {

    @Autowired
    private Environment env;

    @Bean
    public MessageService messageService() {
        return new MessageService();
    }

    @Bean
    public MessageProperties messageProperties()  {
        return new MessageProperties();
    }


    @Bean
    public DataSource dataSource() throws ClassNotFoundException {
        SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
        dataSource.setDriverClass((Class<? extends Driver>)   Class.forName(messageProperties().getJdbcDriver()));
        dataSource.setUsername(messageProperties().getJdbcUserName());
        dataSource.setUrl(messageProperties().getJdbcUrl());
        dataSource.setPassword(messageProperties().getJdbcPassword());

        return dataSource;
    }

    @Bean
    public DataSourceTransactionManager transactionManager() throws ClassNotFoundException {
        return new DataSourceTransactionManager(dataSource());
    }


    @Bean
    public SqlSessionFactory sqlSessionFactory() throws Exception {
        SqlSessionFactoryBean sqlSessionFactory = new SqlSessionFactoryBean();
        sqlSessionFactory.setDataSource(dataSource());
        return (SqlSessionFactory) sqlSessionFactory.getObject();
    }

    @Bean
    public MessageDao messageDao() throws Exception {
        SqlSessionFactory sessionFactory = sqlSessionFactory();
        sessionFactory.getConfiguration().addMapper(MessageMapper.class);

        SqlSessionTemplate sessionTemplate = new SqlSessionTemplate(sqlSessionFactory());
        return sessionTemplate.getMapper(MessageMapper.class);
    }
}


数据库创建

我们可以用反射的方法直接生成创建,删除以及插入表数据的语句, 而不必自己手写SQL, Hibernate框架也用了类似的方法, 这里简单实现一个数据库创建初始化的工具

配置如下 jdbc.properties

jdbc.driverClass=org.sqlite.JDBC
jdbc.url=jdbc:sqlite:/workspace/walter/wfdb.s3db
#jdbc.driverClass=com.mysql.jdbc.Driver
#jdbc.url=jdbc:mysql://localhost/wfdb?useUnicode=true&characterEncoding=utf8
jdbc.username=walter
jdbc.password=pass

MessageDb

package com.github.walterfan.guestbook.db;

import com.github.walterfan.guestbook.domain.Message;

import java.sql.SQLException;
import java.util.Date;
import java.util.UUID;

import static java.lang.System.out;

/**
 * Created by walter on 07/11/2016.
 */
public class MessageDb {

    private final DbConn dbConn;

    private static String CHECK_SQL = "SELECT * FROM sqlite_master WHERE type='table' and name='%s'";

    public MessageDb() throws Exception {
        dbConn = new DbConn("jdbc.properties");
        dbConn.setDebug(true);
        dbConn.createConnection();
    }

    public void init() throws Exception {
        int ret = initTable();
        if(ret > 0) {
            initData();

        }
    }

    public int initTable() throws Exception {
        int ret = check(Message.class);
        if(ret > 0) {
            out.println("found table and drop it firstly ");
            dropTable(Message.class);
        }

        createTable(Message.class);
        return check(Message.class);



    }

    private int initData() throws Exception {
        String id = UUID.randomUUID().toString();
        Message msg = new Message();
        msg.setId(id);
        msg.setTitle("hello guest");
        msg.setContent("this is a test message");
        msg.setTags("test tag");
        msg.setCreateTime(new Date());
        String sql = DbHelper.makeInsertSql(msg);
        out.println("execute " + sql);
        dbConn.execute(sql);

        sql = DbHelper.makeQuerySql(msg.getClass(), String.format("id = '%s'", id));
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public int createTable(Class<?> clazz) throws Exception {

        String sql = DbHelper.makeCreateTableSql(clazz);
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public int dropTable(Class<?> clazz) throws Exception {

        String sql = DbHelper.makeDropTableSql(clazz);
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public void clean() throws SQLException {
        dbConn.commit();
        dbConn.closeConnection();
    }

    public int check(Class<?> clazz) throws Exception {
        String sql = String.format(CHECK_SQL, clazz.getSimpleName().toLowerCase());
        out.println("execute " + sql);
        return dbConn.execute(sql);

    }

    public static void main(String[] argv) throws Exception {
        MessageDb db = new MessageDb();
        db.init();
        db.clean();
    }

 }

其他代码参见 github.com/walterfan/g…

执行结果如下:

execute SELECT * FROM sqlite_master WHERE type='table' and name='message'

typenametbl_namerootpagesql
tablemessagemessage48CREATE TABLE message (id TEXT,title

found table and drop it firstly execute DROP TABLE message execute CREATE TABLE message (id TEXT,title TEXT,content TEXT,author TEXT,tags TEXT,createTime DATETIME) execute SELECT * FROM sqlite_master WHERE type='table' and name='message'

typenametbl_namerootpagesql
tablemessagemessage48CREATE TABLE message (id TEXT,title

execute insert into message(id,title,content,author,tags,createTime) values('fa2ba0d0-d843-48e1-804d-641647d33b5b','hello guest','this is a test message',null,'test tag','2016-11-17 23:08:34.959') execute select * from message where id = 'fa2ba0d0-d843-48e1-804d-641647d33b5b'

idtitlecontentauthortagscreateTime
fa2ba0d0-d843-48e1-804d-641647d33b5bhello guestthis is a test messagetest tag2016-11-17 23:08:34.959

留言板的主要实现

按照标准的 SpringMVC 结构, 主要用五个类来搞定, 前面两个前面已经提过了, 所有代码请参见 github.com/walterfan/g…, 下面是所谓的 CRC( Class Responsibility Collaborator ) 类职责与协作者表格

ClassResponsibilityCollaborator
1. Message留言数据对象所有类
2. MessageConfig留言板配置MessageController, MessageService, MessageDao
3. MessageController留言板控制器MessageService
4. MessageService留言板服务MessageDao, MessageController
5. MessageDao留言板数据存储接口MessageService
6. MessageMapper留言板数据存储实现MessageService

注: 为简单起见, 这里只用了一个数据对象 Message , 也可以细分为

ClassResponsibilityCollaborator
MessageDto数据传输对象MessageService, MessageDao
MessageEntity数据实体对象MessageService, MessageDao
MessageBo数据业务对象MessageService
MessageRequest数据请求对象MessageController
MessageResponse数据响应对象MessageController

MessageContoller

  • 3 . class MessageController
package com.github.walterfan.guestbook.controller;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import com.github.walterfan.guestbook.service.MessageService;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import javax.validation.Valid;
import java.util.List;

@RestController
@RequestMapping(value = "/guestbook/api/v1/", produces = { "application/json" })
public class MessageController {


    protected final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private MessageService messageService;


    @RequestMapping(value = "/messages", method = RequestMethod.POST)
    public Message createMessage(@Valid @RequestBody Message message) throws Exception {
        logger.info("got post request: " + message.toString());
        messageService.createMessage(message);
        return message;
    }

    @RequestMapping(value = {"/messages", "/"}, method = RequestMethod.GET)
    public List<Message> queryMessages(@RequestParam(value = "start",   required = false) Integer start,
                                       @RequestParam(value = "limit",   required = false) Integer limit,
                                       @RequestParam(value = "order",   required = false) String order,
                                       @RequestParam(value = "sortBy",  required = false) String sortBy,
                                       @RequestParam(value = "keyword", required = false) String keyword,
                                       @RequestParam(value = "fieldName",   required = false) String fieldName) {
        logger.info("query messages request");

        GenericQuery query = new GenericQuery();
        if(null != start) query.setStart(start);
        if(null != limit) query.setLimit(limit);
        if(null != order) {
            if("ASC".equalsIgnoreCase(order)) {
                query.setOrder(GenericQuery.OrderType.ASC);
            } else if("DESC".equalsIgnoreCase(order)) {
                query.setOrder(GenericQuery.OrderType.DESC);
            }
        }
        if(StringUtils.isNotBlank(sortBy)) query.setSortBy(sortBy);
        if(StringUtils.isNotBlank(fieldName)) query.setFieldName(fieldName);
        if(StringUtils.isNotBlank(keyword)) query.setKeyword(keyword);

        List<Message> messageList = messageService.queryMessage(query);
        return messageList;
    }

    @RequestMapping(value = "messages/{id}", method = RequestMethod.GET)
    public Message getMessage(@PathVariable("id") String id) throws Exception {
        return messageService.retrieveMessage(id);
    }


    @RequestMapping(value = "messages/{id}", method = RequestMethod.PUT)
    public Message updateMessage(@PathVariable("id") String id, @RequestBody Message message) {
        message.setId(id);
        messageService.updateMessage(message);
        return message;
    }

    @RequestMapping(value = "messages/{id}", method = RequestMethod.DELETE)
    public void deleteMessage(@PathVariable("id") String id) {
        messageService.deleteMessage(id);

    }
}


MessageService

  • 4 . class MessageService
package com.github.walterfan.guestbook.service;

import com.github.walterfan.guestbook.dao.MessageDao;
import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.UUID;


@Service
public class MessageService {

    @Autowired
    private MessageDao messageDao;

    public void createMessage(Message message) {
        UUID id = UUID.randomUUID();
        message.setId(id.toString());
        messageDao.createMessage(message);
    }

    public Message retrieveMessage(String id) {
        return messageDao.retrieveMessage(id);
    }

    public List<Message> queryMessage(GenericQuery query) {
        return messageDao.queryMessage(query);
    }

    public void updateMessage(Message message) {
        messageDao.updateMessage(message);
    }

    public void deleteMessage(String id) {
        messageDao.deleteMessage(id);

    }
}

MessageDao

  • 5 . class MessageDao
package com.github.walterfan.guestbook.dao;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;

import java.util.List;

public interface MessageDao {

    void createMessage(Message message);


    Message retrieveMessage(String id);

    void updateMessage(Message message);


    void deleteMessage(String id);


    List<Message> queryMessage(GenericQuery query);
}

  • 6 . MessageMapper
package com.github.walterfan.guestbook.dao;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import org.apache.ibatis.annotations.Delete;
import org.apache.ibatis.annotations.Insert;
import org.apache.ibatis.annotations.Select;
import org.apache.ibatis.annotations.Update;

import java.util.List;

public interface MessageMapper extends MessageDao {

    //#{author.id}
    @Insert("INSERT into message(id,title,content,tags, createTime) " +
            "VALUES(#{id}, #{title}, #{content}, #{tags}, #{createTime})")
    void createMessage(Message message);

    @Select("SELECT * FROM message WHERE id = #{id}")
    Message retrieveMessage(String id);

    @Update("UPDATE message SET title=#{title}, content =#{content}, tags=#{tags} , " +
            " WHERE id =#{id}")
    void updateMessage(Message message);

    @Delete("DELETE FROM message WHERE id =#{id}")
    void deleteMessage(String id);

    @Select("SELECT * FROM message ")
    List<Message> queryMessage(GenericQuery query);

}

至此, 一个最小的留言板微服务雏形已成, 可以快速看一下效果

mvn jetty:run

Paste_Image.png

好, 就此打住.

好的开始是成功的一半, 虽然我们只完成了整个项目的第一步, 也等于成功了一半. 之后, 让我们一步一步来分析和实现更多功能性和非功能性的需求吧. 即使这么小的一个留言板微服务, 也还有不少细节要仔细考虑和优化.

参考