介绍
作为一个产品线的程序猿，是否每天都会面对很多mr和pr呢？其中code review是不可或缺的一环，但是大家想想如果能在code review之前就能自动检测整个系统，将测试结果作为依据，那么review环节就能准确找到问题所在。
所以将E2E功能测试加入mr或pr流程是有必要的。
工具
建木：建木以触发器、流程编排、任务分发等功能为核心，可以应用在各类使用场景下，包括但不限于，CI/CD、DevOps、自动化运维、多业务系统集成等场景。使用建木能轻松构建一个复杂流程，学习成本低，容易维护。
Cypress：Cypress是基于JavaScript语言的前端自动化测试工具，无需借助外部工具，自集成了一套完整的端到端测试方法，可以对浏览器中运行的所有内容进行快速、简单、可靠的测试，并且可以进行接口测试。我们使用的主要是它E2E测试，根据测试用例对整个程序进行质量扫描。
流程简介
流程讲解：
- coder提交代码
- 发起merge request请求
- 触发建木webhook
- 执行MR分支，跑E2E测试
- 将测试报告发布
- 向merge request列表发送报告结果地址
- reviewer小伙伴根据测试报告定位问题
- reviewer决定是否合并MR
- 合并MR，触发建木webhook
- 执行CICD流程
很多时候直接通过审查代码是无法确定该功能代码是否会影响到其他的功能代码，逻辑无误的代码块放到全局中运行也可能出现意想不到的问题。通过查看E2E测试报告，如果发现其提交的代码影响到程序正常执行，能很快的定位问题代码位置，如果是阻断行错误也可以即时阻止merge request进程。
流程构建
主要流程节点简介
-
git_clone节点
用于拉取目标分支代码
-
shell节点
可以在 DSL 中定义 Shell 节点，在指定镜像启动的容器中，执行多条 Shell 命令
shell节点为官方内置节点
-
scp上传文件节点
将E2E测试报告通过scp上传/替换指定文件到指定主机的指定目录。
- scp上传文件节点（官方：jianmuhub.com/_/scp_resou…）
- docker镜像构建节点构建镜像文件
- docker镜像构建节点（官方：jianmuhub.com/_/docker_im…）
- 推送mr消息
- 推送mr消息节点（官方：jianmuhub.com/_/gitlab）
DSL编排
name: autoops_frontend2
description: 产品前端发版
global:
enabled:
mutable: true
concurrent: true
param:
# 大版本
release_version: V1.0.0
host:
type: STRING
value: *.*.*.*
trigger:
type: webhook
param:
- name: ref
type: STRING
exp: $.body.json.ref
- name: object_kind
type: STRING
exp: $.body.json.object_kind
- name: merge_state
type: STRING
exp: $.body.json.object_attributes.state
- name: merge_action
type: STRING
exp: $.body.json.object_attributes.action
- name: ip
type: STRING
exp: $.header.x-real-ip
- name: id
type: NUMBER
exp: $.body.json.project.id
- name: commit_time
type: STRING
exp: $.body.json.commits[0].timestamp
- name: commit_author
type: STRING
exp: $.body.json.commits[0].author.name
- name: commit_message
type: STRING
exp: $.body.json.commits[0].message
- name: iid
type: NUMBER
exp: $.body.json.object_attributes.iid
- name: branch
type: STRING
exp: $.body.json.object_attributes.source_branch
only: ((${trigger.object_kind} == "merge_request" && ${trigger.merge_state} == "opened" && (${trigger.merge_action} == "open" || ${trigger.merge_action} == "reopen")) || ${trigger.ref} == "refs/heads/main")
workflow:
start:
alias: 开始
type: start
targets:
- condition
condition:
alias: 判断是否mr请求
sources:
- start
type: condition
expression: ${trigger.object_kind} == "merge_request"
cases:
true: git_clone_merge_branch
false: git_clone
git_clone_merge_branch:
alias: 拉取前端代码
sources:
- condition
targets:
- init_test_database
type: "git_clone:1.2.4"
param:
remote_url: "http://*.*.*.*/autoops-templates/autoops/frontend/autoops_frontend2.git"
ref: refs/heads/${trigger.branch}
password: "((autoops.frontend_password))"
username: "((autoops.frontend_user))"
init_test_database:
alias: 测试数据库初始化
sources:
- git_clone_merge_branch
targets:
- cypress_auto_test
image: mariadb:latest
environment:
GIT_PATH: ${git_clone_merge_branch.git_path}
script:
- cd ${GIT_PATH}
- ls ./cypress
- mysql -h *.*.*.* -u * -p * -Dautoops-e2e < ./cypress/sql/reset.sql
cypress_auto_test:
alias: e2e
sources:
- "init_test_database"
targets:
- "scp_resouce"
on-failure: ignore
image: cypress/node:v1.0.0
environment:
GIT_PATH: ${git_clone_merge_branch.git_path}
script:
- cd ${GIT_PATH}
- set -ex
- npm install -g wait-on --registry https://registry.npm.taobao.org/
- export CI=false
- export CYPRESS_INSTALL_BINARY=/app/cypress.zip
- export PROXY_PATH=http://*.*.*.*:18080
- export PROXY_PATH2=http://*.*.*.*:18088
- yarn install
- yarn start &
- wait-on http://localhost:9000
- yarn run:cypress
- cd cypress/results/mocha
- ls
scp_resouce:
alias: 结果文件上传
type: "scp_resouce:1.2.0"
param:
ssh_port: "22"
remote_file: /home/autoOps_cypress_result/merge-${trigger.iid}
ssh_ip: ${global.host}
ssh_private_key: ((autoops.ssh_key))
local_file: ${git_clone_merge_branch.git_path}/cypress/results/mocha
ssh_user: "root"
sources:
- cypress_auto_test
targets:
- cypress-nginx
cypress-nginx:
alias: 构建cypress镜像
sources:
- "scp_resouce"
targets:
- "cypress-deploy"
type: "docker_image_build:1.2.0"
param:
docker_file: "cypress/docker/Dockerfile"
image_name: "${global.host}:5000/autoops-cypress"
workspace: "${git_clone_merge_branch.git_path}"
docker_password: "((autoops.docker_pwd))"
docker_username: "((autoops.docker_user))"
docker_build_path: "."
registry_address: "${global.host}:5000"
cypress-deploy:
alias: cypress部署
sources:
- "cypress-nginx"
targets:
- "git_merge_comment"
type: "ssh_cmd:1.0.1"
param:
ssh_private_key: ((autoops.ssh_key))
ssh_ip: ${global.host}
ssh_port: "22"
ssh_user: root
ssh_cmd: docker stop autoops-cypress || true && docker rm autoops-cypress || true
>/dev/null; docker run -itd -p 8855:80 -v /home/autoOps_cypress_result:/app/html
--name autoops-cypress --restart always ${global.host}:5000/autoops-cypress
git_merge_comment:
alias: MR消息推送
sources:
- "cypress-deploy"
targets:
- "end"
type: "gitlab:1.0.0-merge-comment"
param:
project_iid: ${trigger.id}
port: "80"
host: "*.*.*.*"
comment: "e2e测试已完成，请查阅结果：http://${global.host}:8855/merge-${trigger.iid}/combiine-mochawesome.html"
merge_request_iid: ${trigger.iid}
token: ((autoops.frontend_token))
git_clone:
alias: 拉取前端代码
sources:
- condition
targets:
- appBuild
type: "git_clone:1.2.4"
param:
remote_url: "http://*.*.*.*/autoops-templates/autoops/frontend/autoops_frontend2.git"
ref: refs/heads/main
password: "((autoops.frontend_password))"
username: "((autoops.frontend_user))"
appBuild:
alias: 系统构建
sources:
- git_clone
targets:
- nodeFormatString
- docker_image_build2
image: cypress/node:v1.0.0
environment:
GIT_PATH: ${git_clone.git_path}
script:
- cd ${GIT_PATH}
- set -ex
- export CI=false
- export CYPRESS_INSTALL_BINARY=0
- yarn install
- yarn build
nodeFormatString:
alias: 处理字符串
sources:
- "appBuild"
targets:
- "docker_image_build1"
type: "string:1.0.0-nodejs16.13.1"
param:
expression: ""${trigger.commit_time}".replace(/-|T|:/g,'').replace('+0800', '')"
docker_image_build1:
alias: 记录历史镜像
sources:
- "nodeFormatString"
targets:
- "ssh_cmd"
type: "docker_image_build:1.2.0"
param:
docker_file: "dist/Dockerfile"
image_name: "${global.host}:5000/autoops-ui"
workspace: "${git_clone.git_path}"
docker_password: "((autoops.docker_pwd))"
docker_username: "((autoops.docker_user))"
docker_build_path: "."
registry_address: "${global.host}:5000"
image_tag: "${nodeFormatString.result}"
docker_image_build2:
alias: 发布镜像
sources:
- "appBuild"
targets:
- "ssh_cmd"
type: "docker_image_build:1.2.0"
param:
docker_file: "dist/Dockerfile"
image_name: "${global.host}:5000/autoops-ui"
workspace: "${git_clone.git_path}"
docker_password: "((autoops.docker_pwd))"
docker_username: "((autoops.docker_user))"
docker_build_path: "."
registry_address: "${global.host}:5000"
ssh_cmd:
alias: 部署
sources:
- "docker_image_build1"
- "docker_image_build2"
targets:
- "qywx_notice"
type: "ssh_cmd:1.0.1"
param:
ssh_private_key: ((autoops.ssh_key))
ssh_ip: ${global.host}
ssh_port: "22"
ssh_user: root
ssh_cmd: docker stop autoops-ui || true && docker rm autoops-ui || true
>/dev/null; docker run -itd -p 8877:80 --name autoops-ui
-e JIANMU_ADDRESS=http://*.*.*.*:30180
-e AUTOOPS_APPLICATION_ADDRESS=http://${global.host}:8080
-e AUTOOPS_AUTOOPS_ADDRESS=http://${global.host}:8088
-e AUTOOPS_PROMETHEUS_ADDRESS=http://*.*.*.*:9103
--restart always ${global.host}:5000/autoops-ui:${nodeFormatString.result}
qywx_notice:
alias: 企业微信消息通知
sources:
- "ssh_cmd"
targets:
- "end"
type: "qywx_notice:1.2.1-text"
param:
mentioned_mobile_list: "["@all"]"
bot_webhook_url: "((autoops.wx_webhook))"
text_content: "autoOps前端发版成功,访问地址为：http://${global.host}:8877，触发者：${trigger.commit_author}，提交信息：${trigger.commit_message}"
mentioned_list: "[]"
end:
alias: 结束
type: end
sources:
- qywx_notice
- git_merge_comment
大家看看最终可视化图形界面吧！
如何触发流程呢？
- 复制将建木webhook地址加入gitlab项目webhooks中（一定要选merge request 方式触发哟！）
大家看看最后运行结果吧！
reviewer就可以可根据测试报告处理这个MR了，大大提升了review效率，也极大减少了因review错漏导致环境出现报错情况！
官⽹：jianmu.dev
