030-前端错误日志上报及网站统计(sentry+matomo)

这是坚持技术写作计划（含翻译）的第30篇，定个小目标999，每周最少2篇。

本文配合rancher1.6(手头一个测试集群没升级到最新的2.x)讲解如何搭建并配置日志错误上报框架Sentry及网站统计分析框架matomo 的搭建及接入vue(本文以iview-admin为例)项目。

背景简述

• sentry 项目运行过程中，难免出现bug，前端不像后端可以很方便的采集项目日志(比如log4j + elk)，导致每次出问题后还原车祸现场费时费力。另外现在随着vue等兴起，构建项目时打成min.js，无疑又加大了前端定位问题的难度。而sentry是一款专注于错误采集的框架，支持常见的主流语言

  

  采集，聚合，并推送错误信息。注意，sentry并不是日志平台(e.g. log4j + elk)，也不是监控平台，sentry专注于项目中的Error信息的采集，聚合，报警。
• matomo 前身Pwiki，是一款开源的web网站分析利器。类似于Google Analytics。具体的特性，参见 Premium Web Analytics ，比如绘制页面热力图，录制会话，访问漏斗，A/B Test等(这几样都是收费插件功能)。

注意：本文假设你已经有rancher1.6的环境

安装

matomo

rancher 创建matomo

在rancher主机上

## 创建必要文件夹
mkdir -p /data/matomo/{config,logs,php,maxmind}/

## 安装maxmind ip数据库
wget -P /tmp/ https://github.com/maxmind/geoipupdate/releases/download/v4.0.3/geoipupdate_4.0.3_linux_amd64.deb
dpkg -i /tmp/geoipupdate_4.0.3_linux_amd64.deb
mv /etc/GeoIP.conf{,.bak}
cat << EOF | sudo tee -a /etc/GeoIP.conf
AccountID 0
LicenseKey 000000000000
EditionIDs GeoLite2-Country GeoLite2-City GeoLite2-ASN
DatabaseDirectory /data/matomo/maxmind
EOF

## 下载最新maxmind数据库
geoipupdate
ls -lah /data/matomo/maxmind/
总用量 67M
drwxr-xr-x 2 root root 4.0K 7月   7 17:25 .
drwxr-xr-x 6 root root 4.0K 7月   7 17:23 ..
-rw------- 1 root root    0 7月   7 17:25 .geoipupdate.lock
-rw-r--r-- 1 root root 6.3M 7月   7 17:25 GeoLite2-ASN.mmdb
-rw-r--r-- 1 root root  57M 7月   7 17:25 GeoLite2-City.mmdb
-rw-r--r-- 1 root root 3.7M 7月   7 17:25 GeoLite2-Country.mmdb

## 定时更新ip数据库
cat << EOF | sudo tee -a /etc/cron.d/geoipupdate
50 2 * * 4 root /usr/bin/geoipupdate
EOF

## 设置php.ini
cat << EOF | sudo tee -a /data/matomo/php/php.ini
upload_max_filesize = 128M
post_max_size = 128M
max_execution_time = 200
memory_limit = 256M
EOF

docker-compose.yaml

version: '2'
services:
  matomo:
    image: matomo:latest
    stdin_open: true
    volumes:
    - /data/matomo/config:/var/www/html/config
    - /data/matomo/logs:/var/www/html/logs
    - /data/matomo/php/php.ini:/usr/local/etc/php/php.ini
    - /data/matomo/maxmind/GeoLite2-City.mmdb:/var/www/html/misc/GeoLite2-City.mmdb
    - /data/matomo/maxmind/GeoLite2-Country.mmdb:/var/www/html/misc/GeoLite2-Country.mmdb
    - /data/matomo/maxmind/GeoLite2-ASN.mmdb:/var/www/html/misc/GeoLite2-ASN.mmdb
    tty: true
    ports:
    - 80:80/tcp
    - 443:443/tcp

rancher-compose.yaml

version: '2'
services:
  matomo:
    scale: 1
    start_on_create: true
    health_check:
      response_timeout: 2000
      healthy_threshold: 2
      port: 80
      unhealthy_threshold: 3
      initializing_timeout: 60000
      interval: 2000
      strategy: recreate
      reinitializing_timeout: 60000

配置matomo

选择中文

系统检查

配置数据库

自动建表完成

创建管理员用户(忘记截图了)
设置分析网站(可以随便创建，后边再进行修改),注意跟进实际情况修改时区

复制跟踪代码

配置matomo

登陆

Sentry

rancher 创建Sentry

docker-compose.yml

version: '2'
services:
  cron:
    image: sentry:9
    environment:
      SENTRY_MEMCACHED_HOST: memcached
      SENTRY_REDIS_HOST: redis
      SENTRY_POSTGRES_HOST: postgres
      SENTRY_EMAIL_HOST: smtp
      SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
    stdin_open: true
    volumes:
    - /data/sentry-data:/var/lib/sentry/files
    - /data/sentry-data/config.yml:/etc/sentry/config.yml
    tty: true
    command:
    - run
    - cron
  memcached:
    image: memcached:1.5-alpine
  web:
    image: sentry:9
    environment:
      SENTRY_MEMCACHED_HOST: memcached
      SENTRY_REDIS_HOST: redis
      SENTRY_POSTGRES_HOST: postgres
      SENTRY_EMAIL_HOST: smtp
      SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
    stdin_open: true
    volumes:
    - /data/sentry-data:/var/lib/sentry/files
    - /data/sentry-data/config.yml:/etc/sentry/config.yml
    tty: true
    ports:
    - 9000:9000/tcp
  worker:
    image: sentry:9
    environment:
      SENTRY_MEMCACHED_HOST: memcached
      SENTRY_REDIS_HOST: redis
      SENTRY_POSTGRES_HOST: postgres
      SENTRY_EMAIL_HOST: smtp
      SENTRY_SECRET_KEY: SENTRY_SECRET_KEY_XXX
    stdin_open: true
    volumes:
    - /data/sentry-data:/var/lib/sentry/files
    - /data/sentry-data/config.yml:/etc/sentry/config.yml
    tty: true
    command:
    - run
    - worker
  redis:
    image: redis:3.2-alpine
  postgres:
    restart: unless-stopped
    image: postgres:9.5
    volumes:
    - /data/postgresql/data:/var/lib/postgresql/data
    ports:
    - 5432:5432/tcp

注意把  SENTRY_SECRET_KEY 换成 sentry的实际secret key

rancher-compose.yml

version: '2'
services:
  cron:
    scale: 1
    start_on_create: true
  memcached:
    scale: 1
    start_on_create: true
  web:
    scale: 1
    start_on_create: true
    health_check:
      response_timeout: 2000
      healthy_threshold: 2
      port: 9000
      unhealthy_threshold: 3
      initializing_timeout: 60000
      interval: 2000
      strategy: recreate
      reinitializing_timeout: 60000
  worker:
    scale: 1
    start_on_create: true
  redis:
    scale: 1
    start_on_create: true
  postgres:
    scale: 1
    start_on_create: true
    health_check:
      response_timeout: 2000
      healthy_threshold: 2
      port: 5432
      unhealthy_threshold: 3
      initializing_timeout: 60000
      interval: 2000
      strategy: recreate
      reinitializing_timeout: 60000

先将docker-compose.yml 保存到服务器上，用来初始化db和创建账号

docker-compose run --rm web upgrade
Would you like to create a user account now? [Y/n]: y
Email: anjia0532@gmail.com
Password: 
Repeat for confirmation: 
Should this user be a superuser? [y/N]: y
## 直到输出
Migrated:
 - sentry
 - sentry.nodestore
 - sentry.search
 - social_auth
 - sentry.tagstore
 - sentry_plugins.hipchat_ac
 - sentry_plugins.jira_ac
Creating missing DSNs
Correcting Group.num_comments counter
## 并退出

配置Sentry

配置vue

本文以iview-admin为例

git clone https://gitee.com/anjia/iview-admin.git
cd iview-admin

sentry

注意，网上很多文档，以讹传讹的使用过时的工具，raven-js .从5.x后官方建议使用@sentry/browser和@sentry/integrations

npm install --save @sentry/integrations
npm install --save @sentry/browser

修改 iview-admin\src\main.js

// The Vue build version to load with the `import` command
// (runtime-only or standalone) has been set in webpack.base.conf with an alias.
import Vue from 'vue'
import App from './App'
import router from './router'
import store from './store'
import iView from 'iview'
import i18n from '@/locale'
import config from '@/config'
import importDirective from '@/directive'
import { directive as clickOutside } from 'v-click-outside-x'
import installPlugin from '@/plugin'
import './index.less'
import '@/assets/icons/iconfont.css'
import TreeTable from 'tree-table-vue'
import VOrgTree from 'v-org-tree'
import 'v-org-tree/dist/v-org-tree.css'
import * as Sentry from '@sentry/browser';
import * as Integrations from '@sentry/integrations';
// 实际打包时应该不引入mock
/* eslint-disable */
if (process.env.NODE_ENV !== 'production') require('@/mock')

Vue.use(iView, {
  i18n: (key, value) => i18n.t(key, value)
})
Vue.use(TreeTable)
Vue.use(VOrgTree)
/**
 * @description 注册admin内置插件
 */
installPlugin(Vue)
/**
 * @description 生产环境关掉提示
 */
Vue.config.productionTip = false
/**
 * @description 全局注册应用配置
 */
Vue.prototype.$config = config
/**
 * 注册指令
 */
importDirective(Vue)
Vue.directive('clickOutside', clickOutside)

/* eslint-disable no-new */
new Vue({
  el: '#app',
  router,
  i18n,
  store,
  render: h => h(App)
})

Sentry.init({
  dsn: 'https://xxx@xxx.xxx.com/xxx',
  integrations: [
    new Integrations.Vue({
      Vue,
      attachProps: true,
    }),
  ],
});

npm install
npm run dev

打开 http://localhost:8080/error_store/error_store_page
分别点击两个按钮，模拟出错

打开sentry发现已经有错误上报了，并且对错误进行聚合。

点开查看详细内容。

如果需要生成source-map ,可以参考官方文档 docs.sentry.io/platforms/j…

matomo

npm install --save vue-matomo

修改 iview-admin\src\main.js

// The Vue build version to load with the `import` command
// (runtime-only or standalone) has been set in webpack.base.conf with an alias.
import Vue from 'vue'
import App from './App'
import router from './router'
import store from './store'
import iView from 'iview'
import i18n from '@/locale'
import config from '@/config'
import importDirective from '@/directive'
import { directive as clickOutside } from 'v-click-outside-x'
import installPlugin from '@/plugin'
import './index.less'
import '@/assets/icons/iconfont.css'
import TreeTable from 'tree-table-vue'
import VOrgTree from 'v-org-tree'
import 'v-org-tree/dist/v-org-tree.css'
import * as Sentry from '@sentry/browser'
import * as Integrations from '@sentry/integrations'
import VueMatomo from 'vue-matomo'

// 实际打包时应该不引入mock
/* eslint-disable */
if (process.env.NODE_ENV !== 'production') require('@/mock')

Vue.use(iView, {
  i18n: (key, value) => i18n.t(key, value)
})
Vue.use(TreeTable)
Vue.use(VOrgTree)
/**
 * @description 注册admin内置插件
 */
installPlugin(Vue)
/**
 * @description 生产环境关掉提示
 */
Vue.config.productionTip = false
/**
 * @description 全局注册应用配置
 */
Vue.prototype.$config = config
/**
 * 注册指令
 */
importDirective(Vue)
Vue.directive('clickOutside', clickOutside)

/* eslint-disable no-new */
new Vue({
  el: '#app',
  router,
  i18n,
  store,
  render: h => h(App)
})

Sentry.init({
  dsn: 'https://xxx@xxx.xxx.com/xxx',
  integrations: [
    new Integrations.Vue({
      Vue,
      attachProps: true,
    }),
  ],
});
Vue.use(VueMatomo, {
  // Configure your matomo server and site by providing
  host: '//xxxx.xxxx.com/',
  siteId: xx,
 
  // Changes the default .js and .php endpoint's filename
  // Default: 'piwik'
  trackerFileName: 'matomo.js',
 
  // Overrides the autogenerated tracker endpoint entirely
  // Default: undefined
  trackerUrl: '//xxxx.xxxx.com/matomo.php',
 
  // Enables automatically registering pageviews on the router
  router: router,
 
  // Enables link tracking on regular links. Note that this won't
  // work for routing links (ie. internal Vue router links)
  // Default: true
  enableLinkTracking: true,
 
  // Require consent before sending tracking information to matomo
  // Default: false
  requireConsent: false,
 
  // Whether to track the initial page view
  // Default: true
  trackInitialView: true,
 
  // Whether or not to log debug information
  // Default: false
  debug: false
});
 
 
// or
window._paq.push
 
// or through
window.Piwik.getTracker

打开 http://localhost:8080, 随便访问几个菜单,然后打开matomo

路由已经有数据了

并且将用户的常规数据聚合起来

更多

其实本文只是sentry和matomo简单介绍
更深入的使用，比如sentry，推送邮件，文中一带而过的sourcemap，单点登录(集成内部的权限认证)，自定义上报内容(将错误与用户id关联起来)，,敏感数据脱敏等
比如matomo, 每日发送分析报表，增加kafka插件，进行更深层次的挖掘，自定义上报内容(购物车等),大数据量情况下的优化，优化用户设备指纹，使用了nginx等反代软件后，如何正确识别真实ip，热力图，A/B test,漏斗图等

参考资料

• 我的个人博客
• 我的掘金
• matomo官网
• INSTALLING MATOMO
• Sentry官网
• Installation with Docker
• Automatic Updates for GeoIP2 and GeoIP Legacy Databases
• vue中如何使用sentry对错误日志进行监控