体验下阿里的开源告警组件kube-eventer

96 阅读2分钟

why

1 为什么要用kube-eventer这个组件

我的公司网络环境特别复杂,还是那种传统模式,生产环境和开发环境是网络隔离的,简直不要太痛苦。 prometheus的报警有时候不能够明确的反应哪里的问题

官方地址:github.com/AliyunConta…

how

按照github上新建一个钉钉机器人,把webhook记下来

image.png

后面的设置里面选择关键词

image.png

按照github上的yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    name: kube-eventer
  name: kube-eventer
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kube-eventer
  template:
    metadata:
      labels:
        app: kube-eventer
      annotations:	
        scheduler.alpha.kubernetes.io/critical-pod: ''
    spec:
      dnsPolicy: ClusterFirstWithHostNet
      serviceAccount: kube-eventer
      containers:
        - image: registry.aliyuncs.com/acs/kube-eventer-amd64:v1.2.0-484d9cd-aliyun
          name: kube-eventer
          command:
            - "/kube-eventer"
            - "--source=kubernetes:https://kubernetes.default"
            ## .e.g,dingtalk sink demo
            - --sink=dingtalk:[your_webhook_url]&label=[your_cluster_id]&level=[Normal or Warning(default)]
          env:
          # If TZ is assigned, set the TZ value as the time zone
          - name: TZ
            value: "Asia/Shanghai" 
          volumeMounts:
            - name: localtime
              mountPath: /etc/localtime
              readOnly: true
            - name: zoneinfo
              mountPath: /usr/share/zoneinfo
              readOnly: true
          resources:
            requests:
              cpu: 100m
              memory: 100Mi
            limits:
              cpu: 500m
              memory: 250Mi
      volumes:
        - name: localtime
          hostPath:
            path: /etc/localtime
        - name: zoneinfo
          hostPath:
            path: /usr/share/zoneinfo
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: kube-eventer
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - events
    verbs:
      - get
      - list
      - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: kube-eventer
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: kube-eventer
subjects:
  - kind: ServiceAccount
    name: kube-eventer
    namespace: kube-system
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: kube-eventer
  namespace: kube-system

别的都不用管,只需要配置- --sink=dingtalk:[your_webhook_url]&label=[your_cluster_id]&level=[Normal or Warning(default)] 这一行就可以了

label 填入dingding的自定义关键词cluster level 填入Warning 或者Normal

有需要的也可以加入namespace

最后的地址变为- --sink=dingtalk:https://xxxxx/robot/send?access_token=8867b9254aa5fff23bd8b9086b9cb12758538f3xxxxd118a13a3edf95b5bad&label=cluster&level=Warning

然后就是万能的kubectl apply 安装yaml

image.png

搞定!

PS: 生产环境是不能联网的,发送钉钉需要走层代理,怎么配置都不好使,最后在源码里面找到原因了。

github.com/AliyunConta…

` func (d *DingTalkSink) Ding(event *v1.Event) { value := url.Values{}

if d.Namespaces != nil {
    skip := true
    for _, namespace := range d.Namespaces {
        if namespace == event.Namespace {
            skip = false
            break
        }
    }
    if skip {
        return
    }
}

if d.Kinds != nil {
    skip := true
    for _, kind := range d.Kinds {
        if kind == event.InvolvedObject.Kind {
            skip = false
            break
        }
    }
    if skip {
        return
    }
}

msg := createMsgFromEvent(d, event)
if msg == nil {
    klog.Warningf("failed to create msg from event,because of %v", event)
    return
}

msg_bytes, err := json.Marshal(msg)
if err != nil {
    klog.Warningf("failed to marshal msg %v", msg)
    return
}

value.Set("access_token", d.Token)
if d.Secret != "" {
    t := time.Now().UnixNano() / 1e6
    value.Set("timestamp", fmt.Sprintf("%d", t))
    value.Set("sign", sign(t, d.Secret))
}

b := bytes.NewBuffer(msg_bytes)

request, err := http.NewRequest(http.MethodPost, fmt.Sprintf("https://%s", d.Endpoint), b)
if err != nil {
    klog.Errorf("failed to create http request")
    return
}
request.URL.RawQuery = value.Encode()
request.Header.Add("Content-Type", "application/json;charset=utf-8")
resp, err := (&http.Client{}).Do(request)
if err != nil {
    klog.Errorf("failed to send msg to dingtalk. error: %s", err.Error())
    return
}
defer resp.Body.Close()
if resp != nil && resp.StatusCode != http.StatusOK {
    klog.Errorf("failed to send msg to dingtalk, because the response code is %d", resp.StatusCode)
    return
}

}

`

request, err := http.NewRequest(http.MethodPost, fmt.Sprintf("https://%s", d.Endpoint), b)

代码里面把https写死了,如果有代理模块,需要改为http,在用dockerfile生成镜像。