docker部署
skywalking持久化
本文使用mysql持久化,需要将mysql-connector-java.jar(根据各自版本下载)上传skywalking-oap容器内部/skywalking/oap-libs/目录下。
docker-compose.yml
version: '3'
services:
m_oap:
image: apache/skywalking-oap-server:8.9.0
container_name: m_oap
restart: unless-stopped
ports:
- 11800:11800 # agent 上报数据的端口,这是 gRPC 端口
- 12800:12800 # ui 读取数据的端口, 这是 http 端口
environment:
- TZ=Asia/Shanghai
- SW_STORAGE=mysql
- SW_JDBC_URL="jdbc:mysql://127.0.0.1:3306/skywalking_db"
- SW_DATA_SOURCE_USER=***
- SW_DATA_SOURCE_PASSWORD=****
volumes:
- ./jar/mysql-connector-j-8.2.0.jar:/skywalking/oap-libs/mysql-connector-j-8.2.0.jar
m_skywaling-ui:
image: apache/skywalking-ui:8.9.1
container_name: ms_ui
restart: unless-stopped
depends_on:
- m_oap
links:
- m_oap
ports:
- 8088:8080
#volumes:
# - ./uiconfig/webapp.yml:/skywalking/webapp/webapp.yml
environment:
- SW_OAP_ADDRESS=http://127.0.0.1:12800
- TZ=Asia/Shanghai
访问监控ui页面
下载java agent
官网地址skywalking.apache.org/downloads/,下载java agent并解压
idea内服务配置探针
-javaagent:文件路径
-Dskywalking.agent.service_name=服务名
-Dskywalking.collector.backend_service= skywalking地址
kubesphere 部署skywalking
- 配置字典(ConfigMap)yaml
kind: ConfigMap
apiVersion: v1
metadata:
name: skywalking-config
namespace: ****
annotations:
kubesphere.io/alias-name: skywalking-config
kubesphere.io/creator: admin
data:
SW_DATA_SOURCE_PASSWORD: ***
SW_DATA_SOURCE_USER: ***
SW_JDBC_URL: 'jdbc:mysql://127.0.0.1:3306/skywalking_db'
SW_OAP_ADDRESS: 'http://skywalking-oap-12800.****.svc.cluster.local:12800'
SW_STORAGE: mysql
TZ: Asia/Shanghai
创建持久卷,mysql-connector-j-8.2.0.jar 上传到持久卷目录下,挂载到容器内
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: data-grafana
namespace: cq-test-inside
annotations:
kubesphere.io/alias-name: grafana
kubesphere.io/creator: admin
pv.kubernetes.io/bind-completed: 'yes'
pv.kubernetes.io/bound-by-controller: 'yes'
volume.beta.kubernetes.io/storage-provisioner: test-nfs-storage
finalizers:
- kubernetes.io/pvc-protection
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 2Gi
volumeName: pvc-ab26b967-1139-44dd-af82-cdd99eaacc5e
storageClassName: managed-nfs-storage
volumeMode: Filesystem
工作负载yaml
skywalking-server
kind: Deployment
apiVersion: apps/v1
metadata:
name: skywalking-server
namespace: ****-test-inside
labels:
app: skywalking-server
annotations:
deployment.kubernetes.io/revision: '3'
kubesphere.io/creator: admin
spec:
replicas: 1
selector:
matchLabels:
app: skywalking-server
template:
metadata:
creationTimestamp: null
labels:
app: skywalking-server
annotations:
kubesphere.io/creator: admin
kubesphere.io/imagepullsecrets: '{}'
logging.kubesphere.io/logsidecar-config: '{}'
spec:
volumes:
- name: volume-pmrvoz
persistentVolumeClaim:
claimName: data-grafana
containers:
- name: skywalking
image: '***/skywalking-oap-server:8.9.0'
ports:
- name: tcp-11800
containerPort: 11800
protocol: TCP
- name: tcp-12800
containerPort: 12800
protocol: TCP
env:
- name: SW_DATA_SOURCE_PASSWORD
valueFrom:
configMapKeyRef:
name: skywalking-config
key: SW_DATA_SOURCE_PASSWORD
- name: SW_DATA_SOURCE_USER
valueFrom:
configMapKeyRef:
name: skywalking-config
key: SW_DATA_SOURCE_USER
- name: SW_JDBC_URL
valueFrom:
configMapKeyRef:
name: skywalking-config
key: SW_JDBC_URL
- name: SW_STORAGE
valueFrom:
configMapKeyRef:
name: skywalking-config
key: SW_STORAGE
- name: TZ
valueFrom:
configMapKeyRef:
name: skywalking-config
key: TZ
resources:
limits:
cpu: '1'
memory: 2Gi
requests:
cpu: 500m
memory: 1Gi
volumeMounts:
- name: volume-pmrvoz
mountPath: skywalking/oap-libs/mysql-connector-j-8.2.0.jar
subPath: mysql-connector/mysql-connector-j-8.2.0.jar
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
serviceAccountName: default
serviceAccount: default
securityContext: {}
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
revisionHistoryLimit: 10
progressDeadlineSeconds: 600
skywalking-ui-server
kind: Deployment
apiVersion: apps/v1
metadata:
name: skywaling-ui-server
namespace: ****-test-inside
labels:
app: skywaling-ui-server
annotations:
deployment.kubernetes.io/revision: '2'
kubesphere.io/creator: admin
spec:
replicas: 1
selector:
matchLabels:
app: skywaling-ui-server
template:
metadata:
creationTimestamp: null
labels:
app: skywaling-ui-server
annotations:
kubesphere.io/creator: admin
kubesphere.io/imagepullsecrets: '{}'
spec:
containers:
- name: skywalking-ui
image: '******/skywalking-ui:8.9.1'
ports:
- name: tcp-8080
containerPort: 8080
protocol: TCP
env:
- name: SW_OAP_ADDRESS
valueFrom:
configMapKeyRef:
name: skywalking-config
key: SW_OAP_ADDRESS
- name: TZ
valueFrom:
configMapKeyRef:
name: skywalking-config
key: TZ
resources:
limits:
cpu: '1'
memory: 2Gi
requests:
cpu: 500m
memory: 1Gi
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirst
serviceAccountName: default
serviceAccount: default
securityContext: {}
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
revisionHistoryLimit: 10
progressDeadlineSeconds: 600
配置服务(service),外部访问端口
kind: Service
apiVersion: v1
metadata:
name: skywalking-ui-18088
namespace: ****-test-inside
labels:
app: skywalking-ui-18088
annotations:
kubesphere.io/creator: admin
spec:
ports:
- name: http-18088
protocol: TCP
port: 8080
targetPort: 8080
nodePort: 32741
selector:
app: skywaling-ui-server
clusterIP: 10.96.0.168
clusterIPs:
- 10.96.0.168
type: NodePort
sessionAffinity: None
externalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
internalTrafficPolicy: Cluster
kind: Service
apiVersion: v1
metadata:
name: skywalking-oap-12800
namespace: ****-test-inside
labels:
app: skywalking-oap-12800
annotations:
kubesphere.io/creator: admin
spec:
ports:
- name: http-12800
protocol: TCP
port: 12800
targetPort: 12800
nodePort: 30631
- name: http-11800
protocol: TCP
port: 11800
targetPort: 11800
nodePort: 30557
selector:
app: skywalking-server
clusterIP: 10.96.3.92
clusterIPs:
- 10.96.3.92
type: NodePort
sessionAffinity: None
externalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
internalTrafficPolicy: Cluster
业务服务添加java agent,业务负载启动时初始化javaagent,挂载容器文件
kind: Deployment
apiVersion: apps/v1
metadata:
name: mmmm-server
namespace: hh-test-inside
labels:
app: mmmm-server
annotations:
deployment.kubernetes.io/revision: '45'
kubectl.kubernetes.io/last-applied-configuration: >
{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"annotations":{"deployment.kubernetes.io/revision":"4","kubesphere.io/creator":"allcore"},"labels":{"app":"mmmm-server"},"name":"mmmm-server","namespace":"hh-test-inside"},"spec":{"progressDeadlineSeconds":600,"replicas":1,"revisionHistoryLimit":10,"selector":{"matchLabels":{"app":"mmmm-server"}},"strategy":{"rollingUpdate":{"maxSurge":"25%","maxUnavailable":"25%"},"type":"RollingUpdate"},"template":{"metadata":{"annotations":{"kubesphere.io/restartedAt":"2023-03-06T04:13:39.843Z"},"labels":{"app":"mmmm-server"}},"spec":{"containers":[{"env":[{"name":"NACOS_CONFIG","valueFrom":{"configMapKeyRef":{"key":"NACOS_CONFIG","name":"common-conf"}}},{"name":"NACOS_DISCOVERY","valueFrom":{"configMapKeyRef":{"key":"NACOS_DISCOVERY","name":"common-conf"}}},{"name":"NACOS_NAMESPACE","valueFrom":{"configMapKeyRef":{"key":"NACOS_NAMESPACE","name":"common-conf"}}},{"name":"SPRING_PROFILES_ACTIVE","valueFrom":{"configMapKeyRef":{"key":"SPRING_PROFILES_ACTIVE","name":"common-conf"}}},{"name":"TZ","valueFrom":{"configMapKeyRef":{"key":"TZ","name":"common-conf"}}},{"name":"JAVA_OPTS","value":"-Xmx512m
-Xms512m -Xmn256m -XX:MaxMetaspaceSize=256M -XX:MetaspaceSize=128M
-XX:+PrintGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps
-XX:+HeapDumpOnOutOfMemoryError
-XX:+UseG1GC"}],"image":"zhmg.io/hhffh3.0.0/mmmm-server:2.0.0.RELEASE","imagePullPolicy":"Always","name":"container-mmmm-server","ports":[{"containerPort":18096,"name":"tcp-18096","protocol":"TCP"}],"resources":{"limits":{"cpu":"500m","memory":"1024Mi"},"requests":{"cpu":"200m","memory":"256Mi"}},"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","volumeMounts":[{"mountPath":"/etc/localtime","name":"host-time","readOnly":true}]}],"dnsPolicy":"ClusterFirstWithHostNet","hostNetwork":false,"imagePullSecrets":[{"name":"docker-harbor"}],"restartPolicy":"Always","schedulerName":"default-scheduler","securityContext":{},"serviceAccount":"default","serviceAccountName":"default","terminationGracePeriodSeconds":30,"volumes":[{"hostPath":{"path":"/etc/localtime","type":""},"name":"host-time"}]}}}}
kubesphere.io/creator: allcore
spec:
replicas: 1
selector:
matchLabels:
app: mmmm-server
template:
metadata:
creationTimestamp: null
labels:
app: mmmm-server
annotations:
kubesphere.io/imagepullsecrets: '{}'
kubesphere.io/restartedAt: '2024-06-12T06:29:18.297Z'
logging.kubesphere.io/logsidecar-config: '{}'
spec:
volumes:
- name: host-time
hostPath:
path: /etc/localtime
type: ''
- name: skywalking-agent
emptyDir: {}
initContainers:
- name: agent-container
image: 'apache/skywalking-java-agent:8.5.0-alpine'
command:
- /bin/sh
args:
- '-c'
- cp -R /skywalking/agent /agent/
resources: {}
volumeMounts:
- name: skywalking-agent
mountPath: /agent
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: IfNotPresent
containers:
- name: container-mmmm-server
image: 'zhmg.io/hhffh3.0.0/mmmm-server:2.0.0.RELEASE'
ports:
- name: tcp-18096
containerPort: 18096
protocol: TCP
env:
- name: NACOS_CONFIG
valueFrom:
configMapKeyRef:
name: common-conf
key: NACOS_CONFIG
- name: NACOS_DISCOVERY
valueFrom:
configMapKeyRef:
name: common-conf
key: NACOS_DISCOVERY
- name: NACOS_NAMESPACE
valueFrom:
configMapKeyRef:
name: common-conf
key: NACOS_NAMESPACE
- name: SPRING_PROFILES_ACTIVE
valueFrom:
configMapKeyRef:
name: common-conf
key: SPRING_PROFILES_ACTIVE
- name: TZ
valueFrom:
configMapKeyRef:
name: common-conf
key: TZ
- name: lokiUrl
valueFrom:
configMapKeyRef:
name: common-conf
key: lokiUrl
- name: JAVA_OPTS
value: >-
-Xmx512m -Xms512m -Xmn256m -XX:MaxMetaspaceSize=256M
-XX:MetaspaceSize=128M -XX:+PrintGC -XX:+PrintGCDetails
-XX:+PrintGCDateStamps -XX:+HeapDumpOnOutOfMemoryError
-XX:+UseG1GC -javaagent:/skywalking/agent/skywalking-agent.jar
-Dskywalking.agent.service_name=skywalking-mmmm-service
-Dskywalking.collector.backend_service=skywalking-oap-12800.hh-test-inside.svc.cluster.local:11800
resources:
limits:
cpu: 500m
memory: 1Gi
requests:
cpu: 200m
memory: 256Mi
volumeMounts:
- name: skywalking-agent
mountPath: /skywalking
- name: host-time
readOnly: true
mountPath: /etc/localtime
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
imagePullPolicy: Always
restartPolicy: Always
terminationGracePeriodSeconds: 30
dnsPolicy: ClusterFirstWithHostNet
serviceAccountName: default
serviceAccount: default
securityContext: {}
schedulerName: default-scheduler
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 25%
maxSurge: 25%
revisionHistoryLimit: 10
progressDeadlineSeconds: 600
日志logback集成
pom依赖
<!--skywalking日志集成-->
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>apm-toolkit-logback-1.x</artifactId>
<version>8.14.0</version>
</dependency>
<!-- 该引用用于代码获取tranceId -->
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>apm-toolkit-trace</artifactId>
<version>8.14.0</version>
</dependency>
<!--end skywalking集成-->
<!--xml.value解析表达式-->
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
<version>3.0.6</version>
</dependency>
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<springProperty scope="context" name="applicationName" source="spring.application.name" defaultValue="demo" />
<springProperty scope="context" name="loglevel" source="logging.level.root" defaultValue="info" />
<!-- <springProperty name="lokiUrl" scope="context" source="loki.url"/>-->
<!-- # 新增sky日志的开关,yml中配置-->
<springProperty scope="context" name="skyLogEnable" source="skywalking.log.enable" defaultValue="false"/>
<!--日志渲染-->
<conversionRule conversionWord="clr" converterClass="org.springframework.boot.logging.logback.ColorConverter" />
<conversionRule conversionWord="wex" converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
<conversionRule conversionWord="wEx" converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
<property name="LOG_PATH" value="logs/${applicationName}"/>
<property name="MAX_FILE_SIZE" value="10MB"/>
<property name="MAX_HISTORY" value="30"/>
<!-- skywalking日志样式 -->
<property name="SKY_WALKING_PATTERN" value="[%tid] ${SKY_WALKING_PATTERN:-%clr(%d{${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}}){faint} %clr(${LOG_LEVEL_PATTERN:-%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39} %line){cyan} %clr(:){faint} %m%n${LOG_EXCEPTION_CONVERSION_WORD:-%wEx}}"/>
<!-- 控制台日志样式 -->
<property name="CONSOLE_LOG_PATTERN" value="${CONSOLE_LOG_PATTERN:-%d{[${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}]}-[${LOG_LEVEL_PATTERN:-%5p}]-[%-30.30logger{29}] : %m%n}"/>
<!-- 文件日志样式 -->
<property name="FILE_LOG_PATTERN" value="${FILE_LOG_PATTERN:-%d{[${LOG_DATEFORMAT_PATTERN:-yyyy-MM-dd HH:mm:ss.SSS}]}-[${LOG_LEVEL_PATTERN:-%5p}]-[%-30.30logger{29}] : %m%n}"/>
<!-- 控制台 -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
</encoder>
</appender>
<!-- debug日志文件 -->
<appender name="DEBUG_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_PATH}/debug.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/debug-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<maxFileSize>${MAX_FILE_SIZE}</maxFileSize>
<maxHistory>${MAX_HISTORY}</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- 过滤的级别 -->
<level>DEBUG</level>
</filter>
</appender>
<!-- 日志文件 -->
<appender name="INFO_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_PATH}/info.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/info-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<maxFileSize>${MAX_FILE_SIZE}</maxFileSize>
<maxHistory>${MAX_HISTORY}</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- 过滤的级别 -->
<level>INFO</level>
</filter>
</appender>
<!-- 错误日志文件 我们在k8s环境下 没必要做错误日志文件 -->
<appender name="ERROR_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder>
<pattern>${FILE_LOG_PATTERN}</pattern>
</encoder>
<file>${LOG_PATH}/error.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/error-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<maxFileSize>${MAX_FILE_SIZE}</maxFileSize>
<maxHistory>${MAX_HISTORY}</maxHistory>
</rollingPolicy>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- 过滤的级别 -->
<level>INFO</level>
</filter>
<!-- <filter class="ch.qos.logback.classic.filter.LevelFilter">-->
<!-- <level>ERROR</level>-->
<!-- <onMatch>ACCEPT</onMatch>-->
<!-- <onMismatch>DENY</onMismatch>-->
<!-- </filter>-->
</appender>
<!-- <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">-->
<!-- <http class="com.github.loki4j.logback.ApacheHttpSender">-->
<!-- <url>${lokiUrl}</url>-->
<!-- </http>-->
<!-- <format>-->
<!-- <label>-->
<!-- <pattern>app=${appName},host=${HOSTNAME},level=%level</pattern>-->
<!-- </label>-->
<!-- <message>-->
<!-- <pattern>l=%level h=${HOSTNAME} c=%logger{20} [%tid] t=%thread | %msg %ex</pattern>-->
<!-- </message>-->
<!-- <sortByTime>true</sortByTime>-->
<!-- </format>-->
<!-- </appender>-->
<!-- 异步写日志 -->
<!-- <appender name="ASYNC_FILE" class="ch.qos.logback.classic.AsyncAppender">-->
<!-- <discardingThreshold>0</discardingThreshold>-->
<!-- <queueSize>1024</queueSize>-->
<!-- <appender-ref ref ="DEBUG_FILE"/>-->
<!-- </appender>-->
<!-- <appender name="ASYNC_ERROR_FILE" class="ch.qos.logback.classic.AsyncAppender">
<discardingThreshold>0</discardingThreshold>
<queueSize>1024</queueSize>
<appender-ref ref ="ERROR_FILE"/>
</appender> -->
<!-- skywalking控制台 -->
<appender name="SKY_WALKING_CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.TraceIdPatternLogbackLayout">
<pattern>${SKY_WALKING_PATTERN}</pattern>
</layout>
</encoder>
</appender>
<!--推送日志至管理后台-->
<appender name="GRPC_LOG" class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.log.GRPCLogClientAppender">
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="org.apache.skywalking.apm.toolkit.log.logback.v1.x.mdc.TraceIdMDCPatternLogbackLayout">
<pattern>%date [%file:%line]-%msg%n</pattern>
</layout>
</encoder>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
<root level="INFO">
<!-- <appender-ref ref="ASYNC_FILE"/>-->
<if condition='${skyLogEnable}'> # js表达式
<then>
<appender-ref ref="SKY_WALKING_CONSOLE"/>
<appender-ref ref="GRPC_LOG"/>
</then>
<else>
<appender-ref ref="CONSOLE"/>
</else>
</if>
</root>
<!-- Spring日志级别控制 -->
<logger name="com.allcore" level="info">
<appender-ref ref="INFO_FILE"/>
</logger>
<!-- 控制台的日志级别 -->
<root level="debug">
<if condition='${skyLogEnable}'> # js表达式
<then>
<appender-ref ref="SKY_WALKING_CONSOLE"/>
<appender-ref ref="GRPC_LOG"/>
</then>
<else>
<appender-ref ref="CONSOLE"/>
</else>
</if>
<!-- <appender-ref ref="LOKI" />-->
</root>
<!--系统操作日志-->
<root level="info">
<appender-ref ref="ERROR_FILE" />
<appender-ref ref="DEBUG_FILE" />
</root>
</configuration>
调试
访问服务,拿到日志traceId搜索链路日志,查看链路服务记录响应信息