快速高效实现Django日志的治理之ddtrace

69 阅读1分钟

注册观测云账号

先注册一个账号, 注册地址

注册流程很简单,按照提示点点点就完事了

安装采集器Datakit

登录观测云页面,选择「集成」 - 「DataKit」,选择适合自己环境的安装指令,复制。 我这里是linux环境,其他小伙伴可以按照提示选择对应的安装脚本。

1 复制安装指令

image.png

2 在服务器上安装 DataKit

DK_DATAWAY="https://openway.guance.com?token=**************" bash -c "$(curl -L https://static.guance.com/datakit/install.sh)"

安装完成,服务会自动启动,程序默认安装在/usr/local/datakit/

3 查询 DataKit 状态

执行命令 systemctl status datakit

4 查看数据

Datakit 安装好后,默认会采集一些数据,可在 「观测云」 - 「基础设施」 - 「主机」查看相关数据

LOG

django配置日志

setting.py中设置日志格式


import ddtrace

# 设置datakit接收链路数据ip地址与端口
ddtrace.tracer.configure(
    hostname='127.0.0.1',
    port='9529'
)



os.environ['DD_SERVICE'] = 'DJANGO_TEST'   # 设置服务名
os.environ['DD_TAGS'] = 'project:django_test,env:test,version:v1'   # 设置环境名 设置版本号
os.environ['DD_LOGS_INJECTION'] = 'true'   # 开启log注入


LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'simple': {
            'format': '%(asctime)s - %(name)s - %(levelname)s - %(dd.service)s - %(dd.trace_id)s - %(dd.span_id)s - %(message)s'
        },
        'verbose': {
            # '()': DDTraceLoggingFormatter,
            # 'format': '{levelname} {asctime} {module} {process:d} {thread:d} {message}',
            # 'style': '{',
            'format': '%(asctime)s - %(name)s - %(levelname)s - TraceID:%(trace_id)s - SpanID:%(span_id)s - Service:%(service)s - %(message)s'
        },
    },
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
            'formatter': 'simple',  # 使用自定义的日志格式化程序
        },
        'file': {
            'level': 'DEBUG',  # 设置日志级别
            'class': 'logging.FileHandler',  # 使用 FileHandler 处理器
            'filename': 'myapp.log',  # 日志文件名
            'formatter': 'simple',  # 使用自定义的日志格式化程序
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console','file'],
            'level': 'INFO',  # 设置日志级别,可以根据需求调整
            'propagate': False,
        },
    },
}

view.py中配置

from django.shortcuts import render,HttpResponse
from django.http import JsonResponse

# Create your views here.
from app01.utils.logger import log


def Test(request):
    log.info('hello world')
    return render(request,'index.html')
def Getuser(request):
    data = {"message": "zhangsan"}
    log.info('get user')
    return JsonResponse(data)

url.py中配置

from django.contrib import admin
from django.urls import path

from app01 import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('test/', views.Test),
    path('get_user/', views.Getuser),
]

通过如下命令开启 Python 应用:

ddtrace-run python manage.py runserver 0.0.0.0:8082 

访问后,查看日志数据

...
2023-08-30 05:52:39,765 - django.server - WARNING - DJANGO_TEST - 0 - 0 - "GET /otel_test/ HTTP/1.1" 404 2331
2023-08-30 05:52:45,120 - django - INFO - DJANGO_TEST - 8114496497421676062 - 9327308312721880308 - hello world
2023-08-30 05:52:45,124 - django.server - INFO - DJANGO_TEST - 0 - 0 - "GET /test/ HTTP/1.1" 200 1721
2023-08-30 05:52:47,397 - django - INFO - DJANGO_TEST - 8633758971550947871 - 17177623937713897994 - get user
2023-08-30 05:52:47,398 - django.server - INFO - DJANGO_TEST - 0 - 0 - "GET /get_user/ HTTP/1.1" 200 37
2023-08-30 05:52:55,202 - django - INFO - DJANGO_TEST - 6964601362188057752 - 4924727743778452383 - hello world
2023-08-30 05:52:55,204 - django.server - INFO - DJANGO_TEST - 0 - 0 - "GET /test/ HTTP/1.1" 200 1721
...

通过上面的日志可以看到,链路的traceid和spanid被记录到日志中了

日志收集

设置/usr/local/datakit/conf.d/log/logging.conf

# {"version": "1.9.2", "desc": "do NOT edit this line"}

[[inputs.logging]]
  ## Required
  ## File names or a pattern to tail.
  logfiles = [
    "/usr/local/my_python/ddtrace_test/myapp.log",
  ]

  # Only two protocols are supported:TCP and UDP.
  # sockets = [
  #         "tcp://0.0.0.0:9530",
  #         "udp://0.0.0.0:9531",
  # ]
  ## glob filteer
  ignore = [""]

  ## Your logging source, if it's empty, use 'default'.
  source = "django_test"

  ## Add service tag, if it's empty, use $source.
  service = "django_test"

  ## Grok pipeline script name.
  pipeline = ""

  ## optional status:
  ##   "emerg","alert","critical","error","warning","info","debug","OK"
  ignore_status = []

  ## optional encodings:
  ##    "utf-8", "utf-16le", "utf-16le", "gbk", "gb18030" or ""
  character_encoding = ""

  ## The pattern should be a regexp. Note the use of '''this regexp'''.
  ## regexp link: https://golang.org/pkg/regexp/syntax/#hdr-Syntax
  multiline_match = '''[0-9]{4}-[0-9]{2}-[0-9]{2}'''

  auto_multiline_detection = true
  auto_multiline_extra_patterns = []

  ## Removes ANSI escape codes from text strings.
  remove_ansi_escape_codes = false

  ## If the data sent failure, will retry forevery.
  blocking_mode = true

  ## If file is inactive, it is ignored.
  ## time units are "ms", "s", "m", "h"
  ignore_dead_log = "1h"

  ## Read file from beginning.
  from_beginning = false

  [inputs.logging.tags]
  # some_tag = "some_value"
  # more_tag = "some_other_value"

设置日志pipeline 选择「日志」 - 「Pipeline」,这里可以设置样本,直接调试,非常人性化

image.png

做完日志切割以后,我们看一下切割后的日志

image.png 详细日志记录可以通过显示列添加字段,也可以点击某条日志查看字段

image.png

当日志收集上来以后,可以做一些大盘展示,还有监控告警,可玩性很高,可以自己尝试一下。