麒麟系统docker pull 卡死在extracting

1,599 阅读1分钟

背景

这两天在某Arm64架构下的国产麒麟系统执行docker pull 指令时,发现一直卡死在extracting环节,但是执行docker pull hello-world时,又是正常的,经过很多天的尝试一直无果。

image.png

image.png

转机

今天尝试查找资料的时候,无意中看到一篇解决国产系统 Docker 拉取大镜像卡顿之谜-腾讯云开发者社区-腾讯云 (tencent.com)文章,想起了我之前执行htop的时候,也发现了后台有很多的unpigz进程,占用了相当多的cpu资源。于是想到可能就是这个原因。

按照文章提示,采用debug模式启动docker,并禁用pgiz,后续pull的过程无比顺畅

MOBY_DISABLE_PIGZ=true dockerd --debug

image.png

随后尝试修改service文件,并重载配置,发现无效,pull的时候仍然会调用unpigz导致卡死

[Unit]
 Description=Docker Application Container Engine
 Documentation=https://docs.docker.com
 After=network-online.target firewalld.service
 Wants=network-online.target
 [Service]
 Type=notify
 # the default is not to use systemd for cgroups because the delegate issues still
 # exists and systemd currently does not support the cgroup feature set required
 # for containers run by docker
 ExecStart=MOBY_DISABLE_PIGZ=true /usr/bin/dockerd
 ExecReload=/bin/kill -s HUP $MAINPID
 # Having non-zero Limit*s causes performance problems due to accounting overhead
 # in the kernel. We recommend using cgroups to do container-local accounting.
 LimitNOFILE=infinity
 LimitNPROC=infinity
 LimitCORE=infinity
 # Uncomment TasksMax if your systemd version supports it.
 # Only systemd 226 and above support this version.
 #TasksMax=infinity
 TimeoutStartSec=0
 # set delegate yes so that systemd does not reset the cgroups of docker containers
 Delegate=yes
 # kill only the docker process, not all processes in the cgroup
 KillMode=process
 # restart the docker process if it exits prematurely
 Restart=on-failure
 StartLimitBurst=3
 StartLimitInterval=60s
 [Install]
 WantedBy=multi-user.target

后来想起来,我之前以为是docker安装方式有问题,从手动安装改成了官方yum源安装,所以最新的service文件应该是/usr/lib/systemd/system/docker.service,而不是之前的/etc/systemd/system/docker.service

这样直接修改service的启动命令会报错,参考这篇文章python项目适配arm架构国产麒麟系统 - 个人文章 - SegmentFault 思否,添加个environment参数就可以了

[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service
Wants=network-online.target

[Service]
Type=notify
Environment="MOBY_DISABLE_PIGZ=true"
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
Environment=GOTRACEBACK=crash

ExecStart=/usr/bin/dockerd $OPTIONS \
                           $DOCKER_STORAGE_OPTIONS \
                           $DOCKER_NETWORK_OPTIONS \
                           $INSECURE_REGISTRY
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
# set delegate yes so that systemd does not reset the cgroups of docker containers
Delegate=yes
# kill only the docker process, not all processes in the cgroup
KillMode=process

[Install]
WantedBy=multi-user.target