宿主机混部doris(cgroup)

42 阅读2分钟

多BE模式服务器,/data1, /data2, /data3, /data4 四个磁盘,四个be

安装包在 /home/doris/doris/,安装目录/data1/be,数据目录 /data1/doris,用户 doris


Be conf Sample:

PPROF_TMPDIR="$DORIS_HOME/log/"
JAVA_HOME=/usr/local/java/jdk1.8.0_202
sys_log_level = INFO
be_port = 19061
webserver_port = 18041
heartbeat_service_port = 19051
brpc_port = 18061

storage_root_path = ${DORIS_HOME}/../doris,medium:SSD

priority_networks = 172.16.24.0/24
# 当前需要 mysql-connector-java-5.1.47.jar  ojdbc8.jar  postgresql-42.5.4.jar
jdbc_drivers_dir = /home/doris/apache-doris/jdbc_drivers

fragment_pool_thread_num_max = 10240
fragment_pool_queue_size = 4096
brpc_num_threads = 256
thrift_rpc_timeout_ms=60000
txn_commit_rpc_timeout_ms=60000
compaction_task_num_per_disk = 4
compaction_task_num_per_fast_disk = 8
mem_limit = 95%



4BE 服务器会通过Cgroup限定资源,参考如下:


cd /sys/fs/cgroup/
cgcreate -g cpuset,memory:doris_be1
cgcreate -g cpuset,memory:doris_be2
cgcreate -g cpuset,memory:doris_be3
cgcreate -g cpuset,memory:doris_be4

# cpu限制
echo 0-23 > cpuset/doris_be1/cpuset.cpus
echo 24-47 > cpuset/doris_be2/cpuset.cpus
echo 48-71 > cpuset/doris_be3/cpuset.cpus
echo 72-95 > cpuset/doris_be4/cpuset.cpus

echo 0-1 > cpuset/doris_be1/cpuset.mems
echo 0-1 > cpuset/doris_be2/cpuset.mems
echo 0-1 > cpuset/doris_be3/cpuset.mems
echo 0-1 > cpuset/doris_be4/cpuset.mems

# 内存限制
echo 64G > memory/doris_be1/memory.limit_in_bytes
echo 64G > memory/doris_be2/memory.limit_in_bytes
echo 64G > memory/doris_be3/memory.limit_in_bytes
echo 64G > memory/doris_be4/memory.limit_in_bytes

# be的CPU
chown -R doris:doris /sys/fs/cgroup/cpuset/doris_be1
chown -R doris:doris /sys/fs/cgroup/cpuset/doris_be2
chown -R doris:doris /sys/fs/cgroup/cpuset/doris_be3
chown -R doris:doris /sys/fs/cgroup/cpuset/doris_be4

# be的内存
chown -R doris:doris /sys/fs/cgroup/memory/doris_be1
chown -R doris:doris /sys/fs/cgroup/memory/doris_be2
chown -R doris:doris /sys/fs/cgroup/memory/doris_be3
chown -R doris:doris /sys/fs/cgroup/memory/doris_be4




服务通过supervisor做自动拉起,脚本参考如下:

[program:doris_be1]
environment=JAVA_HOME="/usr/local/java/jdk1.8.0_202",DORIS_HOME="/data1/be/"
process_name=%(program_name)s
directory=/data1/be/
command=cgexec -g cpuset:doris_be1 -g memory:doris_be1 --sticky sh /data1/be/bin/start_be.sh
autostart=true
autorestart=true
user=doris
numprocs=1
startretries=3
stopasgroup=true
killasgroup=true
startsecs=5
stopwaitsecs=10



FE连接信息:172.16.73.225:9030

下载包:Wget apache-doris-releases.oss-accelerate.aliyuncs.com/apache-dori…