[性能工具] Locust 应用-单接口压测

波小艺
387 阅读3分钟
  • 标题：[性能测试工具] Locust 应用篇之单接口压测
  • 分类：性能测试
  • 标签：test、performtest
  • 简介：之前发了一篇文章[性能测试工具] Locust学习之基础篇，专门介绍Locust。这篇文章主要介绍我是如何使用Locust做单接口性能测试的。

之前发了一篇文章[性能测试工具] Locust学习之基础篇，专门介绍Locust。现在专门讲一下我是如何使用Locust做单接口性能测试的。

locustfile_example

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# @Time:2022/11/16 11:27
# @Author:boyizhang
import json
import random

import gevent.monkey
from locust.clients import ResponseContextManager

gevent.monkey.patch_all()

from locust import TaskSet, task, HttpUser


def read_shop_id_from_txt(fileName):
    """
    read shop_ids from txt
    :return: list
    """
    basic_shop_ids = []
    with open(fileName, 'r') as f:
        for row in f.readlines():
            basic_shop_ids.append(int(row))

    return basic_shop_ids


class ListingLimitTask(TaskSet):
    @task
    def get_shop_listing_limit(self):
        # random get shop_id
        shop_ids = random.sample(self.user.basic_shop_ids, 1)

        path = '/your_api_path_xxxxxx'
        # 拼装请求
        payload = json.dumps({
            "shop_ids": shop_ids
        })
        headers = {}
        # 接口调用
        with self.client.get(path, headers=headers, data=payload, catch_response=True) as res:
            res: ResponseContextManager
            # 如果不满足，则标记为failure
            # print(f"res:{res.text}")
            if res.status_code != 200:
                print(f'res:{res.text},body:{payload}')
                res.failure(res.text)

    def on_start(self):
        """
        read basic shop_id group by region
        :return:
        """
        print('start stress...')


class ListingLimitUser(HttpUser):
    host = 'https://xxxxxxxx.xxxx'

    tasks = [ListingLimitTask, ]

    print('user start...')
    # read shop id from file
    basic_shop_ids = read_shop_id_from_txt('listing_limit_shop_ids.txt')
    print('user start done.')
    region = 'ID'


if __name__ == '__main__':
    # debug
    # run_single_user(ListingLimitUser)
    # locust -f listinglimit.py --headless --users 30 --spawn-rate 10 -H https://xxx.xxx -t 300s
    pass
  • 通过read_shop_id_from_txt()从文件中读取测试数据到缓存中。
  • 启动Locust，并开始压测时，根据random.sample(self.user.basic_shop_ids, 1)去随机获取shop_id。
  • 拿到测试数据：shop_id之后，拼装请求，并发起接口调用请求。

locust worker 更新实例的脚本

Locust 支持分布式压测。
我们可以通过--master、--worker分启动master实例与worker实例进行分布式压测。
另外我们也可以通过docker来启动分布式压测。
如果不想在机器上安装docker，那就通过shell脚本来启动master与worker实例，来控制worker实例的数量，使用方式如下：

  • locust_worker_start.sh
    • sh locust_worker_start.sh [worker实例的启动数量]
  • locust_worker_stop.sh
    • sh locust_worker_stop.sh [进程搜索关键字，locust] [worker实例关闭的数量]
#!/bin/bash
# master: locust -f locustfile_example.py --headless --users 30 --spawn-rate 10 -H https://xxx.xxx -t 300s --master --expect-workers 3
# 检查参数个数
if [ $# -ne 1 ]; then
  echo "Usage: bash $0 worker_num"
  exit 1
fi
num=1
worker_num=$1
#echo "-----" >> pid.log
while(( $num<=$worker_num ))
do
    echo "worker - $num is starting"
		nohup locust -f locustfile_example.py --worker --master-host=127.0.0.1  >> worker.log 2>&1 &
    let "num++"
done


#!/bin/bash

# 检查参数个数
if [ $# -ne 2 ]; then
  echo "Usage: bash $0 keyword num_processes"
  exit 1
fi

# 获取搜索关键字和要关闭的进程个数
keyword=$1
num_processes=$2
grep_worker_label="grep -v $0 |grep -ie "--worker""

# 搜索进程并获取前n个进程ID
command="ps aux | grep -i "$keyword" | $grep_worker_label | head -n $num_processes | awk '{print \$2}'"
echo run command: $command
pids=$(eval $command)
echo pids: $pids
# 判断是否找到进程
if [ -z "$pids" ]; then
  echo "No process found with keyword '$keyword'"
else
  # 关闭进程
  for pid in $pids; do
    kill -9 $pid
  done
  echo "Selected processes $pids have been closed."
fi

脚本压测流程

通过以上脚本的压测流程大致如下：

  • 启动master：locust -f locustfile_example.py --headless --users 30 --spawn-rate 10 -H https://xxx.xxx -t 300s --master --expect-workers 3��
    • 备注：通过w键增加1个用户，通过W键增加10个用户。通过s键减少一个用户，通过S键减少10个用户。
  • 启动worker： sh locust_worker_start.sh 3
  • 关闭一定数量的worker：sh locust_worker_stop.sh locust 2

locust worker 脚本优化

也可以将locust_worker_start.sh 与 locust_worker_stop.sh合并成一个脚本，使用如下：
sh locust_worker.sh [keyword] [expect_worker_count]

#!/bin/bash

# 检查参数个数
if [ $# -ne 2 ]; then
  echo "Usage: bash $0 keyword expect_worker_count"
  exit 1
fi

grep_worker_label="grep -v $0 |grep -ie "--worker""
# 获取搜索关键字和要关闭的进程个数
keyword=$1
expect_worker_count=$2

function get_pids_count() {
  exist_pids_command="ps aux | grep -ie "$keyword" | $grep_worker_label  | grep -v $0 | awk '{print \$2}'"
  echo run command: $exist_pids_command
  exist_pids=$(eval $exist_pids_command)
  exist_pid_list=(${exist_pids// / })
  echo "数组的元素为: ${exist_pid_list[*]}"
  exist_pids_count=${#exist_pid_list[*]}
  return $exist_pids_count
}

function remove_worker_instance() {
  # 搜索进程并获取前n个进程ID
  remove_worker_num=$1
  command="ps aux | grep -i "$keyword" | $grep_worker_label | head -n $remove_worker_num | awk '{print \$2}'"
  echo run command: $command
  pids=$(eval $command)
  echo pids: $pids
  # 判断是否找到进程
  if [ -z "$pids" ]; then
    echo "No process found with keyword '$keyword'"
  else
    # 关闭进程
    for pid in $pids; do
      kill -9 $pid
    done
    echo "Selected processes $pids have been closed."
  fi
}
function create_worker_instance() {
  worker_num=$1
  num=1
  while (($num <= $worker_num)); do
    echo "worker - $num is starting"
    nohup locust -f locustfile_example.py --worker --master-host=127.0.0.1 >>worker.log 2>&1 &
    let "num++"
  done
}

get_pids_count
exist_pids_count=$?
# master: locust -f locustfile_example.py --headless --users 30 --spawn-rate 10 -H https://xxx.xxx -t 300s --master --expect-workers 3
# 检查参数个数
echo "expect_worker_count: $expect_worker_count, exist_pids_count: $exist_pids_count"

if (($exist_pids_count < $expect_worker_count)); then
  echo "add worker instance count"
  let worker_num=$expect_worker_count-$exist_pids_count
  create_worker_instance $worker_num

elif (($exist_pids_count > $expect_worker_count)); then
  echo "remove worker instance count"
  let remove_worker_num=$exist_pids_count-$expect_worker_count
  remove_worker_instance $remove_worker_num
fi

get_pids_count
exist_pids_count=$?
echo "worker instance count: $exist_pids_count"

脚本压测流程

通过以上脚本的压测流程大致如下：

  • 启动master：locust -f locustfile_example.py --headless --users 30 --spawn-rate 10 -H https://xxx.xxx -t 300s --master --expect-workers 3��
    • 备注：通过w键增加1个用户，通过W键增加10个用户。通过s键减少一个用户，通过S键减少10个用户。
  • 启动worker或者更新worker实例的数量： sh locust_worker.sh locust 3
avatar
文章
阅读
粉丝