携手创作，共同成长！这是我参与「掘金日新计划 · 8 月更文挑战」的第20天，点击查看活动详情

一、FastDeploy 安装

由于编译配置较为困难（本地环境可能有误），建议大家使用已编译好的whl包。

FastDeploy 预编编译Python Wheel包

FastDeploy提供了在Windows/Linux/Mac上的预先编译Python Wheel包，开发者可以直接下载后安装，也可以自行编译代码。

使用如下命令即可在Linux的Pythotn 3.8环境安装支持CPU部署的FastDeploy

python -m pip install fastdeploy_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl

1.注意事项

不要重复安装fastdeploy-python和fastdeploy-gpu-python
如已安装CPU版本的fastdeploy-python后，在安装GPU版本的fastdeploy-gpu-python，请先执行pip uninstall fastdeploy-python卸载已有版本

2.环境依赖

cuda >= 11.2
cudnn >= 8.0

3.下载地址

3.1 Linux x64平台

CPU 安装包	硬件	Python版本
fastdeploy_python-0.2.0-cp36-cp36m-manylinux1_x86_64.whl	CPU	3.6
fastdeploy_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl	CPU	3.7
fastdeploy_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl	CPU	3.8
fastdeploy_python-0.2.0-cp39-cp39-manylinux1_x86_64.whl	CPU	3.9

GPU 安装包	硬件	Python版本
fastdeploy_gpu_python-0.2.0-cp36-cp36m-manylinux1_x86_64.whl	CPU/GPU	3.6
fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl	CPU/GPU	3.7
fastdeploy_gpu_python-0.2.0-cp38-cp38-manylinux1_x86_64.whl	CPU/GPU	3.8
fastdeploy_gpu_python-0.2.0-cp39-cp39-manylinux1_x86_64.whl	CPU/GPU	3.9

3.2 Windows 10 x64平台

CPU 安装包	硬件	Python版本
fastdeploy_python-0.2.0-cp38-cp38-win_amd64.whl	CPU	3.8
fastdeploy_python-0.2.0-cp39-cp39-win_amd64.whl	CPU	3.9

GPU 安装包	硬件	Python版本
fastdeploy_gpu_python-0.2.0-cp38-cp38-win_amd64.whl	CPU/GPU	3.8
fastdeploy_gpu_python-0.2.0-cp39-cp39-win_amd64.whl	CPU/GPU	3.9

3.3 Linux aarch64平台

安装包	硬件	Python版本
fastdeploy_python-0.2.0-cp36-cp36m-manylinux_2_17_aarch64.whl	CPU	3.6
fastdeploy_python-0.2.0-cp37-cp37m-manylinux_2_17_aarch64.whl	CPU	3.7
fastdeploy_python-0.2.0-cp38-cp38-manylinux_2_17_aarch64.whl	CPU	3.8
fastdeploy_python-0.2.0-cp39-cp39-manylinux_2_17_aarch64.whl	CPU	3.9

3.4 Mac OSX平台

架构	硬件	安装包	Python版本
x86_64	CPU	fastdeploy_python-0.2.0-cp36-cp36m-macosx_10_14_x86_64.whl	3.6
x86_64	CPU	fastdeploy_python-0.2.0-cp37-cp37m-macosx_10_14_x86_64.whl	3.7
x86_64	CPU	fastdeploy_python-0.2.0-cp38-cp38-macosx_10_14_x86_64.whl	3.8
x86_64	CPU	fastdeploy_python-0.2.0-cp39-cp39-macosx_10_14_x86_64.whl	3.9
arm64	CPU	fastdeploy_python-0.2.0-cp38-cp38-macosx_11_0_arm64.whl	3.8
arm64	CPU	fastdeploy_python-0.2.0-cp39-cp39-macosx_11_0_arm64.whl	3.9

4.其它文档

二、实践

1.git下载

 # git下载
 !git clone -b develop https://github.com/PaddlePaddle/FastDeploy.git --depth=1

Cloning into 'FastDeploy'...
remote: Enumerating objects: 1253, done.[K
remote: Counting objects: 100% (1253/1253), done.[K
remote: Compressing objects: 100% (965/965), done.[K
remote: Total 1253 (delta 309), reused 1046 (delta 242), pack-reused 0[K
Receiving objects: 100% (1253/1253), 2.37 MiB | 57.00 KiB/s, done.
Resolving deltas: 100% (309/309), done.
Checking connectivity... done.

2.查看系统环境

!python --version

Python 3.7.4

!nvidia-smi

Wed Aug 17 21:21:49 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  On   | 00000000:43:00.0 Off |                    0 |
| N/A   42C    P0    41W / 300W |      0MiB / 32510MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

可见环境满足条件

python 3.7
cuda 11.2

3.linux安装

!pip install https://bj.bcebos.com/paddlehub/fastdeploy/wheels/fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Collecting fastdeploy-gpu-python==0.2.0
  Downloading https://bj.bcebos.com/paddlehub/fastdeploy/wheels/fastdeploy_gpu_python-0.2.0-cp37-cp37m-manylinux1_x86_64.whl (1508.4 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m1.5/1.5 GB[0m [31m711.9 kB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hRequirement already satisfied: tqdm in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastdeploy-gpu-python==0.2.0) (4.27.0)
Requirement already satisfied: requests in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from fastdeploy-gpu-python==0.2.0) (2.24.0)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (1.25.6)
Requirement already satisfied: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (2.8)
Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests->fastdeploy-gpu-python==0.2.0) (2019.9.11)
Installing collected packages: fastdeploy-gpu-python
Successfully installed fastdeploy-gpu-python-0.2.0

[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m A new release of pip available: [0m[31;49m22.1.2[0m[39;49m -> [0m[32;49m22.2.2[0m
[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpip install --upgrade pip[0m

!python -m pip install --upgrade pip --user

Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: pip in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (22.1.2)
Collecting pip
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/1f/2c/d9626f045e7b49a6225c6b09257861f24da78f4e5f23af2ddbdf852c99b8/pip-22.2.2-py3-none-any.whl (2.0 MB)
[2K     [90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━[0m [32m2.0/2.0 MB[0m [31m4.1 MB/s[0m eta [36m0:00:00[0m00:01[0m00:01[0m
[?25hInstalling collected packages: pip
Successfully installed pip-22.2.2

[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m A new release of pip available: [0m[31;49m22.1.2[0m[39;49m -> [0m[32;49m22.2.2[0m
[1m[[0m[34;49mnotice[0m[1;39;49m][0m[39;49m To update, run: [0m[32;49mpython -m pip install --upgrade pip[0m

4.运行

4.1下载PaddleSeg

!git clone https://gitee.com/PaddlePaddle/PaddleSeg.git --depth=1

Cloning into 'PaddleSeg'...
remote: Enumerating objects: 1815, done.[K
remote: Counting objects: 100% (1815/1815), done.[K
remote: Compressing objects: 100% (1450/1450), done.[K
remote: Total 1815 (delta 485), reused 1227 (delta 287), pack-reused 0[K
Receiving objects: 100% (1815/1815), 48.94 MiB | 9.42 MiB/s, done.
Resolving deltas: 100% (485/485), done.
Checking connectivity... done.

4.2 获取预训练模型

%cd ~/PaddleSeg/
!mkdir bisenet
%cd bisenet
!wget https://paddleseg.bj.bcebos.com/dygraph/cityscapes/bisenet_cityscapes_1024x1024_160k/model.pdparams
%cd ../..

/home/aistudio/PaddleSeg
mkdir: cannot create directory ‘bisenet’: File exists
/home/aistudio/PaddleSeg/bisenet
--2022-08-17 21:41:56--  https://paddleseg.bj.bcebos.com/dygraph/cityscapes/bisenet_cityscapes_1024x1024_160k/model.pdparams
Resolving paddleseg.bj.bcebos.com (paddleseg.bj.bcebos.com)... 100.67.200.6
Connecting to paddleseg.bj.bcebos.com (paddleseg.bj.bcebos.com)|100.67.200.6|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 15301992 (15M) [application/octet-stream]
Saving to: ‘model.pdparams’

model.pdparams      100%[===================>]  14.59M  88.4MB/s    in 0.2s    

2022-08-17 21:41:56 (88.4 MB/s) - ‘model.pdparams’ saved [15301992/15301992]

/home/aistudio

4.3导出静态模型

%cd ~/PaddleSeg/

# 设置1张可用的卡
!export CUDA_VISIBLE_DEVICES=0
# windows下请执行以下命令
# set CUDA_VISIBLE_DEVICES=0
!python export.py \
       --config configs/bisenet/bisenet_cityscapes_1024x1024_160k.yml \
       --model_path bisenet/model.pdparams \
       --save_dir output \
       --input_shape 1 3 512 1024

/home/aistudio/PaddleSeg
/home/aistudio/PaddleSeg/paddleseg/models/losses/rmi_loss.py:78: DeprecationWarning: invalid escape sequence \i
  """
W0817 21:43:59.497787  2170 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 10.1
W0817 21:43:59.501492  2170 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6.
2022-08-17 21:44:01 [INFO]	Loaded trained params of model successfully.
2022-08-17 21:44:03 [INFO]	Model is saved in output.

!ls output/

deploy.yaml  model.pdiparams  model.pdiparams.info  model.pdmodel

三、预测

在部署前，需确认以下两个步骤

1. 软硬件环境满足要求，参考FastDeploy环境要求
1. FastDeploy Python whl包安装，参考FastDeploy Python安装

本目录下提供infer.py快速完成Unet在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成

- 下载部署示例代码
git clone https://github.com/PaddlePaddle/FastDeploy.git
cd FastDeploy/examples/vision/segmentation/paddleseg/python

# 下载Unet模型文件和测试图片
wget https://bj.bcebos.com/paddlehub/fastdeploy/Unet_cityscapes_without_argmax_infer.tgz
tar -xvf Unet_cityscapes_without_argmax_infer.tgz
wget https://paddleseg.bj.bcebos.com/dygraph/demo/cityscapes_demo.png


# CPU推理
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device cpu
# GPU推理
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu
# GPU上使用TensorRT推理 （注意：TensorRT推理第一次运行，有序列化模型的操作，有一定耗时，需要耐心等待）
python infer.py --model Unet_cityscapes_without_argmax_infer --image cityscapes_demo.png --device gpu --use_trt True

运行完成可视化结果如下图所示

1. PaddleSegModel Python接口

fd.vision.segmentation.PaddleSegModel(model_file, params_file, config_file, runtime_option=None, model_format=Frontend.PADDLE)

PaddleSeg模型加载和初始化，其中model_file, params_file以及config_file为训练模型导出的Paddle inference文件，具体请参考其文档说明模型导出

参数

model_file(str): 模型文件路径

params_file(str): 参数文件路径

config_file(str): 推理部署配置文件

runtime_option(RuntimeOption): 后端推理配置，默认为None，即采用默认配置

model_format(Frontend): 模型格式，默认为Paddle格式

2.predict函数

PaddleSegModel.predict(input_image)
模型预测结口，输入图像直接输出检测结果。

参数

input_image(np.ndarray): 输入数据，注意需为HWC，BGR格式

返回

返回fastdeploy.vision.SegmentationResult结构体，结构体说明参考文档视觉模型预测结果

3.类成员属性

3.1 预处理参数

用户可按照自己的实际需求，修改下列预处理参数，从而影响最终的推理和部署效果

is_vertical_screen(bool): PP-HumanSeg系列模型通过设置此参数为true表明输入图片是竖屏，即height大于width的图片

3.2 后处理参数

with_softmax(bool): 当模型导出时，并未指定with_softmax参数，可通过此设置此参数为true，将预测的输出分割标签（label_map）对应的概率结果(score_map)做softmax归一化处理

FastDeploy 安装部署

一、FastDeploy 安装

1.注意事项

2.环境依赖

3.下载地址

3.1 Linux x64平台

3.2 Windows 10 x64平台

3.3 Linux aarch64平台

3.4 Mac OSX平台

4.其它文档

二、实践

1.git下载

2.查看系统环境

3.linux安装

4.运行

4.1下载PaddleSeg

4.2 获取预训练模型

4.3导出静态模型

三、预测

1. PaddleSegModel Python接口

2.predict函数

3.类成员属性

3.1 预处理参数

3.2 后处理参数

4.其它文档