yolov3 darknet 转 TVM 推理输出、一文读懂

墨理学AI
238 阅读4分钟

「这是我参与2022首次更文挑战的第29天，活动详情查看：2022首次更文挑战」

  • 🎉 声明: 作为全网 AI 领域 干货最多的博主之一，❤️ 不负光阴不负卿 ❤️

    • 1-0

    • 🍊 计算机视觉： Yolo专栏、一文读懂

    • 🍊 Yolo 系列推荐：yolov3 darknet 转 TVM Python 推理

    • 📆 最近更新：2022年2月20日

    • 🍊 点赞 👍 收藏 ⭐留言 📝 都是博主坚持写作、更新高质量博文的最大动力！

    • 🍊 👋 Follow me 👋，一起 Get 更多有趣 AI、冲冲冲 🚀 🚀

    📕 tvm 源码安装

    tvm 源码安装，可参考博文

    【初识TVM】| LLVM编译 | tvm 源码安装 | deploy ONNX models with Relay 测试【一文读懂】

    git clone --recursive https://github.com/apache/tvm.git

cd tvm

mkdir build

cp cmake/config.cmake build

cmake ..

make -j8
    • 直接 cmake .. 输出如下
    -- Found Threads: TRUE  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/moli/project/project21/modelTrans/tvm/build
    • make -j8 输出如下
    [100%] Built target tvm_objs
[100%] Linking CXX shared library libtvm.so
[100%] Built target tvm

    📕 此次运行Python代码如下

    该代码支持 YOLO-V2 and YOLO-V3 DarkNet Models 转 TVM 推理输出

    1. 模型下载【代码自动下载、网速不佳、也可手动下载】
    2. 模型转换【DarkNet Models 转 TVM 】
    3. 模型推理【TVM 推理示例】
    
"""
Compile YOLO-V2 and YOLO-V3 in DarkNet Models
=============================================



"""

# numpy and matplotlib
import numpy as np
import matplotlib.pyplot as plt
import sys

# tvm, relay
import tvm
from tvm import te
from tvm import relay
from ctypes import *
from tvm.contrib.download import download_testdata
from tvm.relay.testing.darknet import __darknetffi__
import tvm.relay.testing.yolo_detection
import tvm.relay.testing.darknet

######################################################################
# Choose the model
# -----------------------
# Models are: 'yolov2', 'yolov3' or 'yolov3-tiny'

# Model name
MODEL_NAME = "yolov3"

######################################################################
# Download required files
# -----------------------
# Download cfg and weights file if first time.
CFG_NAME = MODEL_NAME + ".cfg"
WEIGHTS_NAME = MODEL_NAME + ".weights"
REPO_URL = "https://github.com/dmlc/web-data/blob/main/darknet/"
CFG_URL = REPO_URL + "cfg/" + CFG_NAME + "?raw=true"
WEIGHTS_URL = "https://pjreddie.com/media/files/" + WEIGHTS_NAME

cfg_path = download_testdata(CFG_URL, CFG_NAME, module="darknet")
weights_path = download_testdata(WEIGHTS_URL, WEIGHTS_NAME, module="darknet")

# Download and Load darknet library
if sys.platform in ["linux", "linux2"]:
    DARKNET_LIB = "libdarknet2.0.so"
    DARKNET_URL = REPO_URL + "lib/" + DARKNET_LIB + "?raw=true"
elif sys.platform == "darwin":
    DARKNET_LIB = "libdarknet_mac2.0.so"
    DARKNET_URL = REPO_URL + "lib_osx/" + DARKNET_LIB + "?raw=true"
else:
    err = "Darknet lib is not supported on {} platform".format(sys.platform)
    raise NotImplementedError(err)

lib_path = download_testdata(DARKNET_URL, DARKNET_LIB, module="darknet")

DARKNET_LIB = __darknetffi__.dlopen(lib_path)
net = DARKNET_LIB.load_network(cfg_path.encode("utf-8"), weights_path.encode("utf-8"), 0)
dtype = "float32"
batch_size = 1

data = np.empty([batch_size, net.c, net.h, net.w], dtype)
shape_dict = {"data": data.shape}
print("Converting darknet to relay functions...")
mod, params = relay.frontend.from_darknet(net, dtype=dtype, shape=data.shape)

######################################################################
# Import the graph to Relay
# -------------------------
# compile the model
target = tvm.target.Target("llvm", host="llvm")
dev = tvm.cpu(0)
data = np.empty([batch_size, net.c, net.h, net.w], dtype)
shape = {"data": data.shape}
print("Compiling the model...")
with tvm.transform.PassContext(opt_level=3):
    lib = relay.build(mod, target=target, params=params)

[neth, netw] = shape["data"][2:]  # Current image shape is 608x608
######################################################################
# Load a test image
# -----------------
test_image = "dog.jpg"
print("Loading the test image...")
img_url = REPO_URL + "data/" + test_image + "?raw=true"
img_path = download_testdata(img_url, test_image, "data")

data = tvm.relay.testing.darknet.load_image(img_path, netw, neth)
######################################################################
# Execute on TVM Runtime
# ----------------------
# The process is no different from other examples.
from tvm.contrib import graph_executor

m = graph_executor.GraphModule(lib["default"](dev))

# set inputs
m.set_input("data", tvm.nd.array(data.astype(dtype)))
# execute
print("Running the test image...")

# detection
# thresholds
thresh = 0.5
nms_thresh = 0.45

m.run()
# get outputs
tvm_out = []
if MODEL_NAME == "yolov2":
    layer_out = {}
    layer_out["type"] = "Region"
    # Get the region layer attributes (n, out_c, out_h, out_w, classes, coords, background)
    layer_attr = m.get_output(2).numpy()
    layer_out["biases"] = m.get_output(1).numpy()
    out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
    layer_out["output"] = m.get_output(0).numpy().reshape(out_shape)
    layer_out["classes"] = layer_attr[4]
    layer_out["coords"] = layer_attr[5]
    layer_out["background"] = layer_attr[6]
    tvm_out.append(layer_out)

elif MODEL_NAME == "yolov3":
    for i in range(3):
        layer_out = {}
        layer_out["type"] = "Yolo"
        # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
        layer_attr = m.get_output(i * 4 + 3).numpy()
        layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
        layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
        out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
        layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
        layer_out["classes"] = layer_attr[4]
        tvm_out.append(layer_out)

elif MODEL_NAME == "yolov3-tiny":
    for i in range(2):
        layer_out = {}
        layer_out["type"] = "Yolo"
        # Get the yolo layer attributes (n, out_c, out_h, out_w, classes, total)
        layer_attr = m.get_output(i * 4 + 3).numpy()
        layer_out["biases"] = m.get_output(i * 4 + 2).numpy()
        layer_out["mask"] = m.get_output(i * 4 + 1).numpy()
        out_shape = (layer_attr[0], layer_attr[1] // layer_attr[0], layer_attr[2], layer_attr[3])
        layer_out["output"] = m.get_output(i * 4).numpy().reshape(out_shape)
        layer_out["classes"] = layer_attr[4]
        tvm_out.append(layer_out)
        thresh = 0.560

# do the detection and bring up the bounding boxes
img = tvm.relay.testing.darknet.load_image_color(img_path)
_, im_h, im_w = img.shape
dets = tvm.relay.testing.yolo_detection.fill_network_boxes(
    (netw, neth), (im_w, im_h), thresh, 1, tvm_out
)
last_layer = net.layers[net.n - 1]
tvm.relay.testing.yolo_detection.do_nms_sort(dets, last_layer.classes, nms_thresh)

coco_name = "coco.names"
coco_url = REPO_URL + "data/" + coco_name + "?raw=true"
font_name = "arial.ttf"
font_url = REPO_URL + "data/" + font_name + "?raw=true"
coco_path = download_testdata(coco_url, coco_name, module="data")
font_path = download_testdata(font_url, font_name, module="data")

with open(coco_path) as f:
    content = f.readlines()

names = [x.strip() for x in content]

tvm.relay.testing.yolo_detection.show_detections(img, dets, thresh, names, last_layer.classes)
tvm.relay.testing.yolo_detection.draw_detections(
    font_path, img, dets, thresh, names, last_layer.classes
)

plt.imshow(img.transpose(1, 2, 0))
plt.show()
plt.savefig("yolov3_infer.png")

"""
# 代码运行输出如下：

python yolov3_darknet_infer.py

File /home/moli/.tvm_test_data/darknet/yolov3.cfg exists, skip.
File /home/moli/.tvm_test_data/darknet/yolov3.weights exists, skip.
File /home/moli/.tvm_test_data/darknet/libdarknet2.0.so exists, skip.
Converting darknet to relay functions...
Compiling the model...
One or more operators have not been tuned. Please tune your model for better performance. Use DEBUG logging level to see more details.
Loading the test image...
File /home/moli/.tvm_test_data/data/dog.jpg exists, skip.
Running the test image...
File /home/moli/.tvm_test_data/data/coco.names exists, skip.
File /home/moli/.tvm_test_data/data/arial.ttf exists, skip.
class:['dog 0.994'] left:127 right:227 top:316 bottom:533
class:['truck 0.9266'] left:471 right:83 top:689 bottom:169
class:['bicycle 0.9984'] left:111 right:113 top:577 bottom:447

"""

    📕 yolov3 darknet 转 TVM 推理输出

    依赖库

    环境搭建比较常规、主要是顺利安装 tvm 即可

    0-1

    代码会自动下载相关模型和 dog.jpg 测试图片，到如下路径

    • /home/moli/.tvm_test_data

    0-2

    推理运行输出效果如下

    0-9

    📗 此次源码仓库地址

    0-11

    🚀🚀 墨理学AI

  • 🎉 作为全网 AI 领域 干货最多的博主之一，❤️ 不负光阴不负卿 ❤️
  • ❤️ 如果文章对你有帮助、点赞、评论鼓励博主的每一分认真创作

    • 快乐学AI 、深度学习环境搭建 : 一文读懂

    765_blue.jpg

    avatar
    文章
    阅读
    粉丝