Colab复现Monodepth

花花少年
366 阅读5分钟

本文已参与「新人创作礼」活动，一起开启掘金创作之路。​

一、参考资料

源码

二、相关介绍

1. 运行环境

系统：Ubuntu16.04
CPU：AMD Ryzen 5600X 6核12线程
GPU：GeForce RTX 3060，12GB
内存：16GB
python：3.6.13
anconda虚拟环境名称：monodepth-gpu

2. requirements.txt

absl-py==0.13.0
astor==0.8.1
bleach==1.5.0
cached-property==1.5.2
certifi==2021.5.30
cycler==0.10.0
dataclasses==0.8
enum34==1.1.10
gast==0.5.0
google-pasta==0.2.0
grpcio==1.38.1
h5py==3.1.0
html5lib==0.9999999
importlib-metadata==4.6.1
Keras-Applications==1.0.8
Keras-Preprocessing==1.1.2
kiwisolver==1.3.1
Markdown==3.3.4
matplotlib==3.3.4
numpy==1.19.5
opencv-python==4.5.3.56
pandas==1.1.5
Pillow==8.3.1
protobuf==3.17.3
pyparsing==2.4.7
python-dateutil==2.8.2
pytz==2021.1
scipy==1.2.1
six==1.16.0
tensorboard==1.14.0
tensorflow-estimator==1.14.0
tensorflow-gpu==1.4.0
tensorflow-tensorboard==0.4.0
termcolor==1.1.0
typing-extensions==3.10.0.0
Werkzeug==2.0.1
wincertstore==0.2
wrapt==1.12.1
zipp==3.5.0

3. env.yaml

name: monodepth-gpu
channels:
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r
  - http://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/msys2
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=4.5=1_gnu
  - ca-certificates=2021.7.5=h06a4308_1
  - certifi=2021.5.30=py36h06a4308_0
  - ld_impl_linux-64=2.35.1=h7274673_9
  - libffi=3.3=he6710b0_2
  - libgcc-ng=9.3.0=h5101ec6_17
  - libgomp=9.3.0=h5101ec6_17
  - libstdcxx-ng=9.3.0=hd4cf53a_17
  - ncurses=6.2=he6710b0_1
  - openssl=1.1.1k=h27cfd23_0
  - pip=21.2.2=py36h06a4308_0
  - python=3.6.13=h12debd9_1
  - readline=8.1=h27cfd23_0
  - setuptools=52.0.0=py36h06a4308_0
  - sqlite=3.36.0=hc218d9a_0
  - tk=8.6.10=hbc83047_0
  - wheel=0.36.2=pyhd3eb1b0_0
  - xz=5.2.5=h7b6447c_0
  - zlib=1.2.11=h7b6447c_3
  - pip:
    - absl-py==0.13.0
    - astor==0.8.1
    - bleach==1.5.0
    - cached-property==1.5.2
    - cycler==0.10.0
    - dataclasses==0.8
    - enum34==1.1.10
    - gast==0.5.0
    - google-pasta==0.2.0
    - grpcio==1.38.1
    - h5py==3.1.0
    - html5lib==0.9999999
    - importlib-metadata==4.6.1
    - keras-applications==1.0.8
    - keras-preprocessing==1.1.2
    - kiwisolver==1.3.1
    - markdown==3.3.4
    - matplotlib==3.3.4
    - numpy==1.19.5
    - opencv-python==4.5.3.56
    - pandas==1.1.5
    - pillow==8.3.1
    - protobuf==3.17.3
    - pyparsing==2.4.7
    - python-dateutil==2.8.2
    - pytz==2021.1
    - scipy==1.2.1
    - six==1.16.0
    - tensorboard==1.14.0
    - tensorflow-estimator==1.14.0
    - tensorflow-gpu==1.4.0
    - tensorflow-tensorboard==0.4.0
    - termcolor==1.1.0
    - typing-extensions==3.10.0.0
    - werkzeug==2.0.1
    - wincertstore==0.2
    - wrapt==1.12.1
    - zipp==3.5.0
prefix: /home/yoyo/miniconda3/envs/monodepth-gpu

4. 训练集数据

https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_26_drive_0001/2011_09_26_drive_0001_sync.zip

/home/yoyo/MyDocuments/DataSets/KITTI
|   
\---2011_09_26
    \---2011_09_26_drive_0001_sync
        +---image_00  # left_image左视图（灰度图）
        |   |   timestamps.txt
        |   |   
        |   \---data
        |           0000000000.png
        |           0000000001.png
        |           
        +---image_01  # right_image右视图（灰度图）
        |   |   timestamps.txt
        |   |   
        |   \---data
        |           0000000000.png
        |           0000000001.png
        |           
        +---image_02  # left_image左视图（彩色图）
        |   |   timestamps.txt
        |   |   
        |   \---data
        |           0000000000.png
        |           0000000001.png
        |           
        +---image_03  # right_image右视图（彩色图）
        |   |   timestamps.txt
        |   |   
        |   \---data
        |           0000000000.png
        |           0000000001.png
        |           
        +---oxts
        |   |   dataformat.txt
        |   |   timestamps.txt
        |   |   
        |   \---data
        |           0000000000.txt
        |           0000000001.txt
        |           
        \---velodyne_points
            |   timestamps.txt
            |   timestamps_end.txt
            |   timestamps_start.txt
            |   
            \---data
                    0000000000.bin
                    0000000001.bin

5. 测试集数据

Stereo Evaluation 2015

https://s3.eu-central-1.amazonaws.com/avg-kitti/data_scene_flow.zip

├── data_scene_flow
│   ├── testing
│   │   ├── image_2
│   │   └── image_3
│   └── training
│       ├── disp_noc_0
│       ├── disp_noc_1
│       ├── disp_occ_0
│       ├── disp_occ_1
│       ├── flow_noc
│       ├── flow_occ
│       ├── image_2  # 左视图
│       ├── image_3  # 右视图
│       ├── obj_map
│       ├── viz_flow_occ
│       └── viz_flow_occ_dilate_1

6. 项目结构

.
├── average_gradients.py
├── bilinear_sampler.py
├── env.yaml  # 项目运行的anaconda虚拟环境
├── image_disp.npy # 生成的视差数据
├── image_disp.png  # 生成的视差图
├── image.png  # kitti数据集中选取的一张RGB图片
├── LICENSE
├── models  # 新建的文件夹，用于存放models
│   ├── model_cityscapes  # citiscapes预训练模型
│   │   ├── model_cityscapes.data-00000-of-00001
│   │   ├── model_cityscapes.index
│   │   └── model_cityscapes.meta
│   └── model_kitti  # kitti预训练模型
│       ├── model_kitti.data-00000-of-00001
│       ├── model_kitti.index
│       └── model_kitti.meta
├── monodepth_dataloader.py
├── monodepth_main.py
├── monodepth_model.py
├── monodepth_simple.py
├── my_image.jpg  # citiscape数据集中选取的一张RGB图片
├── Notes  # 博主跑代码过程中记录的笔记
│   └── Notes.md
├── __pycache__
├── readme.md
├── requirements-gpu.txt  # 项目运行所依赖的GPU环境
├── requirements.txt  # 项目运行所依赖的CPU环境
└── utils
    ├── evaluate_kitti.py
    ├── evaluation_utils.py
    ├── filenames
    │   ├── cityscapes_test_files.txt
    │   ├── cityscapes_train_files.txt
    │   ├── cityscapes_val_files.txt
    │   ├── eigen_test_files.txt
    │   ├── eigen_train_files.txt
    │   ├── eigen_val_files.txt
    │   ├── kitti_stereo_2015_test_files.txt
    │   ├── kitti_stereo_2015_test_files-副本.txt
    │   ├── kitti_test_files.txt
    │   ├── kitti_train_files.txt
    │   ├── kitti_train_files_副本.txt
    │   └── kitti_val_files.txt
    ├── get_model.sh
    ├── kitti_archives_to_download.txt
    └── __pycache__
        └── evaluation_utils.cpython-36.pyc

三、简单测试

  1. 本地电脑（笔记本电脑）跑通项目代码，导出依赖包：
pip freeze > requirements.txt
  1. 将项目代码上传到google云盘；
  2. Colab挂载google云盘；
from google.colab import drive
drive.mount('/content/drive')
  1. 切换到工作目录；
import os
os.chdir('/content/drive/MyDrive/monodepth-master')
  1. 安装pip；
!apt-get install python3-pip
  1. 安装依赖包，配置运行环境；
!python3.6 -m pip install -r ./requirements.txt
  1. 安装其他依赖包（运行程序报错，有提示）；
!python3.6 -m pip install ipykernel
  1. 运行程序；
!python3.6 monodepth_simple.py --image_path ./image.png --checkpoint_path ./models/model_cityscapes/model_cityscapes
  1. 运行成功； 在这里插入图片描述

四、关键步骤

1. 将KITTI数据集的png图像转换成jpg格式

如果没有安装 parallel，请先安装

sudo apt-get update -y
sudo apt-get install -y parallel

find /PATH/TO/data_scene_flow/training/image_2/ -name '*.png' | parallel 'convert {.}.png {.}.jpg && rm {}'

2. 训练模型

python monodepth_main.py --mode train --model_name model_kitti --data_path /home/yoyo/MyDocuments/DataSets/KITTI/ --filenames_file /home/yoyo/MyDocuments/PyProjects/monodepth-master/utils/filenames/kitti_train_files.txt --log_directory /home/yoyo/MyDocuments/tmp/

3.测试模型

python monodepth_main.py \
--mode test \
--data_path /media/yoyo/新加卷/KITTI/KITTI-stereo/data_scene_flow/training/ \
--filenames_file /home/yoyo/MyDocuments/PyProjects/monodepth-master/utils/filenames/kitti_stereo_2015_test_files.txt \
--log_directory /home/yoyo/MyDocuments/tmp/ \
--checkpoint_path /home/yoyo/MyDocuments/tmp/model_kitti/model-7000

输出结果

now testing 200 files
done.
writing disparities.
done.

4. 评估模型

python utils/evaluate_kitti.py \
--split kitti \
--predicted_disp_path /home/yoyo/MyDocuments/tmp/model_kitti/disparities.npy \
--gt_path /media/yoyo/新加卷/KITTI/KITTI-stereo/data_scene_flow

输出结果

abs_rel,     sq_rel,        rms,    log_rms,     d1_all,         a1,         a2,         a3
 0.3612,     6.6820,     10.291,      0.395,     47.482,      0.658,      0.802,      0.886

五、重要技巧

  • 下载数据集
wget -i utils/kitti_archives_to_download.txt -P ~/my/output/folder/

kitti_archives_to_download.txt文件里的下载连接，可以在Colab中执行上面的指令，然后从Colab中手动下载到本地（注意磁盘空间）。

六、可能存在的问题

  • model模型找不到
tensorflow.python.framework.errors_impl.DataLossError: Unable to open table file ./models/model_cityscapes: Failed precondition: models/model_cityscapes; Is a directory: perhaps your file is in a different file format and you need to use a different restore operator?
         [[node save/RestoreV2 (defined at monodepth_simple.py:66) ]]

python monodepth_simple.py --image_path ./image.png --checkpoint_path ./models/model_cityscapes

错误原因：
model模型目录错误

解决办法：
python monodepth_simple.py --image_path ./image.png --checkpoint_path ./models/model_cityscapes/model_cityscapes
  • TypeError: TF_SessionRun_wrapper: expected all values in input dict to be ndarray
错误原因：
执行指令出错：
>>> python monodepth_simple.py --image_path ./image.png --checkpoint_path ./models/model_cityscapes/model_cityscapes

说明：
./models/model_cityscapes/model_cityscapes目录中有以下3个模型文件
model_cityscapes.data-00000-of-00001
model_cityscapes.index
model_cityscapes.meta

解决办法：
修改 monodepth_simple.py 中的代码:
# RESTORE
# restore_path = args.checkpoint_path.split(".")[0]
restore_path = args.checkpoint_path # 修改之后
train_saver.restore(sess, restore_path)
  • Tensorflow-gpu版本与CUDA版本不匹配
ImportError: DLL load failed: 找不到指定的模块。

  Failed to load the native TensorFlow runtime.

  See https://www.tensorflow.org/install/errors

  for some common reasons and solutions.  Include the entire stack trace
  above this error message when asking for help.

错误原因：
Tensorflow-gpu版本与CUDA版本不匹配

解决办法：
1. 版本对齐，[官网链接](https://tensorflow.google.cn/install/source)
2. 先安装Tensorflow-gpu，再安装对应版本的CUDA
  • CUDA未加载
ImportError: Could not find 'cudart64_100.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable. Download and install CUDA 10.0 from this URL: https://developer.nvidia.com/cuda-90-download-archive

  ImportError: Could not find 'cudart64_100.dll'. TensorFlow requires that this DLL be installed in a directory that is named in your %PATH% environment variable

错误原因：
刚安装CUDA，Pycharm未能加载CUDA

解决办法：
1. 重启Pycharm
2. 如果重启Pycharm失败，则重启电脑
  • scipy的版本问题
File "monodepth_simple.py", line 55, in test_simple
  ​    input_image = scipy.misc.imread(args.image_path, mode="RGB")
  AttributeError: module 'scipy.misc' has no attribute 'imread'

错误原因：
scipy的版本问题

解决办法：
1. 卸载scipy
2. 降低scipy版本
>>> pip install scipy==1.2.1
  • 训练集错误，导致读取数据错误
python monodepth_main.py --mode train --model_name model_kitti --data_path E:\我的文档\datasets\KITTI\ --filenames_file E:\我的文档\MyDocuments\PyProjects\monodepth-master\utils\filenames\kitti_train_files.txt --log_directory E:\我的文档\tmp\

OutOfRangeError (see above for traceback): RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size
0)
         [[Node: shuffle_batch = QueueDequeueManyV2[component_types=[DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](shuffle_batch/r
andom_shuffle_queue, shuffle_batch/n/_453)]]
         [[Node: model/ResizeArea_1/_849 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/gpu:0", send_device="/job:localhost/replica:0/task
:0/cpu:0", send_device_incarnation=1, tensor_name="edge_5870_model/ResizeArea_1", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/gpu:0"]()]]

tensorflow.python.framework.errors_impl.OutOfRangeError: 2 root error(s) found.
  (0) Out of range: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
  ​         [[{{node shuffle_batch}}]]
  (1) Out of range: RandomShuffleQueue '_1_shuffle_batch/random_shuffle_queue' is closed and has insufficient elements (requested 1, current size 0)
  ​         [[{{node shuffle_batch}}]][[Mean_45/_197]]
  0 successful operations.
  0 derived errors ignored.

参考资料：
[Tensorflow调用tf.train.shuffle_batch函数报错“OutOfRangeError: RandomShuffleQueue”](https://blog.csdn.net/admin_maxin/article/details/86547084)
[](https://github.com/mrharicot/monodepth/issues/89)
[](https://github.com/mrharicot/monodepth/issues/238)

错误原因：
tensorflow读取训练数据出错，导致模型训练失败

解决办法：
去KITTI官网下载官方数据集训练
  • 找不到图片路径
Traceback (most recent call last):
  File "utils/evaluate_kitti.py", line 25, in <module>
    gt_disparities = load_gt_disp_kitti(args.gt_path)
  File "/home/yoyo/MyDocuments/PyProjects/monodepth-master/utils/evaluation_utils.py", line 40, in load_gt_disp_kitti
    disp = disp.astype(np.float32) / 256
AttributeError: 'NoneType' object has no attribute 'astype'

错误原因：
图片路径错误，opencv读取图片失败

解决办法：
检查图像路径，检查路径拼接
avatar
文章
阅读
粉丝