提前安装依赖组件

检查Linux服务器上的VGA显卡信息，我们可以看到Nvidia GPU的具体型号

lspci -nn | egrep -i "3d|display|vga"

修改/etc/apt/sources.list,确保添加了contrib、non-free、 non-free-firmware组件

我的设置如下：

#deb cdrom:[Debian GNU/Linux 12.5.0 _Bookworm_ - Official amd64 DVD Binary-1 with firmware 20240210-11:28]/ bookworm contrib main non-free-firmware

deb http://mirrors.huaweicloud.com/debian/ bookworm main contrib non-free non-free-firmware
deb-src http://mirrors.huaweicloud.com/debian/ bookworm main contrib non-free non-free-firmware

deb http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware
deb-src http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware

# bookworm-updates, to get updates before a point release is made;
# see https://www.debian.org/doc/manuals/debian-reference/ch02.en.html#_updates_and_backports
deb http://mirrors.huaweicloud.com/debian/ bookworm-updates main contrib non-free non-free-firmware
deb-src http://mirrors.huaweicloud.com/debian/ bookworm-updates main contrib non-free non-free-firmware

先更新Linux并安装必要的编译组件

sudo apt update -y && sudo apt upgrade -y && sudo apt install vim gcc g++ make python3-pip -y
# 重启
sudo reboot

安装必要的nvidia gpu驱动依赖组件

sudo apt install nvidia-detect -y
sudo apt install linux-headers-amd64 linux-headers-$(uname -r) -y
# 重启
sudo reboot

检查Nvidia GPU硬件信息

nvidia-detect 

安装CUDA 12.3.1

打开CUDA Toolkit 12.3.1，选择Debian 12 X86_64版本并进行本地deb安装。

依次安装如下的命令

wget https://developer.download.nvidia.com/compute/cuda/12.3.1/local_installers/cuda-repo-debian12-12-3-local_12.3.1-545.23.08-1_amd64.deb
sudo dpkg -i cuda-repo-debian12-12-3-local_12.3.1-545.23.08-1_amd64.deb
sudo cp /var/cuda-repo-debian12-12-3-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt install software-properties-common -y
sudo add-apt-repository contrib
sudo apt-get update
sudo apt-get -y install cuda-toolkit-12-3
# 安装驱动
sudo apt-get install -y cuda-drivers firmware-misc-nonfree 

安装完成后进行检查

nvidia-smi
# 重启
sudo reboot

安装完成后再次检查

nvidia-smi

如果发现nvidia-smi无法正确显示显卡信息，可以参考Debian 12 安装Nvidia GPU Driver进行解决

这里我们就完成了CUDA的安装，关于cuDNN和TensorRT可以参考 Ubuntu版本的机器学习环境搭建部分,流程是一样的。