使用aria2c从Hugging下载模型

悟忧
142 阅读2分钟

从一个文本文件中读取多个模型下载链接，并自动根据 URL 中的路径（如 loras, checkpoints, vae 等）创建子目录，将模型下载到指定的根目录下。

✅ 功能亮点：

  • 从 urls.txt 文件读取下载链接
  • 自动提取 URL 中的子路径（如 /loras/, /checkpoints/）
  • 在目标目录中创建对应结构
  • 使用 aria2c 多线程下载
  • 支持断点续传
  • 跳过已存在的文件（可选）
  • 输出清晰日志

📁 文件结构示例

models/
└── loras/
    └── wan2.2_t2v_lightx2v_4steps_lora_v1.1_low_noise.safetensors
└── checkpoints/
    └── wan2.2_t2v_fp16.safetensors

📜 1. 准备 urls.txt 文件（示例内容）

https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/loras/wan2.2_t2v_lightx2v_4steps_lora_v1.1_low_noise.safetensors
https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/checkpoints/wan2.2_t2v_fp16.safetensors
https://huggingface.co/Comfy-Org/Wan_2.2_ComfyUI_Repackaged/resolve/main/split_files/vae/wan2.2_vae.safetensors

✅ 每行一个 URL

📜 2. 批量下载脚本 batch_download.sh

#!/bin/bash

# ========================================
# 批量下载模型脚本
# 使用 aria2c 从 urls.txt 下载模型
# 自动按 URL 路径结构保存
# ========================================

# 配置项
URLS_FILE="urls.txt"            # 存放下载链接的文件
BASE_DIR="./models"             # 模型根目录（可修改）
LOG_FILE="./download.log"       # 日志文件
FAILED_FILE="./download_failed.txt"  # 记录失败的链接

# aria2c 参数
ARIA2_OPTS="-x 16 -s 16 --continue=true --max-connection-per-server=16 --split=16"

# 清空日志
echo "Batch download started at $(date)" > "$LOG_FILE"
echo "Base directory: $BASE_DIR" >> "$LOG_FILE"
echo "" >> "$LOG_FILE"

# 检查依赖
if ! command -v aria2c &> /dev/null; then
    echo "❌ Error: aria2c is not installed. Please install it first."
    echo "   On Ubuntu: sudo apt install aria2"
    echo "   On macOS: brew install aria2"
    exit 1
fi

# 检查 urls.txt 是否存在
if [ ! -f "$URLS_FILE" ]; then
    echo "❌ Error: $URLS_FILE not found!"
    echo "Please create a file with one model URL per line."
    exit 1
fi

# 创建基础目录
mkdir -p "$BASE_DIR"

# 清空失败记录
> "$FAILED_FILE"

# 读取每个 URL 并下载
while IFS= read -r URL || [ -n "$URL" ]; do
    # 跳过空行和注释
    [[ -z "$URL" || "$URL" =~ ^[[:space:]]*# ]] && continue

    echo "🔗 Processing: $URL"

    # 提取 URL 中 '/split_files/' 后的路径（兼容 Hugging Face 结构）
    if [[ "$URL" =~ /split_files/(.*) ]]; then
        REL_PATH="${BASH_REMATCH[1]}"
    else
        echo "⚠️  Skipping (not a /split_files/ path): $URL"
        echo "$URL" >> "$FAILED_FILE"
        continue
    fi

    # 分离目录和文件名
    DIRNAME=$(dirname "$REL_PATH")
    FILENAME=$(basename "$REL_PATH")

    # 构建本地路径
    LOCAL_DIR="$BASE_DIR/$DIRNAME"
    LOCAL_PATH="$LOCAL_DIR/$FILENAME"

    # 创建目录
    mkdir -p "$LOCAL_DIR"

    # 检查是否已存在
    if [ -f "$LOCAL_PATH" ]; then
        echo "✅ Already exists, skipping: $LOCAL_PATH"
        echo "$URL" >> "$LOG_FILE"
        echo "  -> Skipped (already exists)" >> "$LOG_FILE"
        continue
    fi

    # 下载模型
    echo "⬇️  Downloading: $FILENAME"
    echo "   to: $LOCAL_PATH"

    aria2c $ARIA2_OPTS -o "$LOCAL_PATH" "$URL"

    # 检查下载结果
    if [ $? -eq 0 ]; then
        echo "✅ Success: $FILENAME"
        echo "$URL" >> "$LOG_FILE"
        echo "  -> $LOCAL_PATH" >> "$LOG_FILE"
    else
        echo "❌ Failed: $URL"
        echo "$URL" >> "$FAILED_FILE"
        echo "$URL" >> "$LOG_FILE"
        echo "  -> FAILED" >> "$LOG_FILE"
    fi

    echo "" >> "$LOG_FILE"
    sleep 1

done < "$URLS_FILE"

# 总结
echo "========================================"
echo "✅ Batch download completed."
echo "   Models saved to: $BASE_DIR"
if [ -s "$FAILED_FILE" ]; then
    echo "❌ Some downloads failed. See $FAILED_FILE"
    FAILED_COUNT=$(wc -l < "$FAILED_FILE")
    echo "   Failed: $FAILED_COUNT files"
else
    echo "✅ All downloads succeeded!"
fi
echo "📝 Log saved to: $LOG_FILE"
echo "========================================"

[[aria2c批量下载脚本-记录日志]]

🧪 使用方法：

  1. **创建 urls.txt**，每行一个模型 URL
  2. **保存脚本为 **batch_download.sh
  3. 赋予执行权限：
chmod +x batch_download.sh
  1. 运行脚本：
./batch_download.sh

💡 提示与扩展

  • **修改 **BASE_DIR 可指定 ComfyUI 模型目录，例如：
BASE_DIR="/home/user/comfyui/models"
  • 支持 Hugging Face resolve/main 类型链接
  • 可加入 --check-certificate=false 如果遇到 SSL 问题（不推荐生产环境）
  • 可扩展为支持重命名、校验 checksum 等功能

需要我帮你生成一个适用于 ComfyUI 常见模型（LoRA, Checkpoint, VAE, ControlNet）的 urls.txt 示例吗？

avatar
文章
阅读
粉丝