Windows搭建agent环境

Windows编译llama.cpp

Windows安装python

1. 下载模型

国内使用modelscope镜像

www.modelscope.cn/models

安装依赖

uv pip install modelscope
uv pip install transformers
uv pip install torch
uv pip install sentencepiece

2. 下载模型

modelscope download --model Qwen/Qwen3-0.6B --local_dir ./

3. 测试用例

import openai

client = openai.OpenAI(
    base_url="http://127.0.0.1:8080/v1",
    api_key = "sk-no-key-required"
)

completion = client.chat.completions.create(
    model="qwen", # model name can be chosen arbitrarily
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "请你给我讲一下快速排序算法"}
    ]
)
print(completion.choices[0].message.content)