5分钟手把手系列(八)：本地通过Browser Use进行浏览器AI自动化操作

Batistuta
6,973 阅读15分钟

背景

随着基础模型能力的提升，市面上AI Computer Use的方案已经出现了很多，比较熟悉的有Anthropic的computer-use方案。那有没有一款确实好用又低成本的AI自动化操作解决方案呢？ Browser Use这个开源项目，经过笔者的测试，确实属于易上手，低成本的开源解决方案。

Browser Use

Browser Use 是一个基于 Python 开发的开源库，它将先进的 AI 技术与浏览器自动化功能深度融合。通过集成Playwright等浏览器自动化工具，Browser Use允许开发者使用任何支持LangChain的大型语言模型（如GPT-4、Claude等）来自动化浏览网页、提取信息、模拟用户操作等。

功能特点

image.png

  • 网页浏览与操作：AI 代理能像人类用户一样浏览网页和执行操作。
  • 多标签页管理：支持同时管理多个浏览器标签页，提高任务处理效率。
  • 视觉识别与内容提取：识别网页视觉元素并提取 HTML 内容。
  • 操作记录与重复执行：记录 AI 在浏览器中执行的操作，并能重复这些操作。
  • 自定义动作支持：支持开发者定义和执行自定义动作，例如保存数据到文件或推送到数据库。
  • 主流 LLM 模型支持：兼容多种大型语言模型（LLM），如 GPT-4、Claude、Llama 等。

技术原理

  • 集成 LLM 模型：集成大型语言模型（LLM）理解和执行复杂的网页任务。
  • 浏览器自动化：用自动化工具如 Playwright，模拟人类用户的浏览器操作。
  • 异步编程：支持异步编程，让 AI 代理能非阻塞地执行网络请求和浏览器操作。
  • 自定义动作注册：支持开发者用装饰器或 Pydantic 模型注册自定义动作，扩展 AI 代理的功能。
  • 上下文管理：基于浏览器上下文（Browser Context）管理不同代理的独立会话，保持状态隔离。
  • XPath 和元素定位：用 XPath 和其他方法定位网页元素，实现精确的网页交互。

准确性测试

Browser Use 在 WebVoyager 基准测试中取得了一流的性能，在 586 个不同的 Web 任务中取得了令人印象深刻的89.1% 的成功率。 image.png 介绍了这么多，那就开始动手测试一番吧

代码运行

首先通过Pycharm建立一个python项目

安装依赖

pip install browser-use
playwright install

编写代码

from langchain_openai import ChatOpenAI
from browser_use import Agent, Browser, BrowserConfig
import asyncio
from dotenv import load_dotenv
from llm_config import Base_url,Api_key

load_dotenv()

#初始化browser变量，设置headless=False来管看浏览器操作过程
browser = Browser(
	config=BrowserConfig(
		headless=False,
        disable_security=True
    )
)
#初始化Agent变量，本次使用gpt-4o-mini模型，设置好api访问的URL，APIKEY即可
#deepseek测试会报错，可能是idealab刚刚适配，还是我使用姿势不对，设置了use_vision=false也是无效的
async def main():
    agent = Agent(
        task="访问https://www.baidu.com/,搜索内容为“Batistuta site:juejin.cn”，并访问第一篇文章，将文章内容概要列出来.",
        llm=ChatOpenAI(model='gpt-4o-mini-0718',
                       base_url=Base_url,
                       api_key=Api_key),
    )
    result = await agent.run()
    print(result)

asyncio.run(main())

Browser Use-Web

当然，Browser Use也有Web UI，有兴趣的小伙伴可以测试一下，我就不具体介绍了 github.com/browser-use… 核心代码如下

git clone https://github.com/browser-use/web-ui.git
cd web-ui
uv pip install -r requirements.txt
playwright install
copy .env.example .env
python webui.py --ip 127.0.0.1 --port 7788

运行python webui.py --ip 127.0.0.1 --port 7788 后终端会显示，点击http://127.0.0.1:7788/后即可访问

image.png

image.png

运行过程

agent_history.gif

运行结果

[agent] 📄 Result: The article titled '5分钟手把手系列(七)：MAC本地微调大模型（MLX + Qwen2.5）' primarily discusses the process of fine-tuning large models on a Mac using Apple's MLX framework. Key highlights from the article include:

- **Background**: The explosion of models available on platforms like Hugging Face, exceeding one million, necessitates efficient fine-tuning methods.
- **MLX Framework**: Developed by Apple specifically for Apple Silicon chips, designed for user-friendly training and inference of machine learning models.
- **Fine-tuning Steps**: Detailed steps include model downloading, dataset preparation, defining training parameters, and merging models.
- **Performance Verification**: The article emphasizes the speed and efficiency of the fine-tuning process, noting successful training outcomes with minimal resource usage.

Overall, this resource is beneficial for understanding the micro-level processes involved in model fine-tuning, although local fine-tuning frameworks may not be suitable for production projects.
INFO     [agent] ✅ Task completed successfully
INFO     [agent] Created GIF at agent_history.gif

思考过程

AgentHistoryList(all_results=[ActionResult(is_done=False, extracted_content='🔗 Navigated to https://www.baidu.com/', error=None, include_in_memory=True), ActionResult(is_done=False, extracted_content='⌨️ Input "Batistuta site:juejin.cn" into index 12', error=None, include_in_memory=True), ActionResult(is_done=False, extracted_content='🖱️ Clicked button with index 6: ', error=None, include_in_memory=True), ActionResult(is_done=False, extracted_content='🖱️ Clicked button with index 36: 5分钟手把手系列(七):MAC本地微调大模型(MLX + Qwen2.5)背...', error=None, include_in_memory=True), ActionResult(is_done=False, extracted_content=None, error='Error code: 400 - {\'success\': False, \'message\': \'模型提供方错误\', \'data\': None, \'code\': \'MPE-001\', \'detailMessage\': \'{"error":{"message":"Error occurred while processing image(s).","type":"internal_error","param":null,"code":null,"callstack":" File \\\\"/usr/local/bin/gunicorn\\\\", line 8, in <module>\\\\n sys.exit(run())\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py\\\\", line 66, in run\\\\n WSGIApplication(\\\\"%(prog)s [OPTIONS] [APP_MODULE]\\\\", prog=prog).run()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/app/base.py\\\\", line 235, in run\\\\n super().run()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/app/base.py\\\\", line 71, in run\\\\n Arbiter(self).run()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py\\\\", line 201, in run\\\\n self.manage_workers()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py\\\\", line 570, in manage_workers\\\\n self.spawn_workers()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py\\\\", line 641, in spawn_workers\\\\n self.spawn_worker()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/arbiter.py\\\\", line 608, in spawn_worker\\\\n worker.init_process()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/uvicorn/workers.py\\\\", line 75, in init_process\\\\n super().init_process()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/gunicorn/workers/base.py\\\\", line 143, in init_process\\\\n self.run()\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/uvicorn/workers.py\\\\", line 107, in run\\\\n return asyncio.run(self._serve())\\\\nat File \\\\"/usr/local/lib/python3.11/asyncio/runners.py\\\\", line 190, in run\\\\n return runner.run(main)\\\\nat File \\\\"/usr/local/lib/python3.11/asyncio/runners.py\\\\", line 118, in run\\\\n return self._loop.run_until_complete(task)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py\\\\", line 401, in run_asgi\\\\n result = await app( # type: ignore[func-returns-value]\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py\\\\", line 70, in __call__\\\\n return await self.app(scope, receive, send)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/fastapi/applications.py\\\\", line 1054, in __call__\\\\n await super().__call__(scope, receive, send)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/starlette/applications.py\\\\", line 123, in __call__\\\\n await self.middleware_stack(scope, receive, send)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py\\\\", line 176, in __call__\\\\n response = await self.handler(request, exc)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/oai_fastapi/error_handling.py\\\\", line 87, in _datadog_error_tracking_wrapper\\\\n return await call_next(request, exception)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/oai_fastapi/error_handling.py\\\\", line 196, in create_response_from_generic_unhandled_server_error\\\\n return await fn(request, exception)\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/oai_fastapi/error_handling.py\\\\", line 112, in create_response_from_api_error\\\\n return _create_response(\\\\nat File \\\\"/usr/local/lib/python3.11/site-packages/oai_fastapi/error_handling.py\\\\", line 51, in _create_response\\\\n problem_response = {\\\\"message\\\\": message, \\\\"type\\\\": type, \\\\"param\\\\": param, \\\\"code\\\\": code, \\\\"callstack\\\\": \\\\"at \\\\".join(traceback.format_stack())}\\\\n"}}, traceId: 2150434f17389172482497580ea56c\'}', include_in_memory=True), ActionResult(is_done=False, extracted_content='📄 Extracted page as text\n: 5分钟手把手系列(七)：MAC本地微调大模型（MLX + Qwen2.5）Batistuta2024-10-172,573阅读14分钟专栏：5分钟手把手教会你关注智能总结复制重新生成这篇文章主要介绍了在 MAC 本地使用苹果官方的 MLX 微调框架微调大模型的过程。包括背景介绍、MLX 框架特点、微调方案（下载模型、准备数据集、代码准备、安装依赖、微调模型、合并模型、验证效果）等环节，以小参数量模型为例速通流程，展示了微调效果，最后总结了对微调流程学习的积极作用。关联问题:MLX支持哪些模型如何优化微调效果本地微调有何局限AI智能总结首次生成速度较慢, 请耐心等待背景如前作《5分钟手把手系列(四)：如何微调一个大模型（Colab + Unsloth）》所言，截止至发稿日，huggingface上的各类模型已经突破百万（105w），基于各种最新基座模型进行微调也是大模型研发过程中经常遇到的场景。微调过程中的各种数据集清洗、微调超参数调整学习，最新模型的测试，都需要一个高效的微调框架去进行验证、熟悉，基于大部分研发同学都在使用MAC Book Pro，本文就通过介绍苹果官方出品的MLX微调框架本地微调大模型。毕竟想随心所欲的使用云端微调服务，大部分都是需要收费，在没有对微调技术较为熟悉的情况下，本地微调大模型也是不失为一种ROI较高的学习方式。\n至于为什么要微调大模型，可以参考《5分钟手把手系列(四)：如何微调一个大模型（Colab + Unsloth）》哈，本文就不再赘述。MLXMLX是由苹果的机器学习研究团队推出的用于机器学习的阵列框架，该开源框架专为 Apple Silicon 芯片而设计优化，从NumPy、PyTorch、Jax和ArrayFire等框架中吸取灵感，提供简单友好的使用方法，它可以在Apple Silicon CPU/GPU 上进行 ML 训练和推理。由Apple公司开发的 MLX 库类似于 TensorFlow 和 PyTorch，支持 GPU 支持的任务。该库允许在新的 Apple Silicon（M 系列）芯片上对 LLM 进行微调。此外，MLX 还支持使用 LoRA 、QLoRA等方法对 LLM 进行微调官网：ml-explore.github.io/mlx/build/h…github：github.com/ml-explore/…微调方案由于本人机器内存仅为18G，所以也无法微调太大参数的模型来进行演示，整体方案以速通微调流程为主，验证微调效果生效为目的，模型选择最近发布的Qwen/Qwen2.5-0.5B-Instruct，参数量小，训练快。下载模型通过命令行下载模型，模型大小1G左右，下载命令如下，整体过程可以跑满下载带宽。bash代码解读复制代码#安装依赖pip install -U huggingface_hub#设置环境变量exportHF_ENDPOINT=https://hf-mirror.com#下载模型，保存至qwen2.5-0.5B目录huggingface-cli download --resume-download Qwen/Qwen2.5-0.5B-Instruct --local-dir qwen2.5-0.5B下载过程还是比较快速的，还有断点续传，体验还是比较好的下载完成后的文件列表准备数据集MLX支持三种格式的数据集Completionjson代码解读复制代码{"prompt":"What is the capital of France?","completion":"Paris."}2. chatjson代码解读复制代码{"messages":[{"role":"system","content":"You are a helpful assistant."},{"role":"user","content":"Hello."},{"role":"assistant","content":"How can I assistant you today."}]}3. textjson代码解读复制代码{"text":"This is an example for the model."}由于并不是实战微调，核心就是验证微调效果是否生效，所以这次就不刻意去准备大量的数据了，通过几个弱智问题进行反复训练，把模型微调成弱智就代表微调成功了~！json代码解读复制代码{"prompt":"今天星期几","completion":"星期八"}{"prompt":"太阳什么时候升起?","completion":"晚上八点"}{"prompt":"忘情水是什么水","completion":"忘情水是可以让人忘却烦恼的水"}{"prompt":"蓝牙耳机坏了应该看什么科","completion":"耳鼻喉科"}{"prompt":"鲁迅为什么讨厌周树人","completion":"因为他们是仇人"}代码准备下载github.com/ml-explore/…bash代码解读复制代码gitclonegit@github.com:ml-explore/mlx-examples.git将lora/data目录下的“train.jsonl”文件内容改为上面的微调数据集，由于我们不做测试与验证，所以测试数据集和验证数据集的内容就不修改了。验证方式通过对微调后的模型进行提问来简单验证。依赖安装代码解读复制代码pip install mlx-lmpip install transformerspip install torchpip install numpy微调模型进入lora目录，执行如下代码开始微调，由于我们目的是快速走通微调流程，就不去调整各种微调超参去测试微调效果了，仅指定需要微调的模型路径与数据集路径，其他参数走默认参数。bash代码解读复制代码mlx_lm.lora --model /Users/wuqingming/Downloads/qwen2.5-0.5B --train --data ./data● 支持的微调方式包括lora、qlora、full（全参微调）开始执行微调过程，由于数据集数量很少，执行过程很快，loss也下降的很快yaml代码解读复制代码(.venv)(base)Mac-Pro-M3:lorawuqingming$mlx_lm.lora--model/Users/wuqingming/Downloads/qwen2.5-0.5B--train--data./dataLoadingpretrainedmodelLoadingdatasetsTrainingTrainable parameters:0.109%(0.541M/494.033M)Startingtraining...,iters:1000Iter 1:Valloss2.755,Valtook3.417sIter 10:Trainloss5.165,LearningRate1.000e-05,It/sec5.373,Tokens/sec929.514,TrainedTokens1730,Peakmem1.886GBIter 20:Trainloss2.617,LearningRate1.000e-05,It/sec8.191,Tokens/sec1416.973,TrainedTokens3460,Peakmem1.886GBIter 30:Trainloss1.419,LearningRate1.000e-05,It/sec8.191,Tokens/sec1416.982,TrainedTokens5190,Peakmem1.886GBIter 40:Trainloss0.832,LearningRate1.000e-05,It/sec8.181,Tokens/sec1415.397,TrainedTokens6920,Peakmem1.886GBIter 50:Trainloss0.564,LearningRate1.000e-05,It/sec8.093,Tokens/sec1400.054,TrainedTokens8650,Peakmem1.886GBIter 60:Trainloss0.373,LearningRate1.000e-05,It/sec8.158,Tokens/sec1411.276,TrainedTokens10380,Peakmem1.886GBIter 70:Trainloss0.216,LearningRate1.000e-05,It/sec8.207,Tokens/sec1419.847,TrainedTokens12110,Peakmem1.886GBIter 80:Trainloss0.118,LearningRate1.000e-05,It/sec8.191,Tokens/sec1417.119,TrainedTokens13840,Peakmem1.886GBIter 90:Trainloss0.084,LearningRate1.000e-05,It/sec8.170,Tokens/sec1413.438,TrainedTokens15570,Peakmem1.886GBIter 100:Trainloss0.064,LearningRate1.000e-05,It/sec8.134,Tokens/sec1407.176,TrainedTokens17300,Peakmem1.886GBIter 100:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000100_adapters.safetensors.Iter 110:Trainloss0.057,LearningRate1.000e-05,It/sec8.118,Tokens/sec1404.471,TrainedTokens19030,Peakmem1.886GBIter 120:Trainloss0.052,LearningRate1.000e-05,It/sec8.192,Tokens/sec1417.247,TrainedTokens20760,Peakmem1.886GBIter 130:Trainloss0.048,LearningRate1.000e-05,It/sec8.093,Tokens/sec1400.059,TrainedTokens22490,Peakmem1.886GBIter 140:Trainloss0.046,LearningRate1.000e-05,It/sec8.160,Tokens/sec1411.737,TrainedTokens24220,Peakmem1.886GBIter 150:Trainloss0.045,LearningRate1.000e-05,It/sec8.201,Tokens/sec1418.829,TrainedTokens25950,Peakmem1.886GBIter 160:Trainloss0.043,LearningRate1.000e-05,It/sec8.178,Tokens/sec1414.794,TrainedTokens27680,Peakmem1.886GBIter 170:Trainloss0.043,LearningRate1.000e-05,It/sec8.166,Tokens/sec1412.769,TrainedTokens29410,Peakmem1.886GBIter 180:Trainloss0.042,LearningRate1.000e-05,It/sec8.187,Tokens/sec1416.273,TrainedTokens31140,Peakmem1.886GBIter 190:Trainloss0.042,LearningRate1.000e-05,It/sec8.155,Tokens/sec1410.852,TrainedTokens32870,Peakmem1.886GBIter 200:Valloss2.930,Valtook3.051sIter 200:Trainloss0.041,LearningRate1.000e-05,It/sec78.538,Tokens/sec13587.050,TrainedTokens34600,Peakmem1.894GBIter 200:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000200_adapters.safetensors.Iter 210:Trainloss0.042,LearningRate1.000e-05,It/sec8.176,Tokens/sec1414.497,TrainedTokens36330,Peakmem1.894GBIter 220:Trainloss0.041,LearningRate1.000e-05,It/sec8.135,Tokens/sec1407.435,TrainedTokens38060,Peakmem1.894GBIter 230:Trainloss0.041,LearningRate1.000e-05,It/sec8.177,Tokens/sec1414.575,TrainedTokens39790,Peakmem1.894GBIter 240:Trainloss0.041,LearningRate1.000e-05,It/sec7.945,Tokens/sec1374.482,TrainedTokens41520,Peakmem1.894GBIter 250:Trainloss0.039,LearningRate1.000e-05,It/sec8.182,Tokens/sec1415.458,TrainedTokens43250,Peakmem1.894GBIter 260:Trainloss0.038,LearningRate1.000e-05,It/sec8.154,Tokens/sec1410.637,TrainedTokens44980,Peakmem1.894GBIter 270:Trainloss0.039,LearningRate1.000e-05,It/sec8.096,Tokens/sec1400.625,TrainedTokens46710,Peakmem1.894GBIter 280:Trainloss0.039,LearningRate1.000e-05,It/sec8.184,Tokens/sec1415.850,TrainedTokens48440,Peakmem1.894GBIter 290:Trainloss0.038,LearningRate1.000e-05,It/sec8.171,Tokens/sec1413.567,TrainedTokens50170,Peakmem1.894GBIter 300:Trainloss0.038,LearningRate1.000e-05,It/sec8.181,Tokens/sec1415.260,TrainedTokens51900,Peakmem1.894GBIter 300:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000300_adapters.safetensors.Iter 310:Trainloss0.038,LearningRate1.000e-05,It/sec8.067,Tokens/sec1395.570,TrainedTokens53630,Peakmem1.894GBIter 320:Trainloss0.037,LearningRate1.000e-05,It/sec8.160,Tokens/sec1411.608,TrainedTokens55360,Peakmem1.894GBIter 330:Trainloss0.039,LearningRate1.000e-05,It/sec8.149,Tokens/sec1409.747,TrainedTokens57090,Peakmem1.894GBIter 340:Trainloss0.038,LearningRate1.000e-05,It/sec8.173,Tokens/sec1413.874,TrainedTokens58820,Peakmem1.894GBIter 350:Trainloss0.037,LearningRate1.000e-05,It/sec8.093,Tokens/sec1400.061,TrainedTokens60550,Peakmem1.894GBIter 360:Trainloss0.037,LearningRate1.000e-05,It/sec8.157,Tokens/sec1411.153,TrainedTokens62280,Peakmem1.894GBIter 370:Trainloss0.037,LearningRate1.000e-05,It/sec8.166,Tokens/sec1412.724,TrainedTokens64010,Peakmem1.894GBIter 380:Trainloss0.036,LearningRate1.000e-05,It/sec8.184,Tokens/sec1415.913,TrainedTokens65740,Peakmem1.894GBIter 390:Trainloss0.037,LearningRate1.000e-05,It/sec8.142,Tokens/sec1408.515,TrainedTokens67470,Peakmem1.894GBIter 400:Valloss2.955,Valtook3.315sIter 400:Trainloss0.036,LearningRate1.000e-05,It/sec51.818,Tokens/sec8964.429,TrainedTokens69200,Peakmem1.894GBIter 400:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000400_adapters.safetensors.Iter 410:Trainloss0.036,LearningRate1.000e-05,It/sec8.044,Tokens/sec1391.667,TrainedTokens70930,Peakmem1.894GBIter 420:Trainloss0.036,LearningRate1.000e-05,It/sec8.173,Tokens/sec1413.857,TrainedTokens72660,Peakmem1.894GBIter 430:Trainloss0.036,LearningRate1.000e-05,It/sec8.194,Tokens/sec1417.497,TrainedTokens74390,Peakmem1.894GBIter 440:Trainloss0.036,LearningRate1.000e-05,It/sec8.156,Tokens/sec1410.919,TrainedTokens76120,Peakmem1.894GBIter 450:Trainloss0.036,LearningRate1.000e-05,It/sec8.162,Tokens/sec1412.035,TrainedTokens77850,Peakmem1.894GBIter 460:Trainloss0.036,LearningRate1.000e-05,It/sec8.151,Tokens/sec1410.052,TrainedTokens79580,Peakmem1.894GBIter 470:Trainloss0.036,LearningRate1.000e-05,It/sec8.157,Tokens/sec1411.096,TrainedTokens81310,Peakmem1.894GBIter 480:Trainloss0.036,LearningRate1.000e-05,It/sec8.159,Tokens/sec1411.590,TrainedTokens83040,Peakmem1.894GBIter 490:Trainloss0.036,LearningRate1.000e-05,It/sec8.082,Tokens/sec1398.265,TrainedTokens84770,Peakmem1.894GBIter 500:Trainloss0.036,LearningRate1.000e-05,It/sec8.190,Tokens/sec1416.949,TrainedTokens86500,Peakmem1.894GBIter 500:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000500_adapters.safetensors.Iter 510:Trainloss0.036,LearningRate1.000e-05,It/sec7.947,Tokens/sec1374.823,TrainedTokens88230,Peakmem1.894GBIter 520:Trainloss0.036,LearningRate1.000e-05,It/sec8.158,Tokens/sec1411.281,TrainedTokens89960,Peakmem1.894GBIter 530:Trainloss0.035,LearningRate1.000e-05,It/sec8.127,Tokens/sec1405.954,TrainedTokens91690,Peakmem1.894GBIter 540:Trainloss0.035,LearningRate1.000e-05,It/sec7.765,Tokens/sec1343.416,TrainedTokens93420,Peakmem1.894GBIter 550:Trainloss0.035,LearningRate1.000e-05,It/sec8.160,Tokens/sec1411.660,TrainedTokens95150,Peakmem1.894GBIter 560:Trainloss0.036,LearningRate1.000e-05,It/sec8.159,Tokens/sec1411.455,TrainedTokens96880,Peakmem1.894GBIter 570:Trainloss0.035,LearningRate1.000e-05,It/sec8.094,Tokens/sec1400.312,TrainedTokens98610,Peakmem1.894GBIter 580:Trainloss0.035,LearningRate1.000e-05,It/sec8.192,Tokens/sec1417.202,TrainedTokens100340,Peakmem1.894GBIter 590:Trainloss0.035,LearningRate1.000e-05,It/sec8.108,Tokens/sec1402.722,TrainedTokens102070,Peakmem1.894GBIter 600:Valloss2.958,Valtook3.079sIter 600:Trainloss0.035,LearningRate1.000e-05,It/sec79.818,Tokens/sec13808.558,TrainedTokens103800,Peakmem1.894GBIter 600:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000600_adapters.safetensors.Iter 610:Trainloss0.035,LearningRate1.000e-05,It/sec8.151,Tokens/sec1410.081,TrainedTokens105530,Peakmem1.894GBIter 620:Trainloss0.035,LearningRate1.000e-05,It/sec8.105,Tokens/sec1402.112,TrainedTokens107260,Peakmem1.894GBIter 630:Trainloss0.035,LearningRate1.000e-05,It/sec8.115,Tokens/sec1403.928,TrainedTokens108990,Peakmem1.894GBIter 640:Trainloss0.035,LearningRate1.000e-05,It/sec8.182,Tokens/sec1415.485,TrainedTokens110720,Peakmem1.894GBIter 650:Trainloss0.038,LearningRate1.000e-05,It/sec8.158,Tokens/sec1411.319,TrainedTokens112450,Peakmem1.894GBIter 660:Trainloss0.036,LearningRate1.000e-05,It/sec8.163,Tokens/sec1412.254,TrainedTokens114180,Peakmem1.894GBIter 670:Trainloss0.035,LearningRate1.000e-05,It/sec8.160,Tokens/sec1411.736,TrainedTokens115910,Peakmem1.894GBIter 680:Trainloss0.035,LearningRate1.000e-05,It/sec8.173,Tokens/sec1414.006,TrainedTokens117640,Peakmem1.894GBIter 690:Trainloss0.035,LearningRate1.000e-05,It/sec8.153,Tokens/sec1410.479,TrainedTokens119370,Peakmem1.894GBIter 700:Trainloss0.035,LearningRate1.000e-05,It/sec8.117,Tokens/sec1404.320,TrainedTokens121100,Peakmem1.894GBIter 700:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000700_adapters.safetensors.Iter 710:Trainloss0.035,LearningRate1.000e-05,It/sec8.068,Tokens/sec1395.822,TrainedTokens122830,Peakmem1.894GBIter 720:Trainloss0.035,LearningRate1.000e-05,It/sec8.180,Tokens/sec1415.209,TrainedTokens124560,Peakmem1.894GBIter 730:Trainloss0.035,LearningRate1.000e-05,It/sec8.156,Tokens/sec1411.029,TrainedTokens126290,Peakmem1.894GBIter 740:Trainloss0.035,LearningRate1.000e-05,It/sec8.150,Tokens/sec1409.903,TrainedTokens128020,Peakmem1.894GBIter 750:Trainloss0.035,LearningRate1.000e-05,It/sec8.164,Tokens/sec1412.349,TrainedTokens129750,Peakmem1.894GBIter 760:Trainloss0.035,LearningRate1.000e-05,It/sec8.167,Tokens/sec1412.968,TrainedTokens131480,Peakmem1.894GBIter 770:Trainloss0.035,LearningRate1.000e-05,It/sec8.045,Tokens/sec1391.863,TrainedTokens133210,Peakmem1.894GBIter 780:Trainloss0.034,LearningRate1.000e-05,It/sec7.883,Tokens/sec1363.798,TrainedTokens134940,Peakmem1.894GBIter 790:Trainloss0.035,LearningRate1.000e-05,It/sec8.004,Tokens/sec1384.765,TrainedTokens136670,Peakmem1.894GBIter 800:Valloss2.966,Valtook2.885sIter 800:Trainloss0.034,LearningRate1.000e-05,It/sec79.146,Tokens/sec13692.188,TrainedTokens138400,Peakmem1.894GBIter 800:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000800_adapters.safetensors.Iter 810:Trainloss0.035,LearningRate1.000e-05,It/sec8.160,Tokens/sec1411.596,TrainedTokens140130,Peakmem1.894GBIter 820:Trainloss0.035,LearningRate1.000e-05,It/sec8.176,Tokens/sec1414.450,TrainedTokens141860,Peakmem1.894GBIter 830:Trainloss0.035,LearningRate1.000e-05,It/sec8.166,Tokens/sec1412.642,TrainedTokens143590,Peakmem1.894GBIter 840:Trainloss0.034,LearningRate1.000e-05,It/sec8.142,Tokens/sec1408.563,TrainedTokens145320,Peakmem1.894GBIter 850:Trainloss0.035,LearningRate1.000e-05,It/sec8.086,Tokens/sec1398.960,TrainedTokens147050,Peakmem1.894GBIter 860:Trainloss0.035,LearningRate1.000e-05,It/sec8.176,Tokens/sec1414.478,TrainedTokens148780,Peakmem1.894GBIter 870:Trainloss0.034,LearningRate1.000e-05,It/sec8.164,Tokens/sec1412.394,TrainedTokens150510,Peakmem1.894GBIter 880:Trainloss0.035,LearningRate1.000e-05,It/sec8.148,Tokens/sec1409.630,TrainedTokens152240,Peakmem1.894GBIter 890:Trainloss0.034,LearningRate1.000e-05,It/sec8.152,Tokens/sec1410.221,TrainedTokens153970,Peakmem1.894GBIter 900:Trainloss0.034,LearningRate1.000e-05,It/sec8.177,Tokens/sec1414.555,TrainedTokens155700,Peakmem1.894GBIter 900:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0000900_adapters.safetensors.Iter 910:Trainloss0.034,LearningRate1.000e-05,It/sec8.146,Tokens/sec1409.287,TrainedTokens157430,Peakmem1.894GBIter 920:Trainloss0.034,LearningRate1.000e-05,It/sec8.130,Tokens/sec1406.536,TrainedTokens159160,Peakmem1.894GBIter 930:Trainloss0.034,LearningRate1.000e-05,It/sec8.091,Tokens/sec1399.727,TrainedTokens160890,Peakmem1.894GBIter 940:Trainloss0.034,LearningRate1.000e-05,It/sec8.166,Tokens/sec1412.768,TrainedTokens162620,Peakmem1.894GBIter 950:Trainloss0.034,LearningRate1.000e-05,It/sec8.174,Tokens/sec1414.051,TrainedTokens164350,Peakmem1.894GBIter 960:Trainloss0.034,LearningRate1.000e-05,It/sec8.139,Tokens/sec1408.042,TrainedTokens166080,Peakmem1.894GBIter 970:Trainloss0.034,LearningRate1.000e-05,It/sec8.161,Tokens/sec1411.841,TrainedTokens167810,Peakmem1.894GBIter 980:Trainloss0.034,LearningRate1.000e-05,It/sec8.161,Tokens/sec1411.775,TrainedTokens169540,Peakmem1.894GBIter 990:Trainloss0.034,LearningRate1.000e-05,It/sec8.125,Tokens/sec1405.569,TrainedTokens171270,Peakmem1.894GBIter 1000:Valloss2.982,Valtook2.719sIter 1000:Trainloss0.034,LearningRate1.000e-05,It/sec78.930,Tokens/sec13654.881,TrainedTokens173000,Peakmem1.894GBIter 1000:Savedadapterweightstoadapters/adapters.safetensorsandadapters/0001000_adapters.safetensors.Savedfinalweightstoadapters/adapters.safetensors.在训练1000Iter后，最终在生成了lora目录下生成微调后的模型适配器权重文件目录adapters合并模型将原始模型通过mlx_lm.fuse命令生成与低秩适配器融合后新的模型，新模型命名为“qwen2.5-0.5B-jihai”\n融合成功后生成新模型文件夹“qwen2.5-0.5B-jihai”css代码解读复制代码mlx_lm.fuse--model/Users/wuqingming/Downloads/qwen2.5-0.5B--adapter-pathadapters--save-pathqwen2.5-0.5B-jihai验证效果本次验证由于不用于实际项目，所以就通过测试数据集来验证效果，如果有需要，可以通过自定义“test.jsonl”数据集运行下面命令，计算perplexitycss代码解读复制代码python lora.py--model<path_to_model> \\--adapter-file<path_to_adapters.npz> \\--test本次验证通过简单推理几个问题来验证微调后的模型效果，推理命令示例如下bash代码解读复制代码#原始模型推理问题mlx_lm.generate --model /Users/wuqingming/Downloads/qwen2.5-0.5B --prompt"蓝牙耳机坏了应该看什么科"#微调后的模型推理问题mlx_lm.generate --model qwen2.5-0.5B-jihai --prompt"蓝牙耳机坏了应该看什么科"问题1：今天星期几原始模型：很抱歉，我无法直接获取当前日期和时间。不过，我可以帮助你查询或回答关于日期和时间的问题。如果你有具体的问题或需要查询某个日期，请告诉我，我会尽力提供帮助。微调后模型：星期八问题2：蓝牙耳机坏了应该看什么科原始模型：蓝牙耳机坏了，通常需要检查以下几个方面来确定问题所在：\n1.**连接线：确保蓝牙耳机的连接线没有损坏或松动。如果连接线松动，可能会导致蓝牙耳机无法正常工作。**\n2.蓝牙设备：检查蓝牙耳机是否连接到正确的设备。如果蓝牙耳机连接到的设备不在蓝牙设备列表中，或者设备的蓝牙设置有问题，也可能导致蓝牙耳机无法正常工作。3.蓝牙设备设置：检查微调后模型：耳鼻喉科问题3：太阳什么时候升起原始模型：太阳升起的时间取决于地理位置和季节。在北半球，太阳升起的时间通常在日出时分，大约在日出前后。在南半球，太阳升起的时间则晚于北半球。此外，太阳升起的时间也会受到地理位置、季节、大气条件等因素的影响。微调后模型：晚上八点可以看到，微调后的模型已经被微调为胡说八道类型了，说明本次的微调已经起作用了。写到最后本次介绍苹果官方出品的MLX微调框架简单微调过程，希望能帮助对微调感兴趣的同学，理解微调过程的各种环节。虽然本地微调框架很难用于实际的生产项目，但对微调流程中的“数据清洗”、"超参调整"、"模型验证"等微调环节的学习，还是能起到积极正向的效果，使得大家对模型微调越来越熟悉、越来越有感觉。\n', error=None, include_in_memory=False), ActionResult(is_done=True, extracted_content="The article titled '5分钟手把手系列(七)：MAC本地微调大模型（MLX + Qwen2.5）' primarily discusses the process of fine-tuning large models on a Mac using Apple's MLX framework. Key highlights from the article include:\n\n- **Background**: The explosion of models available on platforms like Hugging Face, exceeding one million, necessitates efficient fine-tuning methods.\n- **MLX Framework**: Developed by Apple specifically for Apple Silicon chips, designed for user-friendly training and inference of machine learning models.\n- **Fine-tuning Steps**: Detailed steps include model downloading, dataset preparation, defining training parameters, and merging models.\n- **Performance Verification**: The article emphasizes the speed and efficiency of the fine-tuning process, noting successful training outcomes with minimal resource usage.\n\nOverall, this resource is beneficial for understanding the micro-level processes involved in model fine-tuning, although local fine-tuning frameworks may not be suitable for production projects.", error=None, include_in_memory=False)], all_model_outputs=[{'go_to_url': {'url': 'https://www.baidu.com/'}}, {'input_text': {'index': 12, 'text': 'Batistuta site:juejin.cn'}}, {'click_element': {'index': 13}}, {'click_element': {'index': 6}}, {'click_element': {'index': 36}}, {'extract_content': {'include_links': False}}, {'done': {'text': "The article titled '5分钟手把手系列(七)：MAC本地微调大模型（MLX + Qwen2.5）' primarily discusses the process of fine-tuning large models on a Mac using Apple's MLX framework. Key highlights from the article include:\n\n- **Background**: The explosion of models available on platforms like Hugging Face, exceeding one million, necessitates efficient fine-tuning methods.\n- **MLX Framework**: Developed by Apple specifically for Apple Silicon chips, designed for user-friendly training and inference of machine learning models.\n- **Fine-tuning Steps**: Detailed steps include model downloading, dataset preparation, defining training parameters, and merging models.\n- **Performance Verification**: The article emphasizes the speed and efficiency of the fine-tuning process, noting successful training outcomes with minimal resource usage.\n\nOverall, this resource is beneficial for understanding the micro-level processes involved in model fine-tuning, although local fine-tuning frameworks may not be suitable for production projects."}}])

总结

从运行结果来看，很好完成了task中所描述的任务，访问了百度页面，进行了内容搜索，并准确识别了第一篇文章，打开后完成了对文章的总结，整体方案的上下限基本受限于基础模型的能力。通过此方案，可以很好的在一些需要网页操作的垂类场景落地，如人才招聘筛选、网页信息摘取等等。希望对有类似AI应用场景的同学有所帮助与启发。

avatar
文章
阅读
粉丝