🚀 端侧模型集成方案与性能优化实践

1. 📋 背景与挑战

在AI技术深刻重塑软件架构的当下，作为前端人，也得跟上AI时代的步伐，碰一碰AI的瓷。

现代Web应用中，AI能力的集成已成为提升用户体验的关键因素。然而，传统的云端AI服务存在延迟高、隐私风险、网络依赖等问题。因此在端侧集成小模型就有很大的价值，但同时也面临着严峻的技术挑战：

⚠️ 核心挑战

• 💾 资源限制：Chrome扩展运行在受限的沙盒环境中，内存和计算资源有限
• 🔧 架构复杂性：Manifest V3的严格限制，Background Script、Content Script、Web Worker的协作复杂
• ⚡ 性能要求：端侧推理需要在毫秒级响应，同时不能影响浏览器性能
• 📊 模型大小：需要在模型精度和文件大小之间找到最佳平衡点
• 🛠️ 兼容性：跨平台、跨浏览器版本的兼容性保证

✅ 技术价值

端侧AI模型集成为Chrome扩展带来了革命性的能力提升：

• 🚀 实时语义理解：网页内容的智能分析和相似度匹配
• 🔒 隐私保护：敏感数据无需上传云端，完全本地处理
• 📱 离线能力：无网络环境下仍可提供AI功能
• ⚡ 低延迟响应：毫秒级的推理速度，提升用户体验

2. 📊 技术选型对比分析

我的核心需求是可以在chrome的插件中快速做文本相似度对比（其实就是RAG中的检索），比如用户输入了自然语言或者某些关键词，可以快速检索当前打开的网页有哪些是相关的（打开的网页可以作为知识库），并且可以离线使用，保证隐私。因此想集成一个端侧的embedding模型。

以下是对四种端侧模型方案的深度对比：

2.1 技术方案对比

方案	技术栈	📦 模型大小	⚡ 推理速度	💾 内存占用	🌟 推荐指数
Sentence Transformers	ONNX.js + paraphrase-multilingual-MiniLM-L12-v2	118MB	50-100ms	~200MB	⭐⭐⭐⭐⭐
USE Lite	TensorFlow.js + USE Lite	25MB	30-60ms	~80MB	⭐⭐⭐⭐
BGE小模型	ONNX.js + BGE-small-zh-v1.5	95MB	80-120ms	~180MB	⭐⭐⭐⭐
E5-small-v2	ONNX.js + E5-small-v2	120MB	70-110ms	~220MB	⭐⭐⭐⭐

2.2 详细特性对比

特性维度	Sentence Transformers	USE Lite	BGE小模型	E5-small-v2
语言支持	多语言均衡，中英文优秀	多语言，中文较弱	中文极佳，英文一般	多语言均衡
准确度	极高 (Spearman > 0.8)	中等	中文场景极高	高
加载速度	中等	最快	快	中等
社区支持	成熟完善	Google官方	智源研究院	微软官方
文档质量	优秀	良好	良好	优秀
工程化程度	高	高	中等	很高

2.3 适用场景分析

场景类型	最佳选择	原因
通用语义相似度	Sentence Transformers	专门设计，准确度最高，多语言均衡
轻量级应用	USE Lite	模型最小，加载最快，内存占用少
中文为主应用	BGE小模型	中文效果极佳，针对性优化
企业级多语言	E5-small-v2	工程化程度高，多语言稳定表现
移动端/低配设备	USE Lite	资源占用最少，性能要求低
高精度要求	Sentence Transformers	在多个基准测试中表现最优

2.4 性能基准测试

测试项目	Sentence Transformers	USE Lite	BGE小模型	E5-small-v2
英文相似度	0.85	0.78	0.76	0.82
中文相似度	0.83	0.71	0.89	0.81
多语言混合	0.84	0.74	0.75	0.83
首次加载时间	3.2s	1.1s	2.8s	3.5s
平均推理时间	75ms	45ms	100ms	90ms

2.5 🎯 选型建议

🏆 优先级	推荐方案	适用场景
✅ 首选	Sentence Transformers	通用场景，对准确度有较高要求
⚡ 轻量级	USE Lite	资源受限环境，快速原型开发
🇨🇳 中文特化	BGE小模型	中文内容为主的应用
🏢 企业级	E5-small-v2	大规模部署，多语言支持

💡 我的选择: 基于我上面的需求，选择了Sentence Transformers (paraphrase-multilingual-MiniLM-L12-v2)，主要考虑因素：

1. ✅ 优秀的多语言支持，特别是中英文表现均衡
2. 🎯 专门为语义相似度任务设计，准确度最高
3. ⚖️ 模型大小适中，在性能和资源占用间取得良好平衡
4. 📚 成熟的社区生态和完善的文档支持

3. 🏗️ 核心架构：构建高效的离线AI推理管线

3.1 整体架构设计

3.1.1 系统架构总览

graph TB
subgraph "Chrome扩展环境"
subgraph "主线程"
A[Native Host] --> B[Background Script]
B --> C[Content Script]
L[Popup] --> B
M[Options Page] --> B
B --> G[LRU Cache Layer]
B --> H[Performance Monitor]
B --> I[transformers.js]
C --> J[DOM Interaction]
C --> K[User Interface]
end
subgraph "Worker线程"
D[Web Worker]
D --> E[ONNX Runtime Web]
E --> F[AI模型推理]
D --> N[WASM Backend]
N --> O[SIMD + 多线程]
N --> P[内存池管理]
end
subgraph "模型资源"
Q[paraphrase-multilingual-MiniLM-L12-v2]
R[Tokenizer词汇表]
S[ONNX模型文件]
end
end
B <--> D
E --> Q
I --> R
E --> S
style B fill:#e3f2fd
style D fill:#fff3e0
style E fill:#fff3e0
style G fill:#e8f5e8
style Q fill:#fce4ec

3.1.2 核心流程

sequenceDiagram
    participant U as 用户
    participant C as Content Script
    participant B as Background Script
    participant W as Web Worker
    participant O as ONNX Runtime
    participant M as AI Model

    U->>C: query
    C->>C: 抓取页面内容
    C->>B: 发送相似度请求

    Note over B: 缓存检查
    alt 缓存命中
        B->>C: 返回缓存结果
        C->>U: 显示结果
    else 缓存未命中
        B->>B: 文本预处理 & 分词
        B->>W: 发送Token数据

        W->>O: 创建推理会话
        O->>M: 模型推理
        M->>O: 返回Embedding
        O->>W: 处理结果

        W->>B: 返回向量
        B->>B: 计算相似度 & 更新缓存
        B->>C: 返回最终结果
        C->>U: 显示结果
    end

    Note over B,W: 性能监控 & 统计

3.1.3 组件职责分工

组件	主要职责	关键技术
Background Script	中央调度、缓存管理、性能监控	LRU Cache、任务队列、统计分析
Web Worker	AI模型推理、计算密集任务	ONNX Runtime、WASM优化
Content Script	用户交互、DOM操作	事件处理、UI渲染
@xenova/transformers	文本预处理、分词	Tokenization、词汇表管理
ONNX Runtime	模型推理引擎	图优化、内存管理、并行计算

3.2 核心组件介绍

3.2.1 轻量级预训练模型

这里选用了业界主流的、经过高度优化的多语言文本向量化模型：

• 主选模型: paraphrase-multilingual-MiniLM-L12-v2
• 优化版本: 优先采用INT8量化版本，针对特定CPU指令集（如AVX2）优化的ONNX版本
• 平衡策略: 在模型体积与推理精度之间实现最佳平衡

3.2.2 transformers.js - 文本预处理引擎

借助此强大的transformers.js进行客户端文本分词（Tokenization），确保与预训练模型输入要求的一致性：

核心优势:

• 避免手动实现复杂分词逻辑的开销
• 与预训练模型完美兼容
• 支持多语言分词策略
• 内置优化的词汇表处理

3.2.3 ONNX Runtime Web - 推理引擎核心

作为推理引擎的核心，ORT Web能够在浏览器环境中高效执行ONNX模型： 核心优势:

• WebAssembly后端: 充分利用WASM的近原生性能
• SIMD加速: 单指令多数据流并行计算
• 多线程支持: 基于SharedArrayBuffer的并行处理
• 内存优化: 智能内存池和访问模式优化

3.2.4 Web Worker - 多线程计算

将计算密集型的模型推理任务完全分发到独立的Web Worker线程中执行，是保障主线程（UI线程）流畅、提升用户体验的关键：

实现关键:

• 主线程保护: 所有AI计算在Worker中执行，UI始终响应
• 资源隔离: Worker崩溃不影响扩展核心功能
• 生命周期管理: 与插件功能状态绑定，及时释放资源
• 错误边界: 完善的错误捕获和恢复机制

3.3 核心组件协作模式

Background Script - 中央调度器

职责:

• 模型生命周期管理
• 任务队列调度
• 缓存策略控制
• 性能监控统计

Web Worker - 推理引擎

职责:

• 模型加载和初始化
• 推理计算执行
• 内存管理优化
• 错误处理和恢复

Content Script - 用户交互层

职责:

• DOM元素交互
• 用户界面渲染
• 事件处理和响应
• 数据收集和预处理

3.4 数据流设计与通信优化

3.4.1 文本相似性对比完整流程

flowchart TD
    A[用户输入文本对] --> B{缓存检查}
    B -->|命中| C[返回缓存结果]
    B -->|未命中| D[文本预处理]

    D --> E[分词处理<br/>transformers]
    E --> F[Token序列化]
    F --> G[数据传输到Worker]

    G --> H[Worker接收数据]
    H --> I[ONNX Runtime推理]
    I --> J[模型计算Embedding]
    J --> K[向量后处理]

    K --> L[计算余弦相似度]
    L --> M[结果传回主线程]
    M --> N[更新LRU缓存]
    N --> O[返回相似度分数]

    O --> P[UI展示结果]

    subgraph "主线程 (Background Script)"
        B
        C
        D
        E
        F
        M
        N
        O
    end

    subgraph "Web Worker线程"
        H
        I
        J
        K
        L
    end

    subgraph "缓存层"
        Q[LRU Cache<br/>500个条目]
        R[性能统计]
    end

    B -.-> Q
    N -.-> Q
    O -.-> R

    style A fill:#e1f5fe
    style C fill:#c8e6c9
    style P fill:#c8e6c9
    style I fill:#fff3e0
    style J fill:#fff3e0

3.4.2 批处理优化流程

flowchart LR
    A[多个文本请求] --> B[批处理队列]
    B --> C{队列大小检查}
    C -->|达到批次大小| D[立即处理]
    C -->|未达到| E[等待50ms]
    E --> F[超时处理]

    D --> G[批量分词]
    F --> G
    G --> H[批量推理]
    H --> I[结果分发]
    I --> J[更新缓存]

    subgraph "批处理优化"
        K[减少Worker通信次数]
        L[提高GPU/CPU利用率]
        M[降低平均延迟]
    end

    H -.-> K
    H -.-> L
    H -.-> M

    style A fill:#e3f2fd
    style G fill:#fff3e0
    style H fill:#fff3e0
    style I fill:#e8f5e8

简化的数据流概览:

用户输入 → 缓存检查 → 文本预处理 → 分词 → Worker推理 → 相似度计算 → 缓存更新 → 结果返回

数据传输优化

精心设计主线程与Web Worker之间的通信协议，确保传递的数据结构简洁高效：

// 尽量简洁的消息传递协议
interface WorkerMessage {
    id: number;
    type: 'init' | 'infer' | 'terminate';
    payload: {
        input_ids: number[]; // Token序列
        attention_mask: number[]; // 注意力掩码
        dims: { // 张量维度信息
        input_ids: number[];
        attention_mask: number[];
    };
};
// 数据传输优化示例
class OptimizedDataTransfer {
  private static createTransferableEmbedding(embedding: Float32Array): {
    data: ArrayBuffer;
    transfers: ArrayBuffer[];
  } {
    // 创建新的ArrayBuffer（这是必要的拷贝）
    const buffer = new ArrayBuffer(embedding.length * 4);
    const view = new Float32Array(buffer);
    view.set(embedding);

    return {
      data: buffer,
      transfers: [buffer] // 转移所有权，避免回传时拷贝
    };
  }

  // 批量传输优化：减少通信次数
  private static batchTokenData(tokenBatches: number[][]): {
    flatData: Int32Array;
    offsets: number[];
    transfers: ArrayBuffer[];
  } {
    const totalLength = tokenBatches.reduce((sum, batch) => sum + batch.length, 0);
    const buffer = new ArrayBuffer(totalLength * 4);
    const flatData = new Int32Array(buffer);
    const offsets: number[] = [];

    let currentOffset = 0;
    tokenBatches.forEach(batch => {
      offsets.push(currentOffset);
      flatData.set(batch, currentOffset);
      currentOffset += batch.length;
    });

    return {
      flatData,
      offsets,
      transfers: [buffer]
    };
  }
}

SharedArrayBuffer潜力探索

对于Token ID等序列数据，如果面对更大规模数据传输，可考虑SharedArrayBuffer，它的优势在于共享内存：

// 示例代码如下
class SharedMemoryCache {
  private sharedBuffer: SharedArrayBuffer;
  private metadataView: Int32Array;
  private dataView: Float32Array;

  constructor(maxEmbeddings: number, embeddingDim: number) {
    // 元数据区域：存储缓存状态
    const metadataSize = 1024; // 256个int32用于元数据
    const dataSize = maxEmbeddings * embeddingDim * 4; // embedding数据

    this.sharedBuffer = new SharedArrayBuffer(metadataSize + dataSize);
    this.metadataView = new Int32Array(this.sharedBuffer, 0, 256);
    this.dataView = new Float32Array(this.sharedBuffer, metadataSize);
  }

  // 真正的零拷贝：直接在共享内存中操作
  writeEmbeddingDirect(index: number, embedding: Float32Array): void {
    const offset = index * embedding.length;
    // ✅ 直接写入共享内存，无需拷贝
    this.dataView.set(embedding, offset);

    // 原子操作更新元数据
    Atomics.store(this.metadataView, index, 1); // 标记为已写入
  }

  // Worker中直接读取共享内存
  readEmbeddingDirect(index: number, embeddingDim: number): Float32Array {
    const offset = index * embeddingDim;
    // ✅ 直接从共享内存读取，无需拷贝
    return this.dataView.subarray(offset, offset + embeddingDim);
  }

  // 原子操作实现线程安全的缓存
  tryLockSlot(index: number): boolean {
    const expected = 0; // 空闲状态
    const desired = -1; // 锁定状态
    return Atomics.compareExchange(this.metadataView, index, expected, desired) === expected;
  }
}

🎯 实际优化策略对比:

优化方法	适用场景	真实效果	实现复杂度
Transferable Objects	大型ArrayBuffer	✅ 避免回传拷贝	低
批量传输	多个小数据	✅ 减少通信次数	中
SharedArrayBuffer	频繁读写共享数据	✅ 真正零拷贝	高
数据压缩	大型稀疏数据	✅ 减少传输量	中

关键设计原则：

• 异步非阻塞: 所有AI计算在Worker中执行，不阻塞主线程
• 智能缓存: LRU缓存策略，避免重复计算
• 错误隔离: Worker崩溃不影响扩展主功能
• 资源控制: 并发限制和内存监控
• 通信优化: 最小化数据传输开销，探索零拷贝方案

4. 🔬 核心实现深度解析

4.1 文本相似度对比原理与数学基础

在深入实现细节之前，我们首先需要理解文本相似度计算的数学原理。这是整个系统的理论基础，决定了我们的技术选型和优化策略。

4.1.1 文本向量化原理图

flowchart TD
    A["原始文本<br/>'机器学习很有趣'"] --> B[文本预处理]
    B --> C[分词处理]
    C --> D["Token序列<br/>[101, 1234, 5678, 9012, 102]"]

    D --> E[Embedding查找]
    E --> F["词向量矩阵<br/>768维 × 词汇表大小"]
    F --> G["Token Embeddings<br/>每个token → 768维向量"]

    G --> H[Transformer编码器]
    H --> I["多头自注意力机制<br/>12层 × 12头"]
    I --> J["上下文感知向量<br/>考虑词序和语义关系"]

    J --> K[池化操作]
    K --> L["句子级向量<br/>768维语义表示"]

    subgraph "数学变换"
        M["输入: x ∈ R^n"]
        N["变换: f(x) = Transformer(x)"]
        O["输出: v ∈ R^768"]
    end

    D -.-> M
    L -.-> O

    style A fill:#e3f2fd
    style L fill:#c8e6c9
    style H fill:#fff3e0
    style I fill:#fff3e0

4.1.2 余弦相似度计算原理

graph TB
    subgraph "向量空间表示"
        A["文本A向量<br/>vₐ = [0.2, 0.8, 0.1, ...]<br/>||vₐ|| = 1.0"]
        B["文本B向量<br/>vᵦ = [0.3, 0.7, 0.2, ...]<br/>||vᵦ|| = 1.0"]

        C["高维空间中的向量<br/>768维 Embedding Space"]
        A --> C
        B --> C
    end

    subgraph "相似度计算"
        D["点积计算<br/>vₐ · vᵦ = Σ(aᵢ × bᵢ)"]
        E["向量长度<br/>||vₐ|| = √(Σaᵢ²)<br/>||vᵦ|| = √(Σbᵢ²)"]
        F["余弦相似度<br/>cos(θ) = (vₐ · vᵦ) / (||vₐ|| × ||vᵦ||)"]

        C --> D
        C --> E
        D --> F
        E --> F
    end

    subgraph "结果解释"
        G["相似度分数<br/>范围: [-1, 1]"]
        H["1.0: 完全相同<br/>0.0: 无关<br/>-1.0: 完全相反"]

        F --> G
        G --> H
    end

    style D fill:#fff3e0
    style F fill:#e8f5e8
    style G fill:#c8e6c9

4.1.3 Embedding空间中的语义关系

graph LR
    subgraph "768维语义空间"
        A["'机器学习'<br/>v₁ = [0.2, 0.8, ...]"]
        B["'人工智能'<br/>v₂ = [0.3, 0.7, ...]"]
        C["'深度学习'<br/>v₃ = [0.25, 0.75, ...]"]
        D["'苹果水果'<br/>v₄ = [-0.1, 0.2, ...]"]

        A -.->|cos=0.85| B
        A -.->|cos=0.78| C
        B -.->|cos=0.82| C
        A -.->|cos=0.12| D
        B -.->|cos=0.08| D
        C -.->|cos=0.15| D
    end

    subgraph "语义聚类"
        E["技术概念簇<br/>高相似度区域"]
        F["日常概念簇<br/>低相似度区域"]

        A --> E
        B --> E
        C --> E
        D --> F
    end

    style E fill:#e8f5e8
    style F fill:#ffecb3

4.1.4 数学公式详解

1. 文本向量化过程

输入文本 T = "机器学习很有趣"

分词结果 tokens = [t₁, t₂, t₃, t₄, t₅]

词嵌入 E(tᵢ) ∈ R^768

上下文编码 H = Transformer(E(t₁), E(t₂), ..., E(t₅))

句子向量 v = MeanPooling(H) ∈ R^768

2. 余弦相似度计算

给定两个向量 vₐ, vᵦ ∈ R^768

点积: vₐ · vᵦ = Σᵢ₌₁⁷⁶⁸ aᵢ × bᵢ

向量模长: ||vₐ|| = √(Σᵢ₌₁⁷⁶⁸ aᵢ²)

||vᵦ|| = √(Σᵢ₌₁⁷⁶⁸ bᵢ²)

余弦相似度: similarity = cos(θ) = (vₐ · vᵦ) / (||vₐ|| × ||vᵦ||)

3. 实际计算示例

// 余弦相似度计算实现
function cosineSimilarity(vecA: Float32Array, vecB: Float32Array): number {
    let dotProduct = 0;
    let normA = 0;
    let normB = 0;

    for (let i = 0; i < vecA.length; i++) {
        dotProduct += vecA[i] * vecB[i];
        normA += vecA[i] * vecA[i];
        normB += vecB[i] * vecB[i];
    }

    const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
    return magnitude === 0 ? 0 : dotProduct / magnitude;
}

// 性能优化版本（利用TypedArray的内存布局优势）
function optimizedCosineSimilarity(vecA: Float32Array, vecB: Float32Array): number {
  const length = vecA.length;
  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  // 利用TypedArray的连续内存布局，减少内存访问开销
  // 展开循环减少分支预测失败
  let i = 0;
  const unrollLength = Math.floor(length / 8) * 8;

  // 8路展开循环
  for (; i < unrollLength; i += 8) {
    const a0 = vecA[i], a1 = vecA[i+1], a2 = vecA[i+2], a3 = vecA[i+3];
    const a4 = vecA[i+4], a5 = vecA[i+5], a6 = vecA[i+6], a7 = vecA[i+7];
    const b0 = vecB[i], b1 = vecB[i+1], b2 = vecB[i+2], b3 = vecB[i+3];
    const b4 = vecB[i+4], b5 = vecB[i+5], b6 = vecB[i+6], b7 = vecB[i+7];

    dotProduct += a0*b0 + a1*b1 + a2*b2 + a3*b3 + a4*b4 + a5*b5 + a6*b6 + a7*b7;
    normA += a0*a0 + a1*a1 + a2*a2 + a3*a3 + a4*a4 + a5*a5 + a6*a6 + a7*a7;
    normB += b0*b0 + b1*b1 + b2*b2 + b3*b3 + b4*b4 + b5*b5 + b6*b6 + b7*b7;
  }

  // 处理剩余元素
  for (; i < length; i++) {
    const a = vecA[i], b = vecB[i];
    dotProduct += a * b;
    normA += a * a;
    normB += b * b;
  }

  const magnitude = Math.sqrt(normA) * Math.sqrt(normB);
  return magnitude === 0 ? 0 : dotProduct / magnitude;
}

// WebAssembly + SIMD的优化版本示例
function wasmSIMDCosineSimilarity(vecA: Float32Array, vecB: Float32Array): number {
  // 这需要编译专门的WASM模块
  if (typeof WebAssembly !== 'undefined' && 'simd' in WebAssembly) {
    // 调用WASM SIMD函数
    return wasmModule.cosine_similarity_simd(vecA, vecB);
  }
  // 降级到JavaScript优化版本
  return simdOptimizedCosineSimilarity(vecA, vecB);
}

// WASM SIMD C代码示例
#include <wasm_simd128.h>
float cosine_similarity_simd(float* a, float* b, int length) {
 v128_t dot_sum = wasm_f32x4_splat(0.0f);
 v128_t norm_a_sum = wasm_f32x4_splat(0.0f);
 v128_t norm_b_sum = wasm_f32x4_splat(0.0f);

 for (int i = 0; i < length; i += 4) {
   v128_t va = wasm_v128_load(&a[i]);
   v128_t vb = wasm_v128_load(&b[i]);

   // 真正的SIMD并行计算
   dot_sum = wasm_f32x4_add(dot_sum, wasm_f32x4_mul(va, vb));
   norm_a_sum = wasm_f32x4_add(norm_a_sum, wasm_f32x4_mul(va, va));
   norm_b_sum = wasm_f32x4_add(norm_b_sum, wasm_f32x4_mul(vb, vb));
 }

 // 水平求和
 float dot = horizontal_sum(dot_sum);
 float norm_a = sqrt(horizontal_sum(norm_a_sum));
 float norm_b = sqrt(horizontal_sum(norm_b_sum));

 return dot / (norm_a * norm_b);
}

// ONNX Runtime中的SIMD优化
// 在Worker中启用SIMD
ort.env.wasm.simd = true; // 这里开启真正的SIMD优化
// ONNX Runtime内部会使用SIMD指令进行：
// - 矩阵乘法加速
// - 激活函数并行计算
// - 向量运算优化

4.1.5 语义相似度的几何解释

在768维的Embedding空间中，每个文本都被表示为一个向量。语义相似的文本在这个高维空间中会聚集在相近的区域，而语义差异较大的文本则会分布在较远的位置。

💡 关键洞察:

1. 📐 角度vs距离: 余弦相似度关注向量间的角度而非距离，更适合文本语义比较
2. 🌀 维度诅咒: 在高维空间中，欧几里得距离会失效，余弦相似度保持稳定性
3. 🎯 语义聚类: 相似主题的文本会在Embedding空间中形成自然的聚类
4. 🔄 上下文敏感: Transformer模型生成的向量考虑了词序和上下文关系

⚡ 性能考量:

• 📊 768维向量的余弦相似度计算复杂度为O(n)
• 🚀 批量计算时可利用矩阵运算加速
• 🔧 向量归一化可以简化计算（预计算模长）

4.2 模型加载与初始化策略

预加载机制

// ✅ 智能预加载策略核心代码
class SmartPreloadingEngine {
  private preloadPromise: Promise<void> | null = null;
  private isPreloading = false;
  private preloadedAssets = new Set<string>();

  constructor(options: ModelConfig) {
    this.config = this.mergeConfig(options);
    // 🚀 立即开始预加载关键资源
    this.startPreloading();
    // 📊 监听用户行为，预测性加载
    this.setupPredictiveLoading();
  }

  // 🚀 核心预加载逻辑
  private async startPreloading(): Promise<void> {
    if (this.isPreloading) return;

    this.isPreloading = true;
    this.preloadPromise = this.executePreloadingStrategy();

    try {
      await this.preloadPromise;
      console.log('Preloading completed successfully');
    } catch (error) {
      console.warn('Preloading failed, will load on demand:', error);
    } finally {
      this.isPreloading = false;
    }
  }

  private async executePreloadingStrategy(): Promise<void> {
    // 1. 🔧 预加载WASM文件
    await this.preloadWasmFiles();
    // 2. 📦 预加载模型文件（分块）
    await this.preloadModelFiles();
    // 3. 📚 预加载分词器
    await this.preloadTokenizer();
    // 4. 🔥 预热Worker
    await this.preloadWorker();
  }

  // 🔧 WASM文件预加载
  private async preloadWasmFiles(): Promise<void> {
    const wasmFiles = [
      'ort-wasm-simd-threaded.wasm',
      'ort-wasm-simd-threaded.jsep.wasm',
    ];

    const preloadPromises = wasmFiles.map(async (filename) => {
      try {
        const url = chrome.runtime.getURL(`workers/${filename}`);
        const response = await fetch(url);

        if (response.ok) {
          // 预加载到浏览器缓存
          await response.arrayBuffer();
          this.preloadedAssets.add(filename);
          console.log(`Preloaded WASM: ${filename}`);
        }
      } catch (error) {
        console.warn(`Failed to preload WASM ${filename}:`, error);
      }
    });

    await Promise.allSettled(preloadPromises);
  }

  // 📦 模型文件预加载（分块策略）
  private async preloadModelFiles(): Promise<void> {
    const modelUrl = chrome.runtime.getURL('models/model.onnx');

    try {
      // 🎯 只预加载模型文件的头部信息（前1MB）
      const response = await fetch(modelUrl, {
        headers: { 'Range': 'bytes=0-1048575' } // 前1MB
      });

      if (response.ok) {
        await response.arrayBuffer();
        this.preloadedAssets.add('model_header');
        console.log('Preloaded model header (1MB)');

        // 🔄 后台继续预加载完整模型
        this.backgroundPreloadFullModel(modelUrl);
      }
    } catch (error) {
      console.warn('Failed to preload model header:', error);
    }
  }

  // 🔄 后台预加载完整模型
  private backgroundPreloadFullModel(modelUrl: string): void {
    // 使用低优先级后台任务
    if ('scheduler' in window && 'postTask' in (window as any).scheduler) {
      (window as any).scheduler.postTask(async () => {
        try {
          const response = await fetch(modelUrl);
          if (response.ok) {
            await response.arrayBuffer();
            this.preloadedAssets.add('full_model');
            console.log('Background preloaded full model');
          }
        } catch (error) {
          console.warn('Background model preload failed:', error);
        }
      }, { priority: 'background' });
    } else {
      // 降级到setTimeout
      setTimeout(async () => {
        try {
          const response = await fetch(modelUrl);
          if (response.ok) {
            await response.arrayBuffer();
            this.preloadedAssets.add('full_model');
          }
        } catch (error) {
          console.warn('Background model preload failed:', error);
        }
      }, 2000); // 2秒后开始
    }
  }

  // 📚 分词器预加载
  private async preloadTokenizer(): Promise<void> {
    try {
      // 🚀 提前初始化分词器
      const { AutoTokenizer } = await import('@xenova/transformers');

      this.tokenizer = await AutoTokenizer.from_pretrained(
        this.config.modelIdentifier,
        {
          local_files_only: false,
          cache_dir: './models/',
          // 🎯 只加载必要的文件
          revision: 'main'
        }
      );

      this.preloadedAssets.add('tokenizer');
      console.log('Preloaded tokenizer');
    } catch (error) {
      console.warn('Failed to preload tokenizer:', error);
    }
  }

  // 🔥 Worker预热
  private async preloadWorker(): Promise<void> {
    try {
      // 🚀 提前创建Worker
      this.setupWorker();

      // 🔥 发送初始化消息
      await this.sendToWorker('init', {
        modelPath: chrome.runtime.getURL('models/model.onnx'),
        config: this.config
      });

      this.preloadedAssets.add('worker');
      console.log('Preloaded and warmed up worker');
    } catch (error) {
      console.warn('Failed to preload worker:', error);
    }
  }

  // 📊 预测性加载
  private setupPredictiveLoading(): void {
    // 监听扩展激活事件
    chrome.action?.onClicked.addListener(() => {
      this.triggerPredictiveLoad('user_action');
    });

    // 监听标签页变化
    chrome.tabs?.onActivated.addListener(() => {
      this.triggerPredictiveLoad('tab_change');
    });

    // 监听页面加载完成
    chrome.webNavigation?.onCompleted.addListener(() => {
      this.triggerPredictiveLoad('page_loaded');
    });
  }

  private triggerPredictiveLoad(trigger: string): void {
    // 🎯 基于用户行为预测性加载
    if (!this.isInitialized && !this.isPreloading) {
      console.log(`Predictive loading triggered by: ${trigger}`);
      this.startPreloading();
    }
  }

  // 📊 预加载状态检查
  public getPreloadStatus(): {
    isComplete: boolean;
    loadedAssets: string[];
    progress: number;
  } {
    const totalAssets = ['ort-wasm-simd-threaded.wasm', 'model_header', 'tokenizer', 'worker'];
    const loadedCount = totalAssets.filter(asset => this.preloadedAssets.has(asset)).length;

    return {
      isComplete: loadedCount === totalAssets.length,
      loadedAssets: Array.from(this.preloadedAssets),
      progress: loadedCount / totalAssets.length
    };
  }

  // 🎯 智能初始化（利用预加载结果）
  public async initialize(): Promise<void> {
    // 等待预加载完成（如果还在进行中）
    if (this.preloadPromise) {
      await this.preloadPromise;
    }

    // 🚀 利用预加载的资源快速初始化
    if (this.preloadedAssets.has('tokenizer') && this.tokenizer) {
      console.log('Using preloaded tokenizer');
    } else {
      await this.initializeTokenizer();
    }

    if (this.preloadedAssets.has('worker') && this.worker) {
      console.log('Using preloaded worker');
    } else {
      this.setupWorker();
      await this.initializeModel();
    }

    this.isInitialized = true;
  }
}

// 🔧 配置检测
private detectOptimalProviders(): string[] {
  // WebAssembly SIMD支持检测
  if (typeof WebAssembly === 'object' &&
      WebAssembly.validate(new Uint8Array([0, 97, 115, 109, 1, 0, 0, 0]))) {
    return ['wasm'];
  }
  return ['webgl']; // 降级到WebGL
}

private calculateOptimalConcurrency(): number {
  const cores = navigator.hardwareConcurrency || 4;
  const memory = (navigator as any).deviceMemory || 4; // GB

  // 🎯 基于硬件动态调整并发数
  if (memory >= 8 && cores >= 8) {
    return Math.min(4, Math.floor(cores / 2));
  } else if (memory >= 4 && cores >= 4) {
    return Math.min(2, Math.floor(cores / 3));
  } else {
    return 1; // 低配设备保守策略
  }
}

懒加载优化

// 真正使用时才会调用
public async initialize(): Promise<void> {
  if (this.isInitialized) return Promise.resolve();
  if (this.isInitializing && this.initPromise) return this.initPromise;

  this.isInitializing = true;
  this.initPromise = this._doInitialize().finally(() => {
    this.isInitializing = false;
    this.warmupModel(); // 预热模型
  });
  return this.initPromise;
}

4.2 Web Worker优化策略

消息队列优化：

• ✅ 优先级调度: 不同任务类型的优先级处理
• ✅ 容量控制: 防止内存溢出的队列大小限制
• ✅ 超时管理: 自动清理长时间无响应的任务
• ✅ 批处理优化: 自动合并相似任务减少通信开销

并发控制优化：

• ✅ 多Worker管理: 动态创建和管理多个Worker实例
• ✅ 负载均衡: 智能分配任务到最优Worker
• ✅ 健康监控: 实时监控Worker状态和性能
• ✅ 故障恢复: 自动处理Worker崩溃和重启

消息队列管理

class WorkerMessageQueue {
  private messageQueue: QueuedMessage[] = [];
  private pendingMessages = new Map<number, PendingMessage>();
  private runningTasks = 0;
  private nextMessageId = 1;
  private worker: Worker | null = null;

  // 🎯 配置参数
  private readonly config = {
    maxConcurrentTasks: 3,           // 最大并发任务数
    maxQueueSize: 100,               // 最大队列长度
    taskTimeout: 30000,              // 任务超时时间
    priorityLevels: ['high', 'normal', 'low'] as const,
    batchSize: 8,                    // 批处理大小
    retryAttempts: 3,                // 重试次数
  };

  // 📊 统计信息
  private stats = {
    totalMessages: 0,
    completedMessages: 0,
    failedMessages: 0,
    averageProcessingTime: 0,
    queueWaitTime: 0,
  };

  constructor(worker: Worker) {
    this.worker = worker;
    this.setupWorkerMessageHandler();
    this.startQueueProcessor();
    this.startPerformanceMonitoring();
  }

  // 🚀 核心队列调度方法
  public async sendMessage<T>(
    type: MessageType,
    payload: any,
    options: MessageOptions = {}
  ): Promise<T> {
    const message: QueuedMessage = {
      id: this.nextMessageId++,
      type,
      payload,
      priority: options.priority || 'normal',
      timeout: options.timeout || this.config.taskTimeout,
      retryCount: 0,
      maxRetries: options.maxRetries || this.config.retryAttempts,
      timestamp: Date.now(),
      batchable: options.batchable || false,
    };

    return new Promise<T>((resolve, reject) => {
      message.resolve = resolve;
      message.reject = reject;

      // 🎯 队列容量检查
      if (this.messageQueue.length >= this.config.maxQueueSize) {
        reject(new Error('Message queue is full'));
        return;
      }

      // 📋 按优先级插入队列
      this.insertMessageByPriority(message);
      this.stats.totalMessages++;

      // 🔄 触发队列处理
      this.processQueue();
    });
  }

  // 🎯 智能优先级插入
  private insertMessageByPriority(message: QueuedMessage): void {
    const priorityOrder = { high: 0, normal: 1, low: 2 };
    const insertIndex = this.messageQueue.findIndex(
      msg => priorityOrder[msg.priority] > priorityOrder[message.priority]
    );

    if (insertIndex === -1) {
      this.messageQueue.push(message);
    } else {
      this.messageQueue.splice(insertIndex, 0, message);
    }
  }

  // 🔄 队列处理器
  private async processQueue(): Promise<void> {
    // 检查并发限制
    if (this.runningTasks >= this.config.maxConcurrentTasks) {
      return;
    }

    // 🎯 批处理优化：收集可批处理的消息
    const batchableMessages = this.collectBatchableMessages();
    if (batchableMessages.length >= this.config.batchSize) {
      await this.processBatch(batchableMessages);
      return;
    }

    // 🚀 处理单个消息
    const message = this.messageQueue.shift();
    if (!message) return;

    this.runningTasks++;
    this.pendingMessages.set(message.id, {
      resolve: message.resolve!,
      reject: message.reject!,
      type: message.type,
      timestamp: message.timestamp,
      timeout: setTimeout(() => {
        this.handleMessageTimeout(message.id);
      }, message.timeout)
    });

    try {
      // 📊 记录等待时间
      const waitTime = Date.now() - message.timestamp;
      this.updateWaitTimeStats(waitTime);

      // 🚀 发送消息到Worker
      this.worker?.postMessage({
        id: message.id,
        type: message.type,
        payload: message.payload,
        timestamp: Date.now()
      });

    } catch (error) {
      this.handleMessageError(message.id, error as Error);
    }

    // 🔄 继续处理队列
    setImmediate(() => this.processQueue());
  }

  // 📦 批处理优化
  private collectBatchableMessages(): QueuedMessage[] {
    const batchable: QueuedMessage[] = [];
    const remaining: QueuedMessage[] = [];

    for (const message of this.messageQueue) {
      if (message.batchable && message.type === 'embedding' && batchable.length < this.config.batchSize) {
        batchable.push(message);
      } else {
        remaining.push(message);
      }
    }

    this.messageQueue = remaining;
    return batchable;
  }

  private async processBatch(messages: QueuedMessage[]): Promise<void> {
    this.runningTasks++;

    const batchId = this.nextMessageId++;
    const batchPayload = {
      texts: messages.map(msg => msg.payload.text),
      options: messages[0].payload.options
    };

    try {
      // 🚀 发送批处理请求
      this.worker?.postMessage({
        id: batchId,
        type: 'batch_embedding',
        payload: batchPayload,
        messageIds: messages.map(msg => msg.id)
      });

      // 📋 注册批处理消息
      messages.forEach(msg => {
        this.pendingMessages.set(msg.id, {
          resolve: msg.resolve!,
          reject: msg.reject!,
          type: msg.type,
          timestamp: msg.timestamp,
          batchId: batchId
        });
      });

    } catch (error) {
      messages.forEach(msg => {
        msg.reject?.(error as Error);
      });
    } finally {
      this.runningTasks--;
    }
  }

  // 📨 Worker消息处理
  private setupWorkerMessageHandler(): void {
    this.worker?.addEventListener('message', (event) => {
      const { id, type, payload, error, messageIds } = event.data;

      if (type === 'batch_response' && messageIds) {
        // 🎯 处理批处理响应
        this.handleBatchResponse(messageIds, payload, error);
      } else {
        // 🎯 处理单个消息响应
        this.handleSingleResponse(id, payload, error);
      }
    });

    this.worker?.addEventListener('error', (error) => {
      console.error('Worker error:', error);
      this.handleWorkerError(error);
    });
  }

  private handleSingleResponse(id: number, payload: any, error?: any): void {
    const pending = this.pendingMessages.get(id);
    if (!pending) return;

    // 🧹 清理
    this.pendingMessages.delete(id);
    if (pending.timeout) {
      clearTimeout(pending.timeout);
    }
    this.runningTasks--;

    // 📊 更新统计
    const processingTime = Date.now() - pending.timestamp;
    this.updateProcessingTimeStats(processingTime);

    if (error) {
      this.stats.failedMessages++;
      pending.reject(new Error(error.message || 'Worker processing failed'));
    } else {
      this.stats.completedMessages++;
      pending.resolve(payload);
    }

    // 🔄 继续处理队列
    setImmediate(() => this.processQueue());
  }

  private handleBatchResponse(messageIds: number[], payload: any[], error?: any): void {
    messageIds.forEach((id, index) => {
      const pending = this.pendingMessages.get(id);
      if (!pending) return;

      this.pendingMessages.delete(id);
      if (pending.timeout) {
        clearTimeout(pending.timeout);
      }

      if (error) {
        pending.reject(new Error(error.message || 'Batch processing failed'));
      } else {
        pending.resolve(payload[index]);
      }
    });

    this.runningTasks--;
    setImmediate(() => this.processQueue());
  }

  // ⏰ 超时处理
  private handleMessageTimeout(messageId: number): void {
    const pending = this.pendingMessages.get(messageId);
    if (!pending) return;

    this.pendingMessages.delete(messageId);
    this.runningTasks--;
    this.stats.failedMessages++;

    pending.reject(new Error(`Message ${messageId} timed out`));

    console.warn(`Message ${messageId} timed out after ${pending.timeout}ms`);
    setImmediate(() => this.processQueue());
  }

  // 🔄 重试机制
  private async retryMessage(message: QueuedMessage): Promise<void> {
    if (message.retryCount >= message.maxRetries) {
      message.reject?.(new Error(`Message failed after ${message.maxRetries} retries`));
      return;
    }

    message.retryCount++;
    message.timestamp = Date.now(); // 重置时间戳

    // 🎯 指数退避延迟
    const delay = Math.min(1000 * Math.pow(2, message.retryCount), 10000);
    setTimeout(() => {
      this.insertMessageByPriority(message);
      this.processQueue();
    }, delay);
  }

  // 📊 性能监控
  private startPerformanceMonitoring(): void {
    setInterval(() => {
      const stats = this.getStats();

      // 🚨 性能警告
      if (stats.queueLength > this.config.maxQueueSize * 0.8) {
        console.warn('Message queue is nearly full:', stats);
      }

      if (stats.averageProcessingTime > 5000) {
        console.warn('High processing time detected:', stats);
      }

      // 📊 定期清理过期消息
      this.cleanupExpiredMessages();

    }, 10000); // 每10秒检查一次
  }

  private cleanupExpiredMessages(): void {
    const now = Date.now();
    const expiredIds: number[] = [];

    this.pendingMessages.forEach((pending, id) => {
      if (now - pending.timestamp > this.config.taskTimeout * 2) {
        expiredIds.push(id);
      }
    });

    expiredIds.forEach(id => {
      this.handleMessageTimeout(id);
    });
  }

  // 📊 统计信息更新
  private updateProcessingTimeStats(time: number): void {
    const total = this.stats.completedMessages;
    this.stats.averageProcessingTime =
      (this.stats.averageProcessingTime * (total - 1) + time) / total;
  }

  private updateWaitTimeStats(time: number): void {
    const total = this.stats.totalMessages;
    this.stats.queueWaitTime =
      (this.stats.queueWaitTime * (total - 1) + time) / total;
  }

  // 📊 获取统计信息
  public getStats(): QueueStats {
    return {
      queueLength: this.messageQueue.length,
      runningTasks: this.runningTasks,
      pendingMessages: this.pendingMessages.size,
      totalMessages: this.stats.totalMessages,
      completedMessages: this.stats.completedMessages,
      failedMessages: this.stats.failedMessages,
      successRate: this.stats.completedMessages / this.stats.totalMessages,
      averageProcessingTime: this.stats.averageProcessingTime,
      averageWaitTime: this.stats.queueWaitTime,
    };
  }

  // 🧹 资源清理
  public dispose(): void {
    // 清理所有待处理消息
    this.pendingMessages.forEach(pending => {
      if (pending.timeout) {
        clearTimeout(pending.timeout);
      }
      pending.reject(new Error('Worker queue disposed'));
    });

    this.pendingMessages.clear();
    this.messageQueue.length = 0;
    this.runningTasks = 0;
  }
}

并发控制机制

// ✅ 真正的并发控制机制
class WorkerConcurrencyManager {
  private workers: WorkerInstance[] = [];
  private roundRobinIndex = 0;
  private loadBalancer: LoadBalancer;

  constructor(workerCount: number = 2) {
    this.initializeWorkers(workerCount);
    this.loadBalancer = new LoadBalancer(this.workers);
    this.startHealthMonitoring();
  }

  // 🚀 智能任务分发
  public async executeTask<T>(
    type: MessageType,
    payload: any,
    options: TaskOptions = {}
  ): Promise<T> {
    // 🎯 选择最优Worker
    const worker = await this.selectOptimalWorker(type, options);

    if (!worker) {
      throw new Error('No available workers');
    }

    // 📊 更新负载统计
    worker.currentLoad++;
    worker.totalTasks++;

    try {
      const startTime = Date.now();
      const result = await worker.messageQueue.sendMessage<T>(type, payload, options);

      // 📊 更新性能统计
      const duration = Date.now() - startTime;
      worker.averageTaskTime = (worker.averageTaskTime + duration) / 2;
      worker.lastTaskTime = Date.now();

      return result;
    } finally {
      worker.currentLoad--;
    }
  }

  // 🎯 Worker选择策略
  private async selectOptimalWorker(
    type: MessageType,
    options: TaskOptions
  ): Promise<WorkerInstance | null> {
    const availableWorkers = this.workers.filter(w => w.isHealthy && !w.isTerminating);

    if (availableWorkers.length === 0) {
      return null;
    }

    // 🚨 高优先级任务：选择负载最低的Worker
    if (options.priority === 'high') {
      return availableWorkers.reduce((min, worker) =>
        worker.currentLoad < min.currentLoad ? worker : min
      );
    }

    // 📦 批处理任务：选择有相同类型任务的Worker
    if (options.batchable) {
      const batchWorker = availableWorkers.find(w =>
        w.messageQueue.hasBatchableMessages(type)
      );
      if (batchWorker) return batchWorker;
    }

    // 🔄 默认：轮询负载均衡
    return this.loadBalancer.getNextWorker();
  }

  // 📊 健康监控
  private startHealthMonitoring(): void {
    setInterval(() => {
      this.workers.forEach(worker => {
        this.checkWorkerHealth(worker);
      });

      this.rebalanceLoad();
    }, 5000);
  }

  private checkWorkerHealth(worker: WorkerInstance): void {
    const now = Date.now();
    const timeSinceLastTask = now - worker.lastTaskTime;
    const queueStats = worker.messageQueue.getStats();

    // 🚨 健康检查指标
    worker.isHealthy =
      timeSinceLastTask < 60000 &&           // 1分钟内有活动
      queueStats.successRate > 0.9 &&        // 成功率>90%
      queueStats.averageProcessingTime < 10000 && // 平均处理时间<10s
      worker.currentLoad < 10;                // 当前负载<10

    if (!worker.isHealthy) {
      console.warn(`Worker ${worker.id} health check failed:`, {
        timeSinceLastTask,
        successRate: queueStats.successRate,
        avgProcessingTime: queueStats.averageProcessingTime,
        currentLoad: worker.currentLoad
      });
    }
  }

  // ⚖️ 负载重平衡
  private rebalanceLoad(): void {
    const totalLoad = this.workers.reduce((sum, w) => sum + w.currentLoad, 0);
    const averageLoad = totalLoad / this.workers.length;

    // 🎯 如果负载不均衡，触发重平衡
    const maxLoad = Math.max(...this.workers.map(w => w.currentLoad));
    if (maxLoad > averageLoad * 2) {
      console.log('Load imbalance detected, triggering rebalance');
      this.loadBalancer.rebalance();
    }
  }
}

// 🔧 负载均衡器
class LoadBalancer {
  private workers: WorkerInstance[];
  private strategy: 'round-robin' | 'least-connections' | 'weighted' = 'least-connections';
  private roundRobinIndex = 0;

  constructor(workers: WorkerInstance[]) {
    this.workers = workers;
  }

  getNextWorker(): WorkerInstance | null {
    const healthyWorkers = this.workers.filter(w => w.isHealthy);

    if (healthyWorkers.length === 0) {
      return null;
    }

    switch (this.strategy) {
      case 'round-robin':
        return this.roundRobinSelection(healthyWorkers);
      case 'least-connections':
        return this.leastConnectionsSelection(healthyWorkers);
      case 'weighted':
        return this.weightedSelection(healthyWorkers);
      default:
        return healthyWorkers[0];
    }
  }

  private roundRobinSelection(workers: WorkerInstance[]): WorkerInstance {
    const worker = workers[this.roundRobinIndex % workers.length];
    this.roundRobinIndex++;
    return worker;
  }

  private leastConnectionsSelection(workers: WorkerInstance[]): WorkerInstance {
    return workers.reduce((min, worker) =>
      worker.currentLoad < min.currentLoad ? worker : min
    );
  }

  private weightedSelection(workers: WorkerInstance[]): WorkerInstance {
    // 🎯 基于性能权重选择
    const weights = workers.map(w => {
      const performanceScore = 1000 / (w.averageTaskTime || 1000);
      const loadScore = Math.max(1, 10 - w.currentLoad);
      return performanceScore * loadScore;
    });

    const totalWeight = weights.reduce((sum, w) => sum + w, 0);
    let random = Math.random() * totalWeight;

    for (let i = 0; i < workers.length; i++) {
      random -= weights[i];
      if (random <= 0) {
        return workers[i];
      }
    }

    return workers[0];
  }

  rebalance(): void {
    // 🔄 动态调整策略
    const avgLoad = this.workers.reduce((sum, w) => sum + w.currentLoad, 0) / this.workers.length;

    if (avgLoad > 5) {
      this.strategy = 'least-connections';
    } else {
      this.strategy = 'round-robin';
    }
  }
}

// 🔧 Worker实例定义
interface WorkerInstance {
  id: number;
  worker: Worker;
  messageQueue: WorkerMessageQueue;
  currentLoad: number;
  totalTasks: number;
  averageTaskTime: number;
  lastTaskTime: number;
  isHealthy: boolean;
  isTerminating: boolean;
}

interface TaskOptions extends MessageOptions {
  workerId?: number;
  preferredWorker?: WorkerInstance;
}

4.3 内存管理与缓存优化

LRU缓存实现

// LRUCache简要实现
class LRUCache<K = string, V = any> {
  private capacity: number;
  private cache: Map<K, V>;

  get(key: K): V | null {
    if (this.cache.has(key)) {
      const value = this.cache.get(key)!;
      this.cache.delete(key);
      this.cache.set(key, value); // 移到末尾（最近使用）
      return value;
    }
    return null;
  }

  set(key: K, value: V): void {
    if (this.cache.has(key)) {
      this.cache.delete(key);
    } else if (this.cache.size >= this.capacity) {
      // 删除最久未使用的项
      const firstKey = this.cache.keys().next().value!;
      this.cache.delete(firstKey);
    }
    this.cache.set(key, value);
  }
}

批处理优化

// 批处理优化示例
public async getEmbeddingsBatch(
  texts: string[],
  options: Record<string, any> = {},
): Promise<Float32Array[]> {
  // 缓存命中检查
  const results: (Float32Array | undefined)[] = new Array(texts.length).fill(undefined);
  const uncachedTextsMap = new Map<string, number[]>();

  texts.forEach((text, index) => {
    const cacheKey = this.getCacheKey(text, options);
    const cached = this.embeddingCache.get(cacheKey);
    if (cached) {
      results[index] = cached;
      this.cacheStats.hits++;
    } else {
      // 收集未缓存的文本进行批处理
      if (!uncachedTextsMap.has(text)) {
        uncachedTextsMap.set(text, []);
        textsToTokenize.push(text);
      }
      uncachedTextsMap.get(text)!.push(index);
    }
  });

  // 批量处理未缓存的文本
  if (textsToTokenize.length > 0) {
    const batchEmbeddings = await this.computeBatchEmbeddings(textsToTokenize);
    // 更新缓存和结果
  }

  return results as Float32Array[];
}

5. ⚡ 追求极致：多维度性能优化策略

在基础架构之上，继续实施了一系列深度优化措施，旨在榨干每一分性能潜力：

5.1 🛠️ WASM后端深度调优

5.1.1 SIMD与多线程并行

优先选用支持SIMD（单指令多数据流）和多线程的Wasm模块，充分利用现代CPU的并行处理能力：

// ✅ SIMD与多线程并行优化策略
class WasmSIMDOptimizer {
  private simdCapabilities: SIMDCapabilities;
  private threadPool: ThreadPool;
  private performanceProfiler: PerformanceProfiler;

  constructor() {
    this.simdCapabilities = this.detectSIMDCapabilities();
    this.threadPool = new ThreadPool();
    this.performanceProfiler = new PerformanceProfiler();
    this.configureOptimalSettings();
  }

  // 🔍 SIMD能力检测
  private detectSIMDCapabilities(): SIMDCapabilities {
    const capabilities: SIMDCapabilities = {
      hasWasmSIMD: false,
      hasAVX2: false,
      hasSSE4: false,
      vectorWidth: 128, // 默认128位
      supportedOperations: []
    };

    try {
      // 🧪 检测WebAssembly SIMD支持
      const wasmSIMDTest = new Uint8Array([
        0x00, 0x61, 0x73, 0x6d, // WASM magic
        0x01, 0x00, 0x00, 0x00, // version
        0x01, 0x05, 0x01, 0x60, // type section
        0x00, 0x01, 0x7b,       // func type: () -> v128
        0x03, 0x02, 0x01, 0x00, // function section
        0x0a, 0x0a, 0x01, 0x08, // code section
        0x00, 0xfd, 0x0c,       // v128.const
        0x00, 0x00, 0x00, 0x00, // 16 bytes of zeros
        0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00,
        0x0b                    // end
      ]);

      if (WebAssembly.validate(wasmSIMDTest)) {
        capabilities.hasWasmSIMD = true;
        capabilities.vectorWidth = 128; // WASM SIMD是128位
        capabilities.supportedOperations = [
          'f32x4.add', 'f32x4.mul', 'f32x4.sub',
          'i32x4.add', 'i32x4.mul', 'v128.load', 'v128.store'
        ];
      }

      // 🔍 通过性能测试推断CPU SIMD能力
      this.benchmarkSIMDPerformance(capabilities);

    } catch (error) {
      console.warn('SIMD capability detection failed:', error);
    }

    return capabilities;
  }

  // 📊 SIMD性能基准测试
  private async benchmarkSIMDPerformance(capabilities: SIMDCapabilities): Promise<void> {
    if (!capabilities.hasWasmSIMD) return;

    try {
      // 🧪 创建SIMD性能测试WASM模块
      const testModule = await this.createSIMDTestModule();
      const testResults = await this.runSIMDPerformanceTests(testModule);

      // 📊 分析结果推断硬件能力
      if (testResults.vectorAddThroughput > 8000) {
        capabilities.hasAVX2 = true;
        capabilities.vectorWidth = 256;
      } else if (testResults.vectorAddThroughput > 4000) {
        capabilities.hasSSE4 = true;
        capabilities.vectorWidth = 128;
      }

      console.log('SIMD Performance Benchmark:', testResults);
    } catch (error) {
      console.warn('SIMD performance benchmark failed:', error);
    }
  }

  // 🔧 创建SIMD测试模块
  private async createSIMDTestModule(): Promise<WebAssembly.Module> {
    // 🚀 手写WASM字节码，包含SIMD指令
    const wasmBytes = new Uint8Array([
      0x00, 0x61, 0x73, 0x6d, 0x01, 0x00, 0x00, 0x00, // WASM header

      // Type section: 定义函数签名
      0x01, 0x07, 0x01, 0x60, 0x02, 0x7f, 0x7f, 0x01, 0x7f,

      // Function section: 声明函数
      0x03, 0x02, 0x01, 0x00,

      // Memory section: 声明内存
      0x05, 0x03, 0x01, 0x00, 0x01,

      // Export section: 导出函数和内存
      0x07, 0x11, 0x02, 0x06, 0x6d, 0x65, 0x6d, 0x6f, 0x72, 0x79,
      0x02, 0x00, 0x08, 0x73, 0x69, 0x6d, 0x64, 0x5f, 0x61, 0x64, 0x64,
      0x00, 0x00,

      // Code section: 实际的SIMD代码
      0x0a, 0x20, 0x01, 0x1e, 0x00,
      // 函数体开始
      0x20, 0x00, // local.get 0 (第一个参数：数组偏移)
      0xfd, 0x00, 0x10, 0x00, // v128.load (加载128位向量)

      0x20, 0x01, // local.get 1 (第二个参数：数组偏移)
      0xfd, 0x00, 0x10, 0x00, // v128.load (加载第二个128位向量)

      0xfd, 0xe0, 0x00, // f32x4.add (SIMD加法：4个float32并行相加)

      0x20, 0x00, // local.get 0
      0xfd, 0x0b, 0x10, 0x00, // v128.store (存储结果)

      0x41, 0x01, // i32.const 1 (返回成功标志)
      0x0b // end
    ]);

    return await WebAssembly.compile(wasmBytes);
  }

  // 🧪 运行SIMD性能测试
  private async runSIMDPerformanceTests(module: WebAssembly.Module): Promise<SIMDTestResults> {
    const instance = await WebAssembly.instantiate(module);
    const memory = instance.exports.memory as WebAssembly.Memory;
    const simdAdd = instance.exports.simd_add as Function;

    const testData = new Float32Array(memory.buffer, 0, 1024);

    // 🔥 填充测试数据
    for (let i = 0; i < testData.length; i++) {
      testData[i] = Math.random();
    }

    // 📊 性能测试：向量加法吞吐量
    const iterations = 100000;
    const startTime = performance.now();

    for (let i = 0; i < iterations; i++) {
      simdAdd(0, 64); // 处理64字节 = 16个float32 = 4个SIMD向量
    }

    const endTime = performance.now();
    const duration = endTime - startTime;
    const throughput = (iterations * 16) / duration; // 每毫秒处理的float32数量

    return {
      vectorAddThroughput: throughput,
      averageLatency: duration / iterations,
      memoryBandwidth: (iterations * 64) / duration, // MB/ms
      simdEfficiency: throughput / (4 * 1000) // 相对于理论峰值的效率
    };
  }

  // ⚙️ 配置最优WASM设置
  private configureOptimalSettings(): void {
    const config = this.calculateOptimalConfig();

    // 🚀 SIMD配置
    ort.env.wasm.simd = this.simdCapabilities.hasWasmSIMD;

    // 🧵 多线程配置
    if (this.simdCapabilities.hasWasmSIMD && navigator.hardwareConcurrency >= 4) {
      ort.env.wasm.numThreads = config.optimalThreads;
      ort.env.wasm.wasmPaths = {
        'ort-wasm-simd-threaded.wasm': chrome.runtime.getURL('workers/ort-wasm-simd-threaded.wasm'),
        'ort-wasm-simd.wasm': chrome.runtime.getURL('workers/ort-wasm-simd.wasm'),
      };
    } else {
      ort.env.wasm.numThreads = 1;
      ort.env.wasm.wasmPaths = {
        'ort-wasm.wasm': chrome.runtime.getURL('workers/ort-wasm.wasm'),
      };
    }

    // 🎯 内存配置
    ort.env.wasm.initTimeout = config.initTimeout;
    ort.env.wasm.proxy = false; // Worker中禁用代理

    console.log('🚀 Optimal WASM Configuration:', {
      simd: ort.env.wasm.simd,
      threads: ort.env.wasm.numThreads,
      vectorWidth: this.simdCapabilities.vectorWidth,
      estimatedPerformance: config.estimatedPerformance
    });
  }

  // 📊 计算最优配置
  private calculateOptimalConfig(): OptimalConfig {
    const cores = navigator.hardwareConcurrency || 4;
    const memory = (navigator as any).deviceMemory || 4;

    // 🎯 基于硬件能力计算最优线程数
    let optimalThreads = 1;
    if (this.simdCapabilities.hasWasmSIMD) {
      if (cores >= 8 && memory >= 8) {
        optimalThreads = Math.min(4, Math.floor(cores / 2));
      } else if (cores >= 4 && memory >= 4) {
        optimalThreads = Math.min(2, Math.floor(cores / 2));
      } else {
        optimalThreads = 1;
      }
    }

    // 📈 估算性能提升
    const simdSpeedup = this.simdCapabilities.hasWasmSIMD ?
      (this.simdCapabilities.vectorWidth / 32) : 1; // 32位基准
    const threadSpeedup = Math.min(optimalThreads, cores * 0.8); // 80%效率
    const estimatedPerformance = simdSpeedup * threadSpeedup;

    return {
      optimalThreads,
      initTimeout: memory >= 8 ? 60000 : 30000,
      estimatedPerformance,
      recommendedBatchSize: this.simdCapabilities.hasWasmSIMD ? 8 : 4
    };
  }

  // 🔄 动态性能调优
  public async optimizeRuntime(): Promise<void> {
    // 📊 运行时性能监控
    const performanceMetrics = await this.performanceProfiler.collectMetrics();

    if (performanceMetrics.averageInferenceTime > 1000) {
      // 🚨 性能不佳，尝试调优
      console.warn('Performance below threshold, attempting optimization...');

      // 🎯 尝试减少线程数（避免上下文切换开销）
      if (ort.env.wasm.numThreads > 1) {
        ort.env.wasm.numThreads = Math.max(1, ort.env.wasm.numThreads - 1);
        console.log(`Reduced thread count to ${ort.env.wasm.numThreads}`);
      }

      // 🔧 调整内存分配策略
      if (performanceMetrics.memoryPressure > 0.8) {
        await this.optimizeMemoryUsage();
      }
    }
  }

  // 💾 内存使用优化
  private async optimizeMemoryUsage(): Promise<void> {
    // 🧹 强制垃圾回收（如果可用）
    if ('gc' in window && typeof (window as any).gc === 'function') {
      (window as any).gc();
    }

    // 📊 监控内存使用
    if ('memory' in performance) {
      const memInfo = (performance as any).memory;
      const usageRatio = memInfo.usedJSHeapSize / memInfo.jsHeapSizeLimit;

      if (usageRatio > 0.9) {
        console.warn('High memory usage detected:', {
          used: Math.round(memInfo.usedJSHeapSize / 1024 / 1024) + 'MB',
          limit: Math.round(memInfo.jsHeapSizeLimit / 1024 / 1024) + 'MB',
          ratio: Math.round(usageRatio * 100) + '%'
        });
      }
    }
  }

  // 📊 获取SIMD性能统计
  public getSIMDStats(): SIMDStats {
    return {
      capabilities: this.simdCapabilities,
      currentConfig: {
        simdEnabled: ort.env.wasm.simd,
        threadCount: ort.env.wasm.numThreads,
        wasmModule: this.simdCapabilities.hasWasmSIMD ? 'simd-threaded' : 'basic'
      },
      performanceMetrics: this.performanceProfiler.getLatestMetrics()
    };
  }
}

🚀 SIMD优化的真正价值:

优化层面	传统方法	SIMD优化	性能提升
向量运算	逐个处理	4个并行	300-400%
矩阵乘法	嵌套循环	向量化	200-300%
内存带宽	32位访问	128位访问	400%
指令效率	多条指令	单条SIMD指令	75%减少

5.2 ONNX会话与图优化

// ✅ 真正的ONNX图优化与会话管理系统
class ONNXGraphOptimizer {
  private modelAnalyzer: ModelAnalyzer;
  private graphTransformer: GraphTransformer;
  private memoryOptimizer: MemoryOptimizer;
  private executionPlanner: ExecutionPlanner;

  constructor() {
    this.modelAnalyzer = new ModelAnalyzer();
    this.graphTransformer = new GraphTransformer();
    this.memoryOptimizer = new MemoryOptimizer();
    this.executionPlanner = new ExecutionPlanner();
  }

  // 🔍 模型分析与优化策略选择
  public async analyzeAndOptimize(modelPath: string): Promise<OptimizedSessionConfig> {
    // 1. 📊 分析模型结构
    const modelInfo = await this.modelAnalyzer.analyzeModel(modelPath);

    // 2. 🎯 选择优化策略
    const optimizationStrategy = this.selectOptimizationStrategy(modelInfo);

    // 3. 🔧 应用图变换
    const transformedGraph = await this.graphTransformer.applyTransformations(
      modelInfo,
      optimizationStrategy
    );

    // 4. 💾 优化内存布局
    const memoryConfig = this.memoryOptimizer.optimizeMemoryLayout(transformedGraph);

    // 5. 📋 生成执行计划
    const executionPlan = this.executionPlanner.createExecutionPlan(
      transformedGraph,
      memoryConfig
    );

    return this.generateOptimizedConfig(modelInfo, optimizationStrategy, executionPlan);
  }
}

// 📊 模型分析器
class ModelAnalyzer {
  public async analyzeModel(modelPath: string): Promise<ModelInfo> {
    try {
      // 🔍 加载模型元数据（不加载权重）
      const modelBuffer = await this.loadModelMetadata(modelPath);
      const modelProto = this.parseONNXModel(modelBuffer);

      // 📊 分析计算图结构
      const graphAnalysis = this.analyzeComputationGraph(modelProto.graph);

      // 🎯 识别性能瓶颈
      const bottlenecks = this.identifyBottlenecks(graphAnalysis);

      // 📈 估算计算复杂度
      const complexity = this.estimateComputationalComplexity(graphAnalysis);

      return {
        modelSize: modelBuffer.byteLength,
        inputShape: this.extractInputShape(modelProto),
        outputShape: this.extractOutputShape(modelProto),
        layerCount: graphAnalysis.nodeCount,
        operatorTypes: graphAnalysis.operatorDistribution,
        memoryRequirement: complexity.memoryFootprint,
        computationalComplexity: complexity.flops,
        bottlenecks: bottlenecks,
        optimizationOpportunities: this.identifyOptimizationOpportunities(graphAnalysis)
      };
    } catch (error) {
      console.error('Model analysis failed:', error);
      throw new Error(`Failed to analyze model: ${error.message}`);
    }
  }

  // 🔍 分析计算图结构
  private analyzeComputationGraph(graph: any): GraphAnalysis {
    const analysis: GraphAnalysis = {
      nodeCount: graph.node.length,
      operatorDistribution: new Map(),
      dataFlow: [],
      memoryAccess: [],
      parallelizableOps: [],
      sequentialDependencies: []
    };

    // 📊 统计算子类型分布
    graph.node.forEach((node: any) => {
      const opType = node.opType;
      analysis.operatorDistribution.set(
        opType,
        (analysis.operatorDistribution.get(opType) || 0) + 1
      );
    });

    // 🔄 分析数据流
    analysis.dataFlow = this.traceDataFlow(graph);

    // 🎯 识别可并行化操作
    analysis.parallelizableOps = this.identifyParallelizableOperations(graph);

    // 📋 分析依赖关系
    analysis.sequentialDependencies = this.analyzeDependencies(graph);

    return analysis;
  }

  // 🎯 识别性能瓶颈
  private identifyBottlenecks(analysis: GraphAnalysis): PerformanceBottleneck[] {
    const bottlenecks: PerformanceBottleneck[] = [];

    // 🔍 内存密集型操作
    const memoryIntensiveOps = ['MatMul', 'Conv', 'Gemm'];
    memoryIntensiveOps.forEach(opType => {
      const count = analysis.operatorDistribution.get(opType) || 0;
      if (count > 0) {
        bottlenecks.push({
          type: 'memory_intensive',
          operation: opType,
          count: count,
          impact: this.estimateBottleneckImpact(opType, count),
          optimizationSuggestions: this.getOptimizationSuggestions(opType)
        });
      }
    });

    // 🔄 顺序依赖瓶颈
    if (analysis.sequentialDependencies.length > analysis.parallelizableOps.length) {
      bottlenecks.push({
        type: 'sequential_dependency',
        operation: 'data_dependency',
        count: analysis.sequentialDependencies.length,
        impact: 'high',
        optimizationSuggestions: ['graph_reordering', 'operator_fusion']
      });
    }

    return bottlenecks;
  }

  // 💡 识别优化机会
  private identifyOptimizationOpportunities(analysis: GraphAnalysis): OptimizationOpportunity[] {
    const opportunities: OptimizationOpportunity[] = [];

    // 🔗 算子融合机会
    const fusionCandidates = this.findOperatorFusionCandidates(analysis);
    if (fusionCandidates.length > 0) {
      opportunities.push({
        type: 'operator_fusion',
        description: 'Fuse consecutive operations to reduce memory access',
        candidates: fusionCandidates,
        estimatedSpeedup: 1.2 + (fusionCandidates.length * 0.1)
      });
    }

    // 📊 常量折叠机会
    const constantNodes = this.findConstantFoldingCandidates(analysis);
    if (constantNodes.length > 0) {
      opportunities.push({
        type: 'constant_folding',
        description: 'Pre-compute constant expressions',
        candidates: constantNodes,
        estimatedSpeedup: 1.1 + (constantNodes.length * 0.05)
      });
    }

    // 🎯 内存布局优化
    opportunities.push({
      type: 'memory_layout',
      description: 'Optimize tensor memory layout for cache efficiency',
      candidates: ['input_tensors', 'intermediate_tensors'],
      estimatedSpeedup: 1.15
    });

    return opportunities;
  }
}

// 🔧 图变换器
class GraphTransformer {
  public async applyTransformations(
    modelInfo: ModelInfo,
    strategy: OptimizationStrategy
  ): Promise<TransformedGraph> {
    const transformations: GraphTransformation[] = [];

    // 🔗 算子融合
    if (strategy.enableOperatorFusion) {
      transformations.push(await this.createOperatorFusionTransform(modelInfo));
    }

    // 📊 常量折叠
    if (strategy.enableConstantFolding) {
      transformations.push(await this.createConstantFoldingTransform(modelInfo));
    }

    // 🎯 死代码消除
    if (strategy.enableDeadCodeElimination) {
      transformations.push(await this.createDeadCodeEliminationTransform(modelInfo));
    }

    // 📋 图重排序
    if (strategy.enableGraphReordering) {
      transformations.push(await this.createGraphReorderingTransform(modelInfo));
    }

    return this.executeTransformations(transformations);
  }

  // 🔗 算子融合变换
  private async createOperatorFusionTransform(modelInfo: ModelInfo): Promise<GraphTransformation> {
    return {
      name: 'operator_fusion',
      description: 'Fuse consecutive operations to reduce overhead',
      apply: (graph: any) => {
        // 🎯 识别融合模式
        const fusionPatterns = [
          // Conv + BatchNorm + ReLU 融合
          {
            pattern: ['Conv', 'BatchNormalization', 'Relu'],
            fusedOp: 'ConvBatchNormRelu',
            speedup: 1.3
          },
          // MatMul + Add 融合 (Gemm)
          {
            pattern: ['MatMul', 'Add'],
            fusedOp: 'Gemm',
            speedup: 1.2
          },
          // Add + ReLU 融合
          {
            pattern: ['Add', 'Relu'],
            fusedOp: 'AddRelu',
            speedup: 1.15
          }
        ];

        let fusedCount = 0;
        fusionPatterns.forEach(pattern => {
          const matches = this.findPatternMatches(graph, pattern.pattern);
          matches.forEach(match => {
            this.fuseOperators(graph, match, pattern.fusedOp);
            fusedCount++;
          });
        });

        return {
          success: true,
          fusedOperators: fusedCount,
          estimatedSpeedup: 1 + (fusedCount * 0.1)
        };
      }
    };
  }

  // 📊 常量折叠变换
  private async createConstantFoldingTransform(modelInfo: ModelInfo): Promise<GraphTransformation> {
    return {
      name: 'constant_folding',
      description: 'Pre-compute constant expressions',
      apply: (graph: any) => {
        const constantExpressions = this.findConstantExpressions(graph);
        let foldedCount = 0;

        constantExpressions.forEach(expr => {
          try {
            // 🧮 预计算常量表达式
            const result = this.evaluateConstantExpression(expr);
            this.replaceWithConstant(graph, expr, result);
            foldedCount++;
          } catch (error) {
            console.warn('Failed to fold constant expression:', error);
          }
        });

        return {
          success: true,
          foldedExpressions: foldedCount,
          estimatedSpeedup: 1 + (foldedCount * 0.05)
        };
      }
    };
  }

  // 🎯 死代码消除变换
  private async createDeadCodeEliminationTransform(modelInfo: ModelInfo): Promise<GraphTransformation> {
    return {
      name: 'dead_code_elimination',
      description: 'Remove unused operations and tensors',
      apply: (graph: any) => {
        // 🔍 标记活跃节点（从输出开始反向遍历）
        const activeNodes = new Set<string>();
        const outputNodes = this.findOutputNodes(graph);

        this.markActiveNodes(graph, outputNodes, activeNodes);

        // 🧹 移除死代码
        const originalNodeCount = graph.node.length;
        graph.node = graph.node.filter((node: any) => activeNodes.has(node.name));
        const removedCount = originalNodeCount - graph.node.length;

        return {
          success: true,
          removedNodes: removedCount,
          estimatedSpeedup: 1 + (removedCount * 0.02)
        };
      }
    };
  }
}

// 💾 内存优化器
class MemoryOptimizer {
  public optimizeMemoryLayout(graph: TransformedGraph): MemoryConfig {
    // 🎯 分析内存访问模式
    const memoryAccess = this.analyzeMemoryAccessPattern(graph);

    // 📊 计算最优内存布局
    const layout = this.calculateOptimalLayout(memoryAccess);

    // 🔄 生成内存重用策略
    const reuseStrategy = this.generateMemoryReuseStrategy(graph, layout);

    return {
      tensorLayout: layout,
      memoryReuse: reuseStrategy,
      arenaSize: this.calculateArenaSize(layout),
      alignmentRequirements: this.getAlignmentRequirements(),
      prefetchStrategy: this.generatePrefetchStrategy(memoryAccess)
    };
  }

  // 📊 分析内存访问模式
  private analyzeMemoryAccessPattern(graph: TransformedGraph): MemoryAccessPattern {
    const pattern: MemoryAccessPattern = {
      tensorLifetime: new Map(),
      accessFrequency: new Map(),
      accessOrder: [],
      memoryPressurePoints: []
    };

    // 🔍 分析张量生命周期
    graph.nodes.forEach((node, index) => {
      node.inputs.forEach(input => {
        if (!pattern.tensorLifetime.has(input)) {
          pattern.tensorLifetime.set(input, { birth: index, death: index });
        } else {
          pattern.tensorLifetime.get(input)!.death = index;
        }
      });

      node.outputs.forEach(output => {
        pattern.tensorLifetime.set(output, { birth: index, death: index });
      });
    });

    // 📈 计算访问频率
    graph.nodes.forEach(node => {
      [...node.inputs, ...node.outputs].forEach(tensor => {
        pattern.accessFrequency.set(
          tensor,
          (pattern.accessFrequency.get(tensor) || 0) + 1
        );
      });
    });

    return pattern;
  }

  // 🎯 计算最优内存布局
  private calculateOptimalLayout(accessPattern: MemoryAccessPattern): TensorLayout {
    const layout: TensorLayout = {
      tensorOrder: [],
      memoryOffsets: new Map(),
      alignmentPadding: new Map(),
      cacheOptimization: new Map()
    };

    // 🔄 按生命周期排序张量
    const sortedTensors = Array.from(accessPattern.tensorLifetime.entries())
      .sort((a, b) => {
        const lifetimeA = a[1].death - a[1].birth;
        const lifetimeB = b[1].death - b[1].birth;
        return lifetimeA - lifetimeB; // 短生命周期优先
      });

    // 📊 分配内存偏移
    let currentOffset = 0;
    sortedTensors.forEach(([tensorName, lifetime]) => {
      // 🎯 计算对齐要求
      const alignment = this.calculateTensorAlignment(tensorName);
      const alignedOffset = this.alignOffset(currentOffset, alignment);
      const padding = alignedOffset - currentOffset;

      layout.tensorOrder.push(tensorName);
      layout.memoryOffsets.set(tensorName, alignedOffset);
      layout.alignmentPadding.set(tensorName, padding);

      // 📈 缓存优化提示
      const accessFreq = accessPattern.accessFrequency.get(tensorName) || 0;
      layout.cacheOptimization.set(tensorName, {
        preferL1: accessFreq > 10,
        preferL2: accessFreq > 5,
        prefetchDistance: Math.min(accessFreq, 8)
      });

      currentOffset = alignedOffset + this.estimateTensorSize(tensorName);
    });

    return layout;
  }
}

// 📋 执行计划器
class ExecutionPlanner {
  public createExecutionPlan(
    graph: TransformedGraph,
    memoryConfig: MemoryConfig
  ): ExecutionPlan {
    // 🎯 分析并行化机会
    const parallelGroups = this.identifyParallelExecutionGroups(graph);

    // 📊 生成执行调度
    const schedule = this.generateExecutionSchedule(graph, parallelGroups);

    // 🔄 优化数据流
    const dataFlow = this.optimizeDataFlow(graph, memoryConfig);

    return {
      executionOrder: schedule.executionOrder,
      parallelGroups: parallelGroups,
      memorySchedule: schedule.memorySchedule,
      dataFlowOptimization: dataFlow,
      estimatedExecutionTime: this.estimateExecutionTime(schedule)
    };
  }

  // 🎯 识别并行执行组
  private identifyParallelExecutionGroups(graph: TransformedGraph): ParallelGroup[] {
    const groups: ParallelGroup[] = [];
    const processed = new Set<string>();

    graph.nodes.forEach(node => {
      if (processed.has(node.name)) return;

      // 🔍 查找可以并行执行的节点
      const parallelNodes = this.findParallelizableNodes(graph, node, processed);

      if (parallelNodes.length > 1) {
        groups.push({
          nodes: parallelNodes,
          estimatedSpeedup: Math.min(parallelNodes.length, navigator.hardwareConcurrency),
          memoryRequirement: this.calculateGroupMemoryRequirement(parallelNodes)
        });

        parallelNodes.forEach(n => processed.add(n.name));
      } else {
        processed.add(node.name);
      }
    });

    return groups;
  }
}

// 🎯 生成最优会话配置
private generateOptimizedConfig(
  modelInfo: ModelInfo,
  strategy: OptimizationStrategy,
  executionPlan: ExecutionPlan
): OptimizedSessionConfig {
  const config: OptimizedSessionConfig = {
    // 🚀 基础配置
    executionProviders: this.selectOptimalProviders(modelInfo),

    // 📊 图优化配置
    graphOptimizationLevel: strategy.aggressiveOptimization ? 'all' : 'extended',

    // 💾 内存配置
    enableCpuMemArena: true,
    enableMemPattern: true,
    memArenaExtendStrategy: 'kSameAsRequested',

    // 🧵 并行配置
    executionMode: executionPlan.parallelGroups.length > 0 ? 'parallel' : 'sequential',

    // 🎯 高级优化配置
    sessionConfigEntries: {
      // 线程配置
      'session.intra_op_num_threads': this.calculateOptimalIntraOpThreads(modelInfo).toString(),
      'session.inter_op_num_threads': this.calculateOptimalInterOpThreads(executionPlan).toString(),

      // 内存优化
      'session.memory.enable_memory_arena_shrinkage': 'true',
      'session.memory.memory_arena_extend_strategy': '1',

      // 图优化
      'session.graph_optimization_level': strategy.aggressiveOptimization ? '99' : '1',
      'session.enable_cpu_mem_arena': 'true',
      'session.enable_mem_pattern': 'true',

      // SIMD优化
      'session.use_ort_model_bytes_directly': 'true',
      'session.use_ort_model_bytes_for_initializers': 'true',

      // 调试配置
      'session.log_severity_level': '3',
      'session.log_verbosity_level': '0'
    },

    // 📈 性能预期
    estimatedPerformance: {
      speedupFactor: this.calculateExpectedSpeedup(strategy, executionPlan),
      memoryReduction: this.calculateExpectedMemoryReduction(strategy),
      optimizationApplied: this.getAppliedOptimizations(strategy)
    }
  };

  return config;
}

🚀 ONNX图优化的真正价值:

优化技术	原理	性能提升	内存节省
算子融合	合并连续操作减少开销	20-30%	15-25%
常量折叠	预计算常量表达式	10-15%	5-10%
死代码消除	移除未使用的操作	5-10%	10-20%
内存布局优化	优化张量内存排列	15-20%	20-30%
并行化调度	识别并行执行机会	30-50%	0%

5.3 💾 智能缓存与内存优化

5.3.1 🔄 张量内存池设计

为了进一步优化内存使用效率，还可以实现一个专门的张量内存池，用于复用ONNX Runtime推理过程中频繁创建和销毁的张量对象：

// 高效的张量内存池实现
class TensorMemoryPool {
  private pools: Map<string, ort.Tensor[]> = new Map();
  private maxPoolSize: number = 50; // 每个池的最大大小
  private totalTensors: number = 0;
  private maxTotalTensors: number = 200; // 全局张量数量限制

  /**
   * 获取指定形状和类型的张量
   * @param shape 张量形状
   * @param dtype 数据类型
   * @returns 可复用的张量对象
   */
  getTensor(shape: number[], dtype: ort.Tensor.DataType): ort.Tensor {
    const key = this.generatePoolKey(shape, dtype);

    if (!this.pools.has(key)) {
      this.pools.set(key, []);
    }

    const pool = this.pools.get(key)!;

    // 从池中获取现有张量
    if (pool.length > 0) {
      const tensor = pool.pop()!;
      this.clearTensorData(tensor); // 清理数据但保留结构
      return tensor;
    }

    // 检查全局张量数量限制
    if (this.totalTensors >= this.maxTotalTensors) {
      this.cleanupOldestPool();
    }

    // 创建新张量
    const tensor = new ort.Tensor(dtype, new Float32Array(this.calculateSize(shape)), shape);
    this.totalTensors++;

    return tensor;
  }

  /**
   * 回收张量到内存池
   * @param tensor 要回收的张量
   */
  recycleTensor(tensor: ort.Tensor): void {
    if (!tensor || tensor.size === 0) return;

    const key = this.generatePoolKey(tensor.dims, tensor.type);

    if (!this.pools.has(key)) {
      this.pools.set(key, []);
    }

    const pool = this.pools.get(key)!;

    // 检查池大小限制
    if (pool.length < this.maxPoolSize) {
      pool.push(tensor);
    } else {
      // 池已满，直接释放张量
      this.disposeTensor(tensor);
    }
  }

  /**
   * 生成内存池键值
   */
  private generatePoolKey(shape: number[], dtype: ort.Tensor.DataType): string {
    return `${shape.join('x')}_${dtype}`;
  }

  /**
   * 计算张量大小
   */
  private calculateSize(shape: number[]): number {
    return shape.reduce((acc, dim) => acc * dim, 1);
  }

  /**
   * 清理张量数据但保留结构
   */
  private clearTensorData(tensor: ort.Tensor): void {
    if (tensor.data instanceof Float32Array) {
      tensor.data.fill(0);
    }
  }

  /**
   * 清理最旧的内存池
   */
  private cleanupOldestPool(): void {
    const poolKeys = Array.from(this.pools.keys());
    if (poolKeys.length > 0) {
      const oldestKey = poolKeys[0];
      const pool = this.pools.get(oldestKey)!;

      // 释放池中所有张量
      pool.forEach(tensor => this.disposeTensor(tensor));
      this.pools.delete(oldestKey);
    }
  }

  /**
   * 安全释放张量
   */
  private disposeTensor(tensor: ort.Tensor): void {
    try {
      if (tensor && typeof tensor.dispose === 'function') {
        tensor.dispose();
        this.totalTensors--;
      }
    } catch (error) {
      console.warn('Failed to dispose tensor:', error);
    }
  }

  /**
   * 获取内存池统计信息
   */
  getStats(): {
    poolCount: number;
    totalTensors: number;
    poolSizes: Record<string, number>;
  } {
    const poolSizes: Record<string, number> = {};
    this.pools.forEach((pool, key) => {
      poolSizes[key] = pool.length;
    });

    return {
      poolCount: this.pools.size,
      totalTensors: this.totalTensors,
      poolSizes
    };
  }

  /**
   * 清理所有内存池
   */
  dispose(): void {
    this.pools.forEach(pool => {
      pool.forEach(tensor => this.disposeTensor(tensor));
    });
    this.pools.clear();
    this.totalTensors = 0;
  }
}

在Worker中集成张量内存池:

// Worker中的张量内存池使用
class OptimizedInferenceEngine {
  private tensorPool: TensorMemoryPool;

  constructor() {
    this.tensorPool = new TensorMemoryPool();
  }

  async runInference(inputData: number[], shape: number[]): Promise<Float32Array> {
    // 从内存池获取输入张量
    const inputTensor = this.tensorPool.getTensor(shape, 'float32');

    try {
      // 填充输入数据
      (inputTensor.data as Float32Array).set(inputData);

      // 执行推理
      const results = await this.session.run({ input: inputTensor });
      const outputTensor = results.output;

      // 提取结果数据
      const outputData = new Float32Array(outputTensor.data as Float32Array);

      return outputData;
    } finally {
      // 回收张量到内存池
      this.tensorPool.recycleTensor(inputTensor);
    }
  }

  // 定期清理和监控
  performMaintenance(): void {
    const stats = this.tensorPool.getStats();
    console.log('Tensor Pool Stats:', stats);

    // 如果内存使用过高，主动清理
    if (stats.totalTensors > 150) {
      this.tensorPool.dispose();
      console.log('Tensor pool cleaned due to high memory usage');
    }
  }
}

5.3.2 动态批处理 (Dynamic Batching)

引擎支持将多个独立的文本向量化请求动态组合成批次，一次性提交给模型进行推理：

// 批处理简单示例
class BatchProcessor {
  private batchQueue: BatchItem[] = [];
  private batchTimer: NodeJS.Timeout | null = null;
  private readonly maxBatchSize = 8;
  private readonly batchTimeout = 50; // 50ms批处理窗口

  async addToBatch(text: string, options: any): Promise<Float32Array> {
    return new Promise((resolve, reject) => {
      this.batchQueue.push({ text, options, resolve, reject });

      // 达到批次大小或设置定时器
      if (this.batchQueue.length >= this.maxBatchSize) {
        this.processBatch();
      } else if (!this.batchTimer) {
        this.batchTimer = setTimeout(() => this.processBatch(), this.batchTimeout);
      }
    });
  }

  private async processBatch(): Promise<void> {
    if (this.batchTimer) {
      clearTimeout(this.batchTimer);
      this.batchTimer = null;
    }

    const currentBatch = this.batchQueue.splice(0, this.maxBatchSize);
    if (currentBatch.length === 0) return;

    try {
      const texts = currentBatch.map(item => item.text);
      const embeddings = await this.computeBatchEmbeddings(texts);

      // 分发结果给各个Promise
      currentBatch.forEach((item, index) => {
        item.resolve(embeddings[index]);
      });
    } catch (error) {
      // 批处理失败，逐个处理
      currentBatch.forEach(item => item.reject(error));
    }
  }
}

批处理优势:

• 更好地利用模型并行计算的潜力

• 降低单次推理的平均开销

• 在处理多个文本对或构建相似度矩阵等场景下效果显著

5.4 模型预热

public async warmupModel(): Promise<void> {
  console.log('SemanticSimilarityEngine: Warming up model...');
  const warmupTexts = ['Hello world', '你好世界'];

  try {
    for (const text of warmupTexts) {
      await this.getEmbedding(text);
    }
    // 清理预热数据，避免污染缓存
    this.embeddingCache.clear();
    this.cacheStats = { hits: 0, misses: 0, size: 0 };
  } catch (error) {
    console.warn('Model warmup failed, but this might not be critical.', error);
  }
}

6. 💡 实践总结

6.1 🏗️ 设计原则

1. 🔄 职责分离: Background Script负责调度，Worker负责计算，Content Script负责交互
2. ⚡ 异步优先: 所有AI计算异步执行，避免阻塞用户界面
3. 🛡️ 资源控制: 严格的并发限制和内存监控
4. 🚧 错误隔离: Worker崩溃不影响扩展核心功能

6.2 🚀 性能优化策略

1. 💾 智能缓存: LRU策略，基于使用频率和内存压力动态调整
2. 📦 批处理优化: 合并相似请求，减少Worker通信开销
3. 🔥 预热机制: 应用启动时预热模型，消除首次调用延迟
4. ⚡ WASM加速: 充分利用SIMD指令集和多线程能力

6.3 🛠️ 工程化建议

1. 📈 渐进式加载: 根据用户需求动态加载模型组件
2. 📝 版本管理: 模型文件版本控制和增量更新机制
3. 📊 监控体系: 完善的性能监控和错误上报
4. 🔄 降级方案: 多层降级策略，保证基础功能可用

6.4 🔍 开发调试技巧

1. 📊 性能分析: 使用Chrome DevTools分析Worker性能
2. 💾 内存监控: 定期检查内存使用情况，防止内存泄漏
3. 🐛 错误追踪: 完善的日志系统和错误堆栈追踪
4. 🧪 A/B测试: 不同优化策略的效果对比验证