langchain之function calling

Function calling

OpenAI中叫作function calling，在langchain中被称为tool calling，用于连接模型和外部数据系统。

function calling是一个过程（function + calling），首先定义一系列函数即function，之后模型基于对话内容，判断是否需要使用定义好的函数calling，手动执行函数，将函数返回的结果提供给模型，模型输出结果。 函数可以有多种形式和用途，常见的如

• fetch data，如检索数据
• take actions,如根据邮件内容触发记录代办、安排会议等
• build rich workflow，如数据提取、数据标注
• intercte with application ui,如根据输入内容，更新页面

OpenAI官网提供的function calling流程图

function definition

function definition目的是向模型描述需要使用的函数，包括函数的用途和执行函数需要的入参，格式使用JSON Schema，模型会根据提供的schema生成调用function时所需的参数。

// 分为三部分，函数名、函数描述、函数入参
{
    "name": "get_delivery_date",
    "description": "Get the delivery date for a customer's order. Call this whenever you need to know the delivery date, for example when a customer asks 'Where is my package'",
    // 入参中包含类型、是否可选等内容
    "parameters": {
        "type": "object",
        "properties": {
            "order_id": {
                "type": "string",
                "description": "The customer's order ID."
            }
        },
        "required": ["order_id"],
        "additionalProperties": false
    }
}

langchain中可以使用zod库进行类型定义，描述function的params，使用tool定义function

import { z } from 'zod';
import { tool } from '@langchain/core/tools';

const calculatorSchema = z.object({
  operation: z
    .enum(['add', 'subtract', 'multiply', 'divide'])
    .describe('The type of operation to execute.'),
  number1: z.number().describe('The first number to operate on.'),
  number2: z.number().describe('The second number to operate on.'),
});

const calculatorTool = tool(
  async ({ operation, number1, number2 }) => {
    // Functions must return strings
    if (operation === 'add') {
      return `${number1 + number2}`;
    } else if (operation === 'subtract') {
      return `${number1 - number2}`;
    } else if (operation === 'multiply') {
      return `${number1 * number2}`;
    } else if (operation === 'divide') {
      return `${number1 / number2}`;
    } else {
      throw new Error('Invalid operation.');
    }
  },
  {
    name: 'calculator',
    description: 'Can perform mathematical operations.',
    schema: calculatorSchema,
  }
);

function bind

langchain中使用bindTools将function和模型进行绑定，用于明确哪些function对于模型来说，是可以访问的。

import { ChatOllama } from '@langchain/ollama';
const model = new ChatOllama({
    baseUrl: 'http://localhost:11434',
    model: 'llama3.1',
});
const modelWithTools = model.bindTools([calculatorTool]);

function call

是否决定调用function是由模型根据对话内容决定的，会有不需要调用的情况，区分是否需要调用，可以根据模型返回数据的result来判断，需要调用的情况下，result会有tool_calls变量。

const messages = [new HumanMessage('What is 2 + 2? and 3 - 1?')];
const aiMessage = await modelWithTools.invoke(messages);
console.log(aiMessage);
// 
AIMessage {
  "content": "",
  "additional_kwargs": {},
  "response_metadata": {...},
  "tool_calls": [
    {
      "name": "calculator",
      "args": {
        "number1": 2,
        "number2": 2,
        "operation": "add"
      },
      "id": "dc6acf31-cf47-4467-9ecc-c203c12a2270",
      "type": "tool_call"
    },
    {
      "name": "calculator",
      "args": {
        "number1": 3,
        "number2": 1,
        "operation": "subtract"
      },
      "id": "2277c37a-8ff2-41e3-b365-a3eb15051559",
      "type": "tool_call"
    }
  ],
  "invalid_tool_calls": [],
  "usage_metadata": {...}
}

function execution

根据模型返回的tool_calls内容，执行并获取function返回结果，并将返回结果传递给模型。

messages.push(aiMessage);
for (const toolCall of aiMessage.tool_calls) {
    const toolMessage = await calculatorTool.invoke(toolCall);
    // 将toolMessage收录到对话历史中
    messages.push(toolMessage);
}
// tool.invoke后会生成toolMessage，content中包含执行结果
ToolMessage {
    "content": "4",
    "name": "calculator",
    "additional_kwargs": {},
    "response_metadata": {},
    "tool_call_id": "dc6acf31-cf47-4467-9ecc-c203c12a2270"
  },
  ToolMessage {
    "content": "2",
    "name": "calculator",
    "additional_kwargs": {},
    "response_metadata": {},
    "tool_call_id": "2277c37a-8ff2-41e3-b365-a3eb15051559"
  }

// 进行最后的调用
const res = await modelWithTools.invoke(messages);
console.log(res);
// 
AIMessage {
  "content": "The output of the ipython tool call shows that 2 + 2 = 4, and 3 - 1 = 2. Therefore, the answer to your question is:\n\nYes, you are correct that 2 + 2 = 4 and 3 - 1 = 2.",
  "additional_kwargs": {},
  "response_metadata": {。。。},
  "tool_calls": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 86,
    "output_tokens": 63,
    "total_tokens": 149
  }

TODO more