examples
1748 строк · 56.7 Кб
1{2"cells": [3{4"cell_type": "markdown",5"id": "cc93d05f",6"metadata": {7"id": "cc93d05f"8},9"source": [10"[](https://colab.research.google.com/github/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/03-langchain-conversational-memory.ipynb) [](https://nbviewer.org/github/pinecone-io/examples/blob/master/learn/generation/langchain/handbook/03-langchain-conversational-memory.ipynb)"11]12},13{14"cell_type": "markdown",15"id": "hcqKO0aI6_PI",16"metadata": {17"id": "hcqKO0aI6_PI"18},19"source": [20"#### [LangChain Handbook](https://pinecone.io/learn/langchain)\n",21"\n",22"# Conversational Memory\n",23"\n",24"Conversational memory is how chatbots can respond to our queries in a chat-like manner. It enables a coherent conversation, and without it, every query would be treated as an entirely independent input without considering past interactions.\n",25"\n",26"The memory allows a _\"agent\"_ to remember previous interactions with the user. By default, agents are *stateless* — meaning each incoming query is processed independently of other interactions. The only thing that exists for a stateless agent is the current input, nothing else.\n",27"\n",28"There are many applications where remembering previous interactions is very important, such as chatbots. Conversational memory allows us to do that.\n",29"\n",30"In this notebook we'll explore this form of memory in the context of the LangChain library.\n",31"\n",32"We'll start by importing all of the libraries that we'll be using in this example."33]34},35{36"cell_type": "code",37"execution_count": 1,38"id": "uZR3iGJJtdDE",39"metadata": {40"id": "uZR3iGJJtdDE",41"colab": {42"base_uri": "https://localhost:8080/"43},44"outputId": "98873b1a-5688-4f64-c400-e17be707c56b"45},46"outputs": [47{48"output_type": "stream",49"name": "stdout",50"text": [51"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m344.0/344.0 KB\u001b[0m \u001b[31m6.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",52"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m70.1/70.1 KB\u001b[0m \u001b[31m3.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",53"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.6/1.6 MB\u001b[0m \u001b[31m41.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",54"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m73.5/73.5 KB\u001b[0m \u001b[31m785.2 kB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",55"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m62.8/62.8 KB\u001b[0m \u001b[31m2.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",56"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.1/2.1 MB\u001b[0m \u001b[31m38.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",57"\u001b[?25h"58]59}60],61"source": [62"!pip install -qU langchain openai tiktoken"63]64},65{66"cell_type": "code",67"execution_count": 2,68"id": "66fb9c2a",69"metadata": {70"id": "66fb9c2a"71},72"outputs": [],73"source": [74"import inspect\n",75"\n",76"from getpass import getpass\n",77"from langchain import OpenAI\n",78"from langchain.chains import LLMChain, ConversationChain\n",79"from langchain.chains.conversation.memory import (ConversationBufferMemory, \n",80" ConversationSummaryMemory, \n",81" ConversationBufferWindowMemory,\n",82" ConversationKGMemory)\n",83"from langchain.callbacks import get_openai_callback\n",84"import tiktoken"85]86},87{88"cell_type": "markdown",89"id": "wPdWz1IdxyBR",90"metadata": {91"id": "wPdWz1IdxyBR"92},93"source": [94"To run this notebook, we will need to use an OpenAI LLM. Here we will setup the LLM we will use for the whole notebook, just input your openai api key when prompted. "95]96},97{98"cell_type": "code",99"execution_count": 3,100"id": "c02c4fa2",101"metadata": {102"colab": {103"base_uri": "https://localhost:8080/"104},105"id": "c02c4fa2",106"outputId": "ed941db8-a50d-4e7d-d302-7b6b8c371c25"107},108"outputs": [109{110"name": "stdout",111"output_type": "stream",112"text": [113"··········\n"114]115}116],117"source": [118"OPENAI_API_KEY = getpass()"119]120},121{122"cell_type": "code",123"execution_count": 25,124"id": "baaa74b8",125"metadata": {126"id": "baaa74b8"127},128"outputs": [],129"source": [130"llm = OpenAI(\n",131" temperature=0, \n",132" openai_api_key=OPENAI_API_KEY,\n",133" model_name='text-davinci-003' # can be used with llms like 'gpt-3.5-turbo'\n",134")"135]136},137{138"cell_type": "markdown",139"id": "309g_2pqxzzB",140"metadata": {141"id": "309g_2pqxzzB"142},143"source": [144"Later we will make use of a `count_tokens` utility function. This will allow us to count the number of tokens we are using for each call. We define it as so:"145]146},147{148"cell_type": "code",149"execution_count": 26,150"id": "DsC3szr6yP3L",151"metadata": {152"id": "DsC3szr6yP3L"153},154"outputs": [],155"source": [156"def count_tokens(chain, query):\n",157" with get_openai_callback() as cb:\n",158" result = chain.run(query)\n",159" print(f'Spent a total of {cb.total_tokens} tokens')\n",160"\n",161" return result"162]163},164{165"cell_type": "markdown",166"id": "CnNF6i9r8RY_",167"metadata": {168"id": "CnNF6i9r8RY_"169},170"source": [171"Now let's dive into **Conversational Memory**."172]173},174{175"cell_type": "markdown",176"id": "6e1f31b4",177"metadata": {178"id": "6e1f31b4"179},180"source": [181"## What is memory?"182]183},184{185"cell_type": "markdown",186"id": "5b919c3a",187"metadata": {188"id": "5b919c3a"189},190"source": [191"**Definition**: Memory is an agent's capacity of remembering previous interactions with the user (think chatbots)\n",192"\n",193"The official definition of memory is the following:\n",194"\n",195"\n",196"> By default, Chains and Agents are stateless, meaning that they treat each incoming query independently. In some applications (chatbots being a GREAT example) it is highly important to remember previous interactions, both at a short term but also at a long term level. The concept of “Memory” exists to do exactly that.\n",197"\n",198"\n",199"As we will see, although this sounds really straightforward there are several different ways to implement this memory capability."200]201},202{203"cell_type": "markdown",204"id": "3343a0e2",205"metadata": {206"id": "3343a0e2"207},208"source": [209"Before we delve into the different memory modules that the library offers, we will introduce the chain we will be using for these examples: the `ConversationChain`."210]211},212{213"cell_type": "markdown",214"id": "6c9c13e9",215"metadata": {216"id": "6c9c13e9"217},218"source": [219"As always, when understanding a chain it is interesting to peek into its prompt first and then take a look at its `._call` method. As we saw in the chapter on chains, we can check out the prompt by accessing the `template` within the `prompt` attribute."220]221},222{223"cell_type": "code",224"execution_count": 27,225"id": "96ff1ce3",226"metadata": {227"id": "96ff1ce3"228},229"outputs": [],230"source": [231"conversation = ConversationChain(\n",232" llm=llm, \n",233")"234]235},236{237"cell_type": "code",238"execution_count": 28,239"id": "90ad394d",240"metadata": {241"colab": {242"base_uri": "https://localhost:8080/"243},244"id": "90ad394d",245"outputId": "1c641d37-b3e7-40d5-815b-936fcd2d9a2a"246},247"outputs": [248{249"output_type": "stream",250"name": "stdout",251"text": [252"The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\n",253"\n",254"Current conversation:\n",255"{history}\n",256"Human: {input}\n",257"AI:\n"258]259}260],261"source": [262"print(conversation.prompt.template)"263]264},265{266"cell_type": "markdown",267"id": "9f8b1e0c",268"metadata": {269"id": "9f8b1e0c"270},271"source": [272"Interesting! So this chain's prompt is telling it to chat with the user and try to give truthful answers. If we look closely, there is a new component in the prompt that we didn't see when we were tinkering with the `LLMMathChain`: _history_. This is where our memory will come into play."273]274},275{276"cell_type": "markdown",277"id": "4a7e7770",278"metadata": {279"id": "4a7e7770"280},281"source": [282"What is this chain doing with this prompt? Let's take a look."283]284},285{286"cell_type": "code",287"execution_count": 29,288"id": "43bfd2da",289"metadata": {290"colab": {291"base_uri": "https://localhost:8080/"292},293"id": "43bfd2da",294"outputId": "489437a5-0f0b-412a-f817-f0df817211c2"295},296"outputs": [297{298"output_type": "stream",299"name": "stdout",300"text": [301" def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:\n",302" known_values = self.prep_inputs(inputs.copy())\n",303" return self.apply([known_values])[0]\n",304" def apply(self, input_list: List[Dict[str, Any]]) -> List[Dict[str, str]]:\n",305" \"\"\"Utilize the LLM generate method for speed gains.\"\"\"\n",306" response = self.generate(input_list)\n",307" return self.create_outputs(response)\n",308"\n"309]310}311],312"source": [313"print(inspect.getsource(conversation._call), inspect.getsource(conversation.apply))"314]315},316{317"cell_type": "markdown",318"id": "84e664af",319"metadata": {320"id": "84e664af"321},322"source": [323"Nothing really magical going on here, just a straightforward pass through an LLM. In fact, this chain inherits these methods directly from the `LLMChain` without any modification:"324]325},326{327"cell_type": "code",328"execution_count": 30,329"id": "d8f4aa79",330"metadata": {331"colab": {332"base_uri": "https://localhost:8080/"333},334"id": "d8f4aa79",335"outputId": "ca3413ec-1ceb-4160-f6e9-2031350780a0"336},337"outputs": [338{339"output_type": "stream",340"name": "stdout",341"text": [342" def _call(self, inputs: Dict[str, Any]) -> Dict[str, str]:\n",343" known_values = self.prep_inputs(inputs.copy())\n",344" return self.apply([known_values])[0]\n",345" def apply(self, input_list: List[Dict[str, Any]]) -> List[Dict[str, str]]:\n",346" \"\"\"Utilize the LLM generate method for speed gains.\"\"\"\n",347" response = self.generate(input_list)\n",348" return self.create_outputs(response)\n",349"\n"350]351}352],353"source": [354"print(inspect.getsource(LLMChain._call), inspect.getsource(LLMChain.apply))"355]356},357{358"cell_type": "markdown",359"id": "6aaa70bf",360"metadata": {361"id": "6aaa70bf"362},363"source": [364"So basically this chain combines an input from the user with the conversation history to generate a meaningful (and hopefully truthful) response."365]366},367{368"cell_type": "markdown",369"id": "19f5172f",370"metadata": {371"id": "19f5172f"372},373"source": [374"Now that we've understood the basics of the chain we'll be using, we can get into memory. Let's dive in!"375]376},377{378"cell_type": "markdown",379"id": "0f1a33f6",380"metadata": {381"id": "0f1a33f6"382},383"source": [384"## Memory types"385]386},387{388"cell_type": "markdown",389"id": "4d732b7a",390"metadata": {391"id": "4d732b7a"392},393"source": [394"In this section we will review several memory types and analyze the pros and cons of each one, so you can choose the best one for your use case."395]396},397{398"cell_type": "markdown",399"id": "04d70642",400"metadata": {401"id": "04d70642"402},403"source": [404"### Memory type #1: ConversationBufferMemory"405]406},407{408"cell_type": "markdown",409"id": "53d3cb2b",410"metadata": {411"id": "53d3cb2b"412},413"source": [414"The `ConversationBufferMemory` does just what its name suggests: it keeps a buffer of the previous conversation excerpts as part of the context in the prompt."415]416},417{418"cell_type": "markdown",419"id": "d80a974a",420"metadata": {421"id": "d80a974a"422},423"source": [424"**Key feature:** _the conversation buffer memory keeps the previous pieces of conversation completely unmodified, in their raw form._"425]426},427{428"cell_type": "code",429"execution_count": 31,430"id": "2267f1f0",431"metadata": {432"id": "2267f1f0"433},434"outputs": [],435"source": [436"conversation_buf = ConversationChain(\n",437" llm=llm,\n",438" memory=ConversationBufferMemory()\n",439")"440]441},442{443"cell_type": "markdown",444"id": "lseziAMcAyvX",445"metadata": {446"id": "lseziAMcAyvX"447},448"source": [449"We pass a user prompt the the `ConversationBufferMemory` like so:"450]451},452{453"cell_type": "code",454"execution_count": 32,455"id": "M0cwooC5A5Id",456"metadata": {457"colab": {458"base_uri": "https://localhost:8080/"459},460"id": "M0cwooC5A5Id",461"outputId": "8a8178eb-b9ac-45cf-baed-255b413b0630"462},463"outputs": [464{465"output_type": "execute_result",466"data": {467"text/plain": [468"{'input': 'Good morning AI!',\n",469" 'history': '',\n",470" 'response': \" Good morning! It's a beautiful day today, isn't it? How can I help you?\"}"471]472},473"metadata": {},474"execution_count": 32475}476],477"source": [478"conversation_buf(\"Good morning AI!\")"479]480},481{482"cell_type": "markdown",483"id": "xlKINTFYA9eo",484"metadata": {485"id": "xlKINTFYA9eo"486},487"source": [488"This one call used a total of `85` tokens, but we can't see that from the above. If we'd like to count the number of tokens being used we just pass our conversation chain object and the message we'd like to input via the `count_tokens` function we defined earlier:"489]490},491{492"cell_type": "code",493"execution_count": 33,494"id": "d1bd5a88",495"metadata": {496"colab": {497"base_uri": "https://localhost:8080/",498"height": 89499},500"id": "d1bd5a88",501"outputId": "cb593afd-7efd-4c0e-cf04-82dc1a324aff"502},503"outputs": [504{505"output_type": "stream",506"name": "stdout",507"text": [508"Spent a total of 179 tokens\n"509]510},511{512"output_type": "execute_result",513"data": {514"text/plain": [515"' Interesting! Large Language Models are a type of artificial intelligence that can process natural language and generate text. They can be used to generate text from a given context, or to answer questions about a given context. Integrating them with external knowledge can help them to better understand the context and generate more accurate results. Is there anything else I can help you with?'"516],517"application/vnd.google.colaboratory.intrinsic+json": {518"type": "string"519}520},521"metadata": {},522"execution_count": 33523}524],525"source": [526"count_tokens(\n",527" conversation_buf, \n",528" \"My interest here is to explore the potential of integrating Large Language Models with external knowledge\"\n",529")"530]531},532{533"cell_type": "code",534"execution_count": 34,535"id": "146170ca",536"metadata": {537"colab": {538"base_uri": "https://localhost:8080/",539"height": 89540},541"id": "146170ca",542"outputId": "dbb6f78c-b169-463e-c1c8-a35151894f56"543},544"outputs": [545{546"output_type": "stream",547"name": "stdout",548"text": [549"Spent a total of 268 tokens\n"550]551},552{553"output_type": "execute_result",554"data": {555"text/plain": [556"' Well, integrating Large Language Models with external knowledge can open up a lot of possibilities. For example, you could use them to generate more accurate and detailed summaries of text, or to answer questions about a given context more accurately. You could also use them to generate more accurate translations, or to generate more accurate predictions about future events.'"557],558"application/vnd.google.colaboratory.intrinsic+json": {559"type": "string"560}561},562"metadata": {},563"execution_count": 34564}565],566"source": [567"count_tokens(\n",568" conversation_buf,\n",569" \"I just want to analyze the different possibilities. What can you think of?\"\n",570")"571]572},573{574"cell_type": "code",575"execution_count": 35,576"id": "3e15411a",577"metadata": {578"colab": {579"base_uri": "https://localhost:8080/",580"height": 89581},582"id": "3e15411a",583"outputId": "f6857844-ee6f-49ef-df50-54335f248bd3"584},585"outputs": [586{587"output_type": "stream",588"name": "stdout",589"text": [590"Spent a total of 360 tokens\n"591]592},593{594"output_type": "execute_result",595"data": {596"text/plain": [597"' There are a variety of data sources that could be used to give context to a Large Language Model. These include structured data sources such as databases, unstructured data sources such as text documents, and even audio and video data sources. Additionally, you could use external knowledge sources such as Wikipedia or other online encyclopedias to provide additional context.'"598],599"application/vnd.google.colaboratory.intrinsic+json": {600"type": "string"601}602},603"metadata": {},604"execution_count": 35605}606],607"source": [608"count_tokens(\n",609" conversation_buf, \n",610" \"Which data source types could be used to give context to the model?\"\n",611")"612]613},614{615"cell_type": "code",616"execution_count": 36,617"id": "3352cc48",618"metadata": {619"colab": {620"base_uri": "https://localhost:8080/",621"height": 53622},623"id": "3352cc48",624"outputId": "62294954-cc7e-4ef3-e5fc-19a5c4ffc4c1"625},626"outputs": [627{628"output_type": "stream",629"name": "stdout",630"text": [631"Spent a total of 388 tokens\n"632]633},634{635"output_type": "execute_result",636"data": {637"text/plain": [638"' Your aim is to explore the potential of integrating Large Language Models with external knowledge.'"639],640"application/vnd.google.colaboratory.intrinsic+json": {641"type": "string"642}643},644"metadata": {},645"execution_count": 36646}647],648"source": [649"count_tokens(\n",650" conversation_buf, \n",651" \"What is my aim again?\"\n",652")"653]654},655{656"cell_type": "markdown",657"id": "431b74ff",658"metadata": {659"id": "431b74ff"660},661"source": [662"Our LLM with `ConversationBufferMemory` can clearly remember earlier interactions in the conversation. Let's take a closer look to how the LLM is saving our previous conversation. We can do this by accessing the `.buffer` attribute for the `.memory` in our chain."663]664},665{666"cell_type": "code",667"execution_count": 37,668"id": "984afd09",669"metadata": {670"colab": {671"base_uri": "https://localhost:8080/"672},673"id": "984afd09",674"outputId": "4233d17f-1001-48e5-d256-0595e00dbf40"675},676"outputs": [677{678"output_type": "stream",679"name": "stdout",680"text": [681"\n",682"Human: Good morning AI!\n",683"AI: Good morning! It's a beautiful day today, isn't it? How can I help you?\n",684"Human: My interest here is to explore the potential of integrating Large Language Models with external knowledge\n",685"AI: Interesting! Large Language Models are a type of artificial intelligence that can process natural language and generate text. They can be used to generate text from a given context, or to answer questions about a given context. Integrating them with external knowledge can help them to better understand the context and generate more accurate results. Is there anything else I can help you with?\n",686"Human: I just want to analyze the different possibilities. What can you think of?\n",687"AI: Well, integrating Large Language Models with external knowledge can open up a lot of possibilities. For example, you could use them to generate more accurate and detailed summaries of text, or to answer questions about a given context more accurately. You could also use them to generate more accurate translations, or to generate more accurate predictions about future events.\n",688"Human: Which data source types could be used to give context to the model?\n",689"AI: There are a variety of data sources that could be used to give context to a Large Language Model. These include structured data sources such as databases, unstructured data sources such as text documents, and even audio and video data sources. Additionally, you could use external knowledge sources such as Wikipedia or other online encyclopedias to provide additional context.\n",690"Human: What is my aim again?\n",691"AI: Your aim is to explore the potential of integrating Large Language Models with external knowledge.\n"692]693}694],695"source": [696"print(conversation_buf.memory.buffer)"697]698},699{700"cell_type": "markdown",701"id": "4570267d",702"metadata": {703"id": "4570267d"704},705"source": [706"Nice! So every piece of our conversation has been explicitly recorded and sent to the LLM in the prompt."707]708},709{710"cell_type": "markdown",711"id": "acf1a90b",712"metadata": {713"id": "acf1a90b"714},715"source": [716"### Memory type #2: ConversationSummaryMemory"717]718},719{720"cell_type": "markdown",721"id": "01f61fe9",722"metadata": {723"id": "01f61fe9"724},725"source": [726"The problem with the `ConversationBufferMemory` is that as the conversation progresses, the token count of our context history adds up. This is problematic because we might max out our LLM with a prompt that is too large to be processed."727]728},729{730"cell_type": "markdown",731"id": "0516c7d4",732"metadata": {733"id": "0516c7d4"734},735"source": [736"Enter `ConversationSummaryMemory`.\n",737"\n",738"Again, we can infer from the name what is going on.. we will keep a summary of our previous conversation snippets as our history. How will we summarize these? LLM to the rescue."739]740},741{742"cell_type": "markdown",743"id": "86b0a905",744"metadata": {745"id": "86b0a905"746},747"source": [748"**Key feature:** _the conversation summary memory keeps the previous pieces of conversation in a summarized form, where the summarization is performed by an LLM._"749]750},751{752"cell_type": "markdown",753"id": "0ea6050c",754"metadata": {755"id": "0ea6050c"756},757"source": [758"In this case we need to send the llm to our memory constructor to power its summarization ability."759]760},761{762"cell_type": "code",763"execution_count": 38,764"id": "f33a16a7",765"metadata": {766"id": "f33a16a7"767},768"outputs": [],769"source": [770"conversation_sum = ConversationChain(\n",771" llm=llm, \n",772" memory=ConversationSummaryMemory(llm=llm)\n",773")"774]775},776{777"cell_type": "markdown",778"id": "b64c4896",779"metadata": {780"id": "b64c4896"781},782"source": [783"When we have an llm, we always have a prompt ;) Let's see what's going on inside our conversation summary memory:"784]785},786{787"cell_type": "code",788"execution_count": 39,789"id": "c476824d",790"metadata": {791"colab": {792"base_uri": "https://localhost:8080/"793},794"id": "c476824d",795"outputId": "282be20e-9048-4f37-fc89-8a7eb8dfe1a3"796},797"outputs": [798{799"output_type": "stream",800"name": "stdout",801"text": [802"Progressively summarize the lines of conversation provided, adding onto the previous summary returning a new summary.\n",803"\n",804"EXAMPLE\n",805"Current summary:\n",806"The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good.\n",807"\n",808"New lines of conversation:\n",809"Human: Why do you think artificial intelligence is a force for good?\n",810"AI: Because artificial intelligence will help humans reach their full potential.\n",811"\n",812"New summary:\n",813"The human asks what the AI thinks of artificial intelligence. The AI thinks artificial intelligence is a force for good because it will help humans reach their full potential.\n",814"END OF EXAMPLE\n",815"\n",816"Current summary:\n",817"{summary}\n",818"\n",819"New lines of conversation:\n",820"{new_lines}\n",821"\n",822"New summary:\n"823]824}825],826"source": [827"print(conversation_sum.memory.prompt.template)"828]829},830{831"cell_type": "markdown",832"id": "df90cdf3",833"metadata": {834"id": "df90cdf3"835},836"source": [837"Cool! So each new interaction is summarized and appended to a running summary as the memory of our chain. Let's see how this works in practice!"838]839},840{841"cell_type": "code",842"execution_count": 40,843"id": "34343665",844"metadata": {845"colab": {846"base_uri": "https://localhost:8080/",847"height": 53848},849"id": "34343665",850"outputId": "ac04f6bc-9dcb-446c-d4b9-8fd2311d605e"851},852"outputs": [853{854"output_type": "stream",855"name": "stdout",856"text": [857"Spent a total of 290 tokens\n"858]859},860{861"output_type": "execute_result",862"data": {863"text/plain": [864"\" Good morning! It's a beautiful day today, isn't it? How can I help you?\""865],866"application/vnd.google.colaboratory.intrinsic+json": {867"type": "string"868}869},870"metadata": {},871"execution_count": 40872}873],874"source": [875"# without count_tokens we'd call `conversation_sum(\"Good morning AI!\")`\n",876"# but let's keep track of our tokens:\n",877"count_tokens(\n",878" conversation_sum, \n",879" \"Good morning AI!\"\n",880")"881]882},883{884"cell_type": "code",885"execution_count": 41,886"id": "b757bba3",887"metadata": {888"colab": {889"base_uri": "https://localhost:8080/",890"height": 71891},892"id": "b757bba3",893"outputId": "9de1823a-0dfe-45ff-fadc-26eff6fdce99"894},895"outputs": [896{897"output_type": "stream",898"name": "stdout",899"text": [900"Spent a total of 440 tokens\n"901]902},903{904"output_type": "execute_result",905"data": {906"text/plain": [907"\" That sounds like an interesting project! I'm familiar with Large Language Models, but I'm not sure how they could be integrated with external knowledge. Could you tell me more about what you have in mind?\""908],909"application/vnd.google.colaboratory.intrinsic+json": {910"type": "string"911}912},913"metadata": {},914"execution_count": 41915}916],917"source": [918"count_tokens(\n",919" conversation_sum, \n",920" \"My interest here is to explore the potential of integrating Large Language Models with external knowledge\"\n",921")"922]923},924{925"cell_type": "code",926"execution_count": 42,927"id": "d0a373e2",928"metadata": {929"colab": {930"base_uri": "https://localhost:8080/",931"height": 106932},933"id": "d0a373e2",934"outputId": "d4f561d7-d1c7-45e5-99ba-266130ee67ba"935},936"outputs": [937{938"output_type": "stream",939"name": "stdout",940"text": [941"Spent a total of 664 tokens\n"942]943},944{945"output_type": "execute_result",946"data": {947"text/plain": [948"' I can think of a few possibilities. One option is to use a large language model to generate a set of candidate answers to a given query, and then use external knowledge to filter out the most relevant answers. Another option is to use the large language model to generate a set of candidate answers, and then use external knowledge to score and rank the answers. Finally, you could use the large language model to generate a set of candidate answers, and then use external knowledge to refine the answers.'"949],950"application/vnd.google.colaboratory.intrinsic+json": {951"type": "string"952}953},954"metadata": {},955"execution_count": 42956}957],958"source": [959"count_tokens(\n",960" conversation_sum, \n",961" \"I just want to analyze the different possibilities. What can you think of?\"\n",962")"963]964},965{966"cell_type": "code",967"execution_count": 43,968"id": "2e286f0d",969"metadata": {970"colab": {971"base_uri": "https://localhost:8080/",972"height": 89973},974"id": "2e286f0d",975"outputId": "9558ef92-5f9c-4818-be8b-1e7e6ec19864"976},977"outputs": [978{979"output_type": "stream",980"name": "stdout",981"text": [982"Spent a total of 799 tokens\n"983]984},985{986"output_type": "execute_result",987"data": {988"text/plain": [989"' There are many different types of data sources that could be used to give context to the model. These could include structured data sources such as databases, unstructured data sources such as text documents, or even external APIs that provide access to external knowledge. Additionally, the model could be trained on a combination of these data sources to provide a more comprehensive understanding of the context.'"990],991"application/vnd.google.colaboratory.intrinsic+json": {992"type": "string"993}994},995"metadata": {},996"execution_count": 43997}998],999"source": [1000"count_tokens(\n",1001" conversation_sum, \n",1002" \"Which data source types could be used to give context to the model?\"\n",1003")"1004]1005},1006{1007"cell_type": "code",1008"execution_count": 44,1009"id": "891180f2",1010"metadata": {1011"colab": {1012"base_uri": "https://localhost:8080/",1013"height": 531014},1015"id": "891180f2",1016"outputId": "8035333e-d7c0-4a46-d8b8-acb3501d27e1"1017},1018"outputs": [1019{1020"output_type": "stream",1021"name": "stdout",1022"text": [1023"Spent a total of 853 tokens\n"1024]1025},1026{1027"output_type": "execute_result",1028"data": {1029"text/plain": [1030"' Your aim is to explore the potential of integrating Large Language Models with external knowledge.'"1031],1032"application/vnd.google.colaboratory.intrinsic+json": {1033"type": "string"1034}1035},1036"metadata": {},1037"execution_count": 441038}1039],1040"source": [1041"count_tokens(\n",1042" conversation_sum, \n",1043" \"What is my aim again?\"\n",1044")"1045]1046},1047{1048"cell_type": "code",1049"execution_count": 45,1050"id": "2d768e44",1051"metadata": {1052"colab": {1053"base_uri": "https://localhost:8080/"1054},1055"id": "2d768e44",1056"outputId": "3bd42ac9-d56b-45f4-99ac-45cd5a656b94"1057},1058"outputs": [1059{1060"output_type": "stream",1061"name": "stdout",1062"text": [1063"\n",1064"The human greeted the AI with a good morning, to which the AI responded with a good morning and asked how it could help. The human expressed interest in exploring the potential of integrating Large Language Models with external knowledge, to which the AI responded positively and asked for more information. The human asked the AI to think of different possibilities, and the AI suggested three options: using the large language model to generate a set of candidate answers and then using external knowledge to filter out the most relevant answers, score and rank the answers, or refine the answers. The human then asked which data source types could be used to give context to the model, to which the AI responded that there are many different types of data sources that could be used, such as structured data sources, unstructured data sources, or external APIs. Additionally, the model could be trained on a combination of these data sources to provide a more comprehensive understanding of the context. The human then asked what their aim was again, to which the AI responded that their aim was to explore the potential of integrating Large Language Models with external knowledge.\n"1065]1066}1067],1068"source": [1069"print(conversation_sum.memory.buffer)"1070]1071},1072{1073"cell_type": "markdown",1074"id": "0dd35c8c",1075"metadata": {1076"id": "0dd35c8c"1077},1078"source": [1079"You might be wondering.. if the aggregate token count is greater in each call here than in the buffer example, why should we use this type of memory? Well, if we check out buffer we will realize that although we are using more tokens in each instance of our conversation, our final history is shorter. This will enable us to have many more interactions before we reach our prompt's max length, making our chatbot more robust to longer conversations.\n",1080"\n",1081"We can count the number of tokens being used (without making a call to OpenAI) using the `tiktoken` tokenizer like so:"1082]1083},1084{1085"cell_type": "code",1086"execution_count": 46,1087"id": "nzijj4RZFX3I",1088"metadata": {1089"colab": {1090"base_uri": "https://localhost:8080/"1091},1092"id": "nzijj4RZFX3I",1093"outputId": "dc272cbb-acfd-4b4a-f854-8fa63f9732d6"1094},1095"outputs": [1096{1097"output_type": "stream",1098"name": "stdout",1099"text": [1100"Buffer memory conversation length: 334\n",1101"Summary memory conversation length: 219\n"1102]1103}1104],1105"source": [1106"# initialize tokenizer\n",1107"tokenizer = tiktoken.encoding_for_model('text-davinci-003')\n",1108"\n",1109"# show number of tokens for the memory used by each memory type\n",1110"print(\n",1111" f'Buffer memory conversation length: {len(tokenizer.encode(conversation_buf.memory.buffer))}\\n'\n",1112" f'Summary memory conversation length: {len(tokenizer.encode(conversation_sum.memory.buffer))}'\n",1113")"1114]1115},1116{1117"cell_type": "markdown",1118"id": "2bab0c09",1119"metadata": {1120"id": "2bab0c09"1121},1122"source": [1123"_Practical Note: the `text-davinci-003` and `gpt-3.5-turbo` models [have](https://platform.openai.com/docs/api-reference/completions/create#completions/create-max_tokens) a large max tokens count of 4096 tokens between prompt and answer._"1124]1125},1126{1127"cell_type": "markdown",1128"id": "494830ea",1129"metadata": {1130"id": "494830ea"1131},1132"source": [1133"### Memory type #3: ConversationBufferWindowMemory"1134]1135},1136{1137"cell_type": "markdown",1138"id": "00762844",1139"metadata": {1140"id": "00762844"1141},1142"source": [1143"Another great option for these cases is the `ConversationBufferWindowMemory` where we will be keeping a few of the last interactions in our memory but we will intentionally drop the oldest ones - short-term memory if you'd like. Here the aggregate token count **and** the per-call token count will drop noticeably. We will control this window with the `k` parameter."1144]1145},1146{1147"cell_type": "markdown",1148"id": "206a5915",1149"metadata": {1150"id": "206a5915"1151},1152"source": [1153"**Key feature:** _the conversation buffer window memory keeps the latest pieces of the conversation in raw form_"1154]1155},1156{1157"cell_type": "code",1158"execution_count": 60,1159"id": "45be373a",1160"metadata": {1161"id": "45be373a"1162},1163"outputs": [],1164"source": [1165"conversation_bufw = ConversationChain(\n",1166" llm=llm, \n",1167" memory=ConversationBufferWindowMemory(k=1)\n",1168")"1169]1170},1171{1172"cell_type": "code",1173"execution_count": 61,1174"id": "fc4dd8a0",1175"metadata": {1176"colab": {1177"base_uri": "https://localhost:8080/",1178"height": 531179},1180"id": "fc4dd8a0",1181"outputId": "c4ec1cc8-f218-4f7b-e27e-f5fb73e59228"1182},1183"outputs": [1184{1185"output_type": "stream",1186"name": "stdout",1187"text": [1188"Spent a total of 85 tokens\n"1189]1190},1191{1192"output_type": "execute_result",1193"data": {1194"text/plain": [1195"\" Good morning! It's a beautiful day today, isn't it? How can I help you?\""1196],1197"application/vnd.google.colaboratory.intrinsic+json": {1198"type": "string"1199}1200},1201"metadata": {},1202"execution_count": 611203}1204],1205"source": [1206"count_tokens(\n",1207" conversation_bufw, \n",1208" \"Good morning AI!\"\n",1209")"1210]1211},1212{1213"cell_type": "code",1214"execution_count": 62,1215"id": "b9992e8d",1216"metadata": {1217"colab": {1218"base_uri": "https://localhost:8080/",1219"height": 891220},1221"id": "b9992e8d",1222"outputId": "ac7ae1af-2329-4766-ac5e-8fce24a1d272"1223},1224"outputs": [1225{1226"output_type": "stream",1227"name": "stdout",1228"text": [1229"Spent a total of 178 tokens\n"1230]1231},1232{1233"output_type": "execute_result",1234"data": {1235"text/plain": [1236"' Interesting! Large Language Models are a type of artificial intelligence that can process natural language and generate text. They can be used to generate text from a given context, or to answer questions about a given context. Integrating them with external knowledge can help them to better understand the context and generate more accurate results. Do you have any specific questions about this integration?'"1237],1238"application/vnd.google.colaboratory.intrinsic+json": {1239"type": "string"1240}1241},1242"metadata": {},1243"execution_count": 621244}1245],1246"source": [1247"count_tokens(\n",1248" conversation_bufw, \n",1249" \"My interest here is to explore the potential of integrating Large Language Models with external knowledge\"\n",1250")"1251]1252},1253{1254"cell_type": "code",1255"execution_count": 63,1256"id": "3f2e98d9",1257"metadata": {1258"colab": {1259"base_uri": "https://localhost:8080/",1260"height": 891261},1262"id": "3f2e98d9",1263"outputId": "dc60726a-4be2-480f-892b-443da9b2859e"1264},1265"outputs": [1266{1267"output_type": "stream",1268"name": "stdout",1269"text": [1270"Spent a total of 233 tokens\n"1271]1272},1273{1274"output_type": "execute_result",1275"data": {1276"text/plain": [1277"' There are many possibilities for integrating Large Language Models with external knowledge. For example, you could use external knowledge to provide additional context to the model, or to provide additional training data. You could also use external knowledge to help the model better understand the context of a given text, or to help it generate more accurate results.'"1278],1279"application/vnd.google.colaboratory.intrinsic+json": {1280"type": "string"1281}1282},1283"metadata": {},1284"execution_count": 631285}1286],1287"source": [1288"count_tokens(\n",1289" conversation_bufw, \n",1290" \"I just want to analyze the different possibilities. What can you think of?\"\n",1291")"1292]1293},1294{1295"cell_type": "code",1296"execution_count": 64,1297"id": "a2a8d062",1298"metadata": {1299"colab": {1300"base_uri": "https://localhost:8080/",1301"height": 1061302},1303"id": "a2a8d062",1304"outputId": "dbb27cf0-2e87-41d0-a733-68921d250481"1305},1306"outputs": [1307{1308"output_type": "stream",1309"name": "stdout",1310"text": [1311"Spent a total of 245 tokens\n"1312]1313},1314{1315"output_type": "execute_result",1316"data": {1317"text/plain": [1318"' Data sources that could be used to give context to the model include text corpora, structured databases, and ontologies. Text corpora provide a large amount of text data that can be used to train the model and provide additional context. Structured databases provide structured data that can be used to provide additional context to the model. Ontologies provide a structured representation of knowledge that can be used to provide additional context to the model.'"1319],1320"application/vnd.google.colaboratory.intrinsic+json": {1321"type": "string"1322}1323},1324"metadata": {},1325"execution_count": 641326}1327],1328"source": [1329"count_tokens(\n",1330" conversation_bufw, \n",1331" \"Which data source types could be used to give context to the model?\"\n",1332")"1333]1334},1335{1336"cell_type": "code",1337"execution_count": 65,1338"id": "ff199a3f",1339"metadata": {1340"colab": {1341"base_uri": "https://localhost:8080/",1342"height": 531343},1344"id": "ff199a3f",1345"outputId": "81573cf0-7f39-4a8c-8ccd-e79cd80f2523"1346},1347"outputs": [1348{1349"output_type": "stream",1350"name": "stdout",1351"text": [1352"Spent a total of 186 tokens\n"1353]1354},1355{1356"output_type": "execute_result",1357"data": {1358"text/plain": [1359"' Your aim is to use data sources to give context to the model.'"1360],1361"application/vnd.google.colaboratory.intrinsic+json": {1362"type": "string"1363}1364},1365"metadata": {},1366"execution_count": 651367}1368],1369"source": [1370"count_tokens(\n",1371" conversation_bufw, \n",1372" \"What is my aim again?\"\n",1373")"1374]1375},1376{1377"cell_type": "markdown",1378"id": "f5f59f77",1379"metadata": {1380"id": "f5f59f77"1381},1382"source": [1383"As we can see, it effectively 'fogot' what we talked about in the first interaction. Let's see what it 'remembers'. Given that we set k to be `1`, we would expect it remembers only the last interaction."1384]1385},1386{1387"cell_type": "markdown",1388"id": "6b354c8d",1389"metadata": {1390"id": "6b354c8d"1391},1392"source": [1393"We need to access a special method here since, in this memory type, the buffer is first passed through this method to be sent later to the llm."1394]1395},1396{1397"cell_type": "code",1398"execution_count": 66,1399"id": "85266406",1400"metadata": {1401"id": "85266406"1402},1403"outputs": [],1404"source": [1405"bufw_history = conversation_bufw.memory.load_memory_variables(\n",1406" inputs=[]\n",1407")['history']"1408]1409},1410{1411"cell_type": "code",1412"execution_count": 67,1413"id": "5904ae2a",1414"metadata": {1415"colab": {1416"base_uri": "https://localhost:8080/"1417},1418"id": "5904ae2a",1419"outputId": "bd0aa797-7a43-4af5-a531-209aa6272dd4"1420},1421"outputs": [1422{1423"output_type": "stream",1424"name": "stdout",1425"text": [1426"Human: What is my aim again?\n",1427"AI: Your aim is to use data sources to give context to the model.\n"1428]1429}1430],1431"source": [1432"print(bufw_history)"1433]1434},1435{1436"cell_type": "markdown",1437"id": "ae8b937d",1438"metadata": {1439"id": "ae8b937d"1440},1441"source": [1442"Makes sense. \n",1443"\n",1444"On the plus side, we are shortening our conversation length when compared to buffer memory _without_ a window:"1445]1446},1447{1448"cell_type": "code",1449"execution_count": 68,1450"id": "9fbb50fe",1451"metadata": {1452"colab": {1453"base_uri": "https://localhost:8080/"1454},1455"id": "9fbb50fe",1456"outputId": "c35dca36-a7c7-4d61-da19-c28173fa8319"1457},1458"outputs": [1459{1460"output_type": "stream",1461"name": "stdout",1462"text": [1463"Buffer memory conversation length: 334\n",1464"Summary memory conversation length: 219\n",1465"Buffer window memory conversation length: 26\n"1466]1467}1468],1469"source": [1470"print(\n",1471" f'Buffer memory conversation length: {len(tokenizer.encode(conversation_buf.memory.buffer))}\\n'\n",1472" f'Summary memory conversation length: {len(tokenizer.encode(conversation_sum.memory.buffer))}\\n'\n",1473" f'Buffer window memory conversation length: {len(tokenizer.encode(bufw_history))}'\n",1474")"1475]1476},1477{1478"cell_type": "markdown",1479"id": "69842cc1",1480"metadata": {1481"id": "69842cc1"1482},1483"source": [1484"_Practical Note: We are using `k=2` here for illustrative purposes, in most real world applications you would need a higher value for k._"1485]1486},1487{1488"cell_type": "markdown",1489"id": "2aea5fc8",1490"metadata": {1491"id": "2aea5fc8"1492},1493"source": [1494"### More memory types!"1495]1496},1497{1498"cell_type": "markdown",1499"id": "daeb5162",1500"metadata": {1501"id": "daeb5162"1502},1503"source": [1504"Given that we understand memory already, we will present a few more memory types here and hopefully a brief description will be enough to understand their underlying functionality."1505]1506},1507{1508"cell_type": "markdown",1509"id": "f0365333",1510"metadata": {1511"id": "f0365333"1512},1513"source": [1514"#### ConversationSummaryBufferMemory"1515]1516},1517{1518"cell_type": "markdown",1519"id": "317f298e",1520"metadata": {1521"id": "317f298e"1522},1523"source": [1524"**Key feature:** _the conversation summary memory keeps a summary of the earliest pieces of conversation while retaining a raw recollection of the latest interactions._"1525]1526},1527{1528"cell_type": "markdown",1529"id": "57ef5c8b",1530"metadata": {1531"id": "57ef5c8b"1532},1533"source": [1534"#### ConversationKnowledgeGraphMemory"1535]1536},1537{1538"cell_type": "markdown",1539"id": "40248f03",1540"metadata": {1541"id": "40248f03"1542},1543"source": [1544"This is a super cool memory type that was introduced just [recently](https://twitter.com/LangChainAI/status/1625158388824043522). It is based on the concept of a _knowledge graph_ which recognizes different entities and connects them in pairs with a predicate resulting in (subject, predicate, object) triplets. This enables us to compress a lot of information into highly significant snippets that can be fed into the model as context. If you want to understand this memory type in more depth you can check out [this](https://apex974.com/articles/explore-langchain-support-for-knowledge-graph) blogpost."1545]1546},1547{1548"cell_type": "markdown",1549"id": "91952cd1",1550"metadata": {1551"id": "91952cd1"1552},1553"source": [1554"**Key feature:** _the conversation knowledge graph memory keeps a knowledge graph of all the entities that have been mentioned in the interactions together with their semantic relationships._"1555]1556},1557{1558"cell_type": "code",1559"execution_count": 69,1560"id": "02241bc3",1561"metadata": {1562"id": "02241bc3"1563},1564"outputs": [],1565"source": [1566"# you may need to install this library\n",1567"# !pip install -qU networkx"1568]1569},1570{1571"cell_type": "code",1572"execution_count": 70,1573"id": "c5f10a89",1574"metadata": {1575"id": "c5f10a89"1576},1577"outputs": [],1578"source": [1579"conversation_kg = ConversationChain(\n",1580" llm=llm, \n",1581" memory=ConversationKGMemory(llm=llm)\n",1582")"1583]1584},1585{1586"cell_type": "code",1587"execution_count": 71,1588"id": "65957fe2",1589"metadata": {1590"colab": {1591"base_uri": "https://localhost:8080/",1592"height": 531593},1594"id": "65957fe2",1595"outputId": "c9561a4a-412a-4d92-865d-9e81a09bb101"1596},1597"outputs": [1598{1599"output_type": "stream",1600"name": "stdout",1601"text": [1602"Spent a total of 1565 tokens\n"1603]1604},1605{1606"output_type": "execute_result",1607"data": {1608"text/plain": [1609"\" Hi Human! My name is AI. It's nice to meet you. I like mangoes too! Did you know that mangoes are a great source of vitamins A and C?\""1610],1611"application/vnd.google.colaboratory.intrinsic+json": {1612"type": "string"1613}1614},1615"metadata": {},1616"execution_count": 711617}1618],1619"source": [1620"count_tokens(\n",1621" conversation_kg, \n",1622" \"My name is human and I like mangoes!\"\n",1623")"1624]1625},1626{1627"cell_type": "markdown",1628"id": "74054534",1629"metadata": {1630"id": "74054534"1631},1632"source": [1633"The memory keeps a knowledge graph of everything it learned so far."1634]1635},1636{1637"cell_type": "code",1638"execution_count": 72,1639"id": "5a8c54fb",1640"metadata": {1641"colab": {1642"base_uri": "https://localhost:8080/"1643},1644"id": "5a8c54fb",1645"outputId": "adf96679-087b-4b77-c00d-9bf9e98f9278"1646},1647"outputs": [1648{1649"output_type": "execute_result",1650"data": {1651"text/plain": [1652"[('human', 'human', 'name'), ('human', 'mangoes', 'likes')]"1653]1654},1655"metadata": {},1656"execution_count": 721657}1658],1659"source": [1660"conversation_kg.memory.kg.get_triples()"1661]1662},1663{1664"cell_type": "markdown",1665"id": "e1a1ca15",1666"metadata": {1667"id": "e1a1ca15"1668},1669"source": [1670"#### ConversationEntityMemory"1671]1672},1673{1674"cell_type": "markdown",1675"id": "41e9aeaf",1676"metadata": {1677"id": "41e9aeaf"1678},1679"source": [1680"**Key feature:** _the conversation entity memory keeps a recollection of the main entities that have been mentioned, together with their specific attributes._"1681]1682},1683{1684"cell_type": "markdown",1685"id": "2900a385",1686"metadata": {1687"id": "2900a385"1688},1689"source": [1690"The way this works is quite similar to the `ConversationKnowledgeGraphMemory`, you can refer to the [docs](https://python.langchain.com/en/latest/modules/memory/types/entity_summary_memory.html) if you want to see it in action. "1691]1692},1693{1694"cell_type": "markdown",1695"id": "d45112bd",1696"metadata": {1697"id": "d45112bd"1698},1699"source": [1700"## What else can we do with memory?"1701]1702},1703{1704"cell_type": "markdown",1705"id": "78296bff",1706"metadata": {1707"id": "78296bff"1708},1709"source": [1710"There are several cool things we can do with memory in langchain. We can:\n",1711"* implement our own custom memory module\n",1712"* use multiple memory modules in the same chain\n",1713"* combine agents with memory and other tools\n",1714"\n",1715"If this piques your interest, we suggest you to go take a look at the memory [how-to](https://langchain.readthedocs.io/en/latest/modules/memory/how_to_guides.html) section in the docs!"1716]1717}1718],1719"metadata": {1720"colab": {1721"provenance": []1722},1723"kernelspec": {1724"display_name": "Python 3",1725"language": "python",1726"name": "python3"1727},1728"language_info": {1729"codemirror_mode": {1730"name": "ipython",1731"version": 31732},1733"file_extension": ".py",1734"mimetype": "text/x-python",1735"name": "python",1736"nbconvert_exporter": "python",1737"pygments_lexer": "ipython3",1738"version": "3.11.2"1739},1740"vscode": {1741"interpreter": {1742"hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"1743}1744}1745},1746"nbformat": 4,1747"nbformat_minor": 51748}1749