How to use Chat GPT and LangChain to get responses from your documents
In the previous post, I explained methods that you can use to train Chat GPT on your data. In this post, we will use one of those methods “embeddings” to get Chat GPT response from our documents. Let’s dive in!
I am using an this text file that is a biography of a fictitious character, to create embeddings. We will use this file to generate responses from Chat GPT. Below is the snippet of the text file.
Early Life and Education:
Born on March 15th, 1985, in a small town named Greenridge,
John Anderson displayed an inquisitive nature from an early age.
Growing up in a supportive and nurturing family environment,
John was encouraged to pursue his interests and cultivate his talents. Import Required Packages
Let’s begin by importing the required packages. Please note that if you want to use Azure OpenAI endpoint, then use AzureOpenAI package. Otherwise, use openai. Read more about it here. I have used ChromaDB as a vector db to store the embeddings.
#Import required packages
#If you have your endpoing then use AzureOpenAI
#else use
#import openai
from langchain.llms import AzureOpenAI
#This will help us create embeddings
from langchain.embeddings.openai import OpenAIEmbeddings
#Using ChromaDB as a vector store for the embeddigns
from langchain.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain.document_loaders import DirectoryLoader
import osSet API Key and Endpoint
Please replace “OPENAI_API_KEY” with your secret API Key. Read how to get your secret key here. Replace “OPENAI_API_BASE” with your endpoint name. Read how to get these details here.
os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_VERSION"] = "2022-12-01"
#Set your API endpoint (API BASE) here if you are using Azure OpenAI
#If you are using openai common endpoing then you do not need to set this.
os.environ["OPENAI_API_BASE"] = "OPENAI_API_BASE"
os.environ["OPENAI_API_KEY"] = "OPENAI_API_KEY"Load Documents
Now we load our documents from a directory. The code below reads all the txt files in “docs” directory. If you want to read other file types, read the documentation here.
After loading all the text from the documents, we break them into small chunks to create embeddings.
#Load all the .txt files from docs directory
loader = DirectoryLoader('./docs/',glob = "**/*.txt")
docs = loader.load()
#Split text into tokens
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(docs)Create Embeddings
After ingesting the documents, let’s create embeddings using OpenAI Embeddings. You can provide the name of your deployment, if you are using Azure OpenAI. If not, then do not use this parameter. Read more about deployment name here.
Once embeddings are created, we can store them in Chroma vector db. The embeddings will be stored in “chromadb” directory, which will be created in your working directory.
#Turn the text into embeddings
embeddings = OpenAIEmbeddings(deployment="NAME_OF_YOUR_MODEL_DEPLOYMENT", chunk_size=1) #This model should be able to generate embeddings. For example, text-embedding-ada-002
#Store the embeddings into chromadb directory
docsearch = Chroma.from_documents(documents=texts, embedding=embeddings, persist_directory="./chromadb")Ask Questions!
We have reached the coolest part of the code. We can now ask Chat GPT any query and pass these embeddings to get response from our data.
#Use AzureOpenAI, if you're using a endpoint from Azure Open AI
#This can be any QnA model. For example, davinci.
#Remember to provide deployment name here and not the model name
llm = AzureOpenAI(deployment_name="NAME_OF_YOUR_MODEL_DEPLOYMENT")
#Use OpenAI if you're using a Azure OpenAI endpoint
#llm = ChatOpenAI(temperature = 0.7, model_name='MODEL_NAME')
qa = RetrievalQA.from_chain_type(llm=llm,
chain_type="stuff",
retriever=docsearch.as_retriever(),
return_source_documents=False
)
query = "Where was John born?"
qa.run(query)All Together
Let’s put it all together. You can find the complete notebook here.
#Import required packages
#If you have your endpoing then use AzureOpenAI
#else use
import openai
from langchain.llms import AzureOpenAI
#This will help us create embeddings
from langchain.embeddings.openai import OpenAIEmbeddings
#Using ChromaDB as a vector store for the embeddigns
from langchain.vectorstores import Chroma
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains import RetrievalQA
from langchain.document_loaders import DirectoryLoader
import os
os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_VERSION"] = "2022-12-01"
#Set your API endpoint (API BASE) here if you are using Azure OpenAI
#If you are using openai common endpoing then you do not need to set this.
os.environ["OPENAI_API_BASE"] = "OPENAI_API_BASE"
#Set your OPENAI API KEY here
os.environ["OPENAI_API_KEY"] = "OPENAI_API_KEY"
#Load all the .txt files from docs directory
loader = DirectoryLoader('./docs/',glob = "**/*.txt")
docs = loader.load()
#Split text into tokens
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
texts = text_splitter.split_documents(docs)
#Turn the text into embeddings
embeddings = OpenAIEmbeddings(deployment="NAME_OF_YOUR_MODEL_DEPLOYMENT", chunk_size=1) #This model should be able to generate embeddings. For example, text-embedding-ada-002
#Store the embeddings into chromadb directory
docsearch = Chroma.from_documents(documents=texts, embedding=embeddings, persist_directory="./chromadb")
#Use AzureOpenAI, if you're using a endpoint from Azure Open AI
llm = AzureOpenAI(deployment_name="NAME_OF_YOUR_MODEL_DEPLOYMENT") #This can be any QnA model. For example, davinci.
#Use OpenAI if you're using a Azure OpenAI endpoint
#llm = ChatOpenAI(temperature = 0.7, model_name='MODEL_NAME')
qa = RetrievalQA.from_chain_type(llm=llm,
chain_type="stuff",
retriever=docsearch.as_retriever(),
return_source_documents=False
)
query = "Where was John born?"
qa.run(query)
