Skip to main content
This tutorial shows you how to build a simple RAG chatbot in Python using Pinecone for the vector database and embedding model, OpenAI for the LLM, and LangChain for the RAG workflow.
To run through this tutorial in your browser, use this colab notebook. For a more complex, multitenant RAG sample app and tutorial, see Namespace Notes.

​
How it works

GenAI chatbots built on Large Language Models (LLMs) can answer many questions. However, when the questions concern private data that the LLMs have not been trained on, you can get answers that sound convincing but are factually wrong. This behavior is referred to as “hallucination”. Retrieval augmented generation (RAG) is a framework that prevents hallucination by providing LLMs the knowledge that they are missing, based on private data stored in a vector database like Pinecone. RAG overview

​
Before you begin

Ensure you have the following:

​
1. Set up your environment

  1. Install the Pinecone and LangChain libraries required for this tutorial:
  2. Set environment variables for your Pinecone and OpenAI API keys:

​
2. Store knowledge in Pinecone

For this tutorial, you’ll use a document about a fictional product called the WonderVector5000 that LLMs do not have any information about. First, you’ll use LangChain to chunk the document into smaller segments and create vector embeddings for each segment via Pinecone Inference. Then you’ll create a Pinecone index and upsert the vector embeddings into it.
Pinecone Inference is an API service that gives you access to embedding models and reranking models hosted on Pinecone’s infrastructure. You can use the Inference API directly or through Langchain’s PineconeEmbeddings class, as shown in this tutorial.
Markdown
  1. Since your document is in Markdown, chunk the content based on structure to get semantically coherent segments. In this case, headers_to_split_on specifies h2 headers as the indicators of where to split.
    Python
    Response
  2. Initialize a LangChain embedding object. This is what you will use to create vectors from the document above and store them in Pinecone. Note that this step uses a Pinecone API key you set as an environment variable earlier.
    Python
  3. Create a Pinecone index to store the document in. Define the cloud region and index name and set the index dimensions and distance metric to match those of the Pinecone multilingual-e5-large model you’ll use to create the embeddings:
    Python
    Response
  4. Embed and upsert each chunk as a distinct record in a namespace called wondervector5000. Namespaces let you partition records within an index and are essential for implementing multitenancy when you need to isolate the data of each customer/user.
    Python
    Response
  5. Use Pinecone’s list and query operations to look at one of the records:
    Python
    Response

​
3. Use the chatbot

Now that your document is stored as embeddings in Pinecone, when you send questions to the LLM, you can add relevant knowledge from your Pinecone index to ensure that the LLM returns an accurate response.
  1. Initialize a LangChain object for chatting with OpenAI’s gpt-4o-mini LLM. OpenAI is a paid service, so running the remainder of this tutorial may incur some small cost.
    Python
  2. Define a few questions about the WonderVector5000. These questions require specific, private knowledge of the product, which the LLM does not have by default.
    Python
  3. Send query1 to the LLM without relevant context from Pinecone:
    Python
    Notice that this first response sounds convincing but is entirely fabricated. This is an hallucination.
    Response
  4. Try again, but this time provide the right context from Pinecone:
    Python
    Notice that this second response provides very accurate getting started steps, matching closely the information in the WonderVector5000 document.
    Response
  5. Repeat the process with query2:
    Python
    Again, notice how this first response sounds convincing but is entirely fabricated. This is an hallucination.
    Response
  6. Try again, but this time provide the right contexst from Pinecone:
    Python
    Notice that this second response provides very accurate troubleshooting guidance, matching closely the information in the WonderVector5000 document.
    Response

​
4. Clean up

When you no longer need the rag-getting-started index, use the delete_index operation to delete it:
Python

​
Next steps