Q&A Bot (RAG)

Use Case Overview

This tutorial is based on the work done by Vulture Prime in creating a Q&A Chatbot. The Chatbot in this case interacts with user-uploaded documents, answering questions based on the content.

User Flow

Frontend Development

Implement the User Interface

In the frontend development phase of the Q&A Chatbot project, we chose the following libraries and frameworks:

• Next.js: Installed using create-next-app for project setup.

• Tailwind CSS: Utilized for styling the user interface.

• react-hook-form: Employed for form management, reducing complexity in handling form states.

• zod: Used in conjunction with react-hook-form for type-safe input validation.

Connect Frontend with Backend

Understanding the backend API is crucial in this step. Key points include:

• Familiarity with API endpoints, data formats (parameters or body), and the sequence of steps.

• Handling CORS issues, ensuring the frontend and backend can communicate seamlessly.

API Interfaces obtained:

• API LoadAndStore Interface:

  • GET {endpoint}/loadAndStore

  • Response: 200 (OK) with information about the default URL, chunk size, chunk overlap, and collection name.

  Request Body

{
  "url": "string", // default = https://lilianweng.github.io/posts/2023-06-23-agent
  "chunk_size": 1024, // default
  "chunk_overlap": 0, // default
  "collection_name": "string" // default = temp1
}

• API Query Interfaces:

  • POST {endpoint}/queryWithOutRetrieval

  • POST {endpoint}/queryWithRetrieval

  • Body: Contains the query and collection name.

  Request Body

{
  "query": "string",
  "collection_name": "string"
}

Implement URL Input Validation

URL input validation is crucial to prevent incorrect data from being sent to the backend. This involved using react-hook-form and zod for frontend validation.

Example validation schema using zod:

javascriptCopy codeexport const endpointScheme = z.object({
  endpoint: z.string().url().optional().or(z.literal('')),
});

export const collectionScheme = z.object({
  url: z.string({ required_error: 'Invalid url', invalid_type_error: 'Invalid url' }),
  // ...other fields
});

export const askScheme = z.object({
  query: z.string({ required_error: '', invalid_type_error: '' }),
  bot: z.string({}).optional(),
});

Implement Error Handling

Error handling for API calls was implemented to display meaningful messages on the website in case of failures. This ensures users can understand and troubleshoot issues easily.

Test, Optimize, and Deploy Frontend

After error handling, thorough testing of the frontend was conducted to ensure it met the project requirements. Optimization focused on improving code readability and user experience. The frontend was then deployed, involving building and addressing any errors before handing over deployment to the DevOps team.

Link to Frontend Code

Backend Development

How to create RAG with Langchain Link

Set Up EC2 Server

The backend development started with setting up an EC2 server. Python was confirmed to be installed, and necessary libraries such as langchain, chromadb, fastapi, and uvicorn were installed.

python3 --version

pip install langchain
pip install chromadb
pip install fastapi
pip install "uvicorn[standard]"

Develop FastAPI

FastAPI was used to create the backend API. A simple initial API endpoint was created, and additional functionality like file upload was added.

Begin by creating a main.py file with a simple endpoint:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def root():
    return {"message": "Hello World"}

To add more functionality, you can implement methods like POST for file uploads.

@app.post("/upload")
def update():
    # ...your code...
    return {"message": "Uploaded"}

Run FastAPI using:

uvicorn main:app

Integrate with RAG

Integration with RAG involved understanding the main steps: importing data and querying data. Functions for uploading data and querying data were developed and associated with corresponding API methods.

Chroma Vector Database

Chroma vector database was integrated to enhance data querying capabilities. The chromadb library was installed and configured as the vector database for Langchain.

Allow CORS

CORS (Cross-Origin Resource Sharing) was configured to enhance security for frontend integration. Add the following middleware in FastAPI:

from fastapi.middleware.cors import CORSMiddleware

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

Deploy Your Code

The backend code was deployed, and the use of tools like screen for detached execution was explained.

# Create a new screen session
screen -S name

# Navigate to the API folder
cd path/to/api

# Start FastAPI
uvicorn main:app

# Detach from the screen session
Ctrl+a d

Ensure that the security group of the EC2 instance allows traffic on the specified port (default is 8000).

Link FastAPI with API Gateway

AWS API Gateway was introduced to manage traffic and authentication for added security and features.

Test and Tuning

The backend was tested by querying data, and tuning involved adjusting parameters like data division and AI creativity to achieve desired results.

Link to Backend Code