OpenAI with Rate Limit
Last updated
Last updated
we will refer to the basics demonstrated in the demo created by Vulture Prime. It is a website to generate API keys for users who want to connect to Vulture Prime's OpenAI. Users can generate API keys through the website automatically without the need for admin approval. Users can also choose the desired rate limit per day.
Note: In cases where there is a user dashboard, users need to log in before using the system.
To facilitate management, we have created a UI page that allows users to access the API key easily without contacting the admin. Users can obtain the OpenAI API key by simply clicking a few times on the website. They can select the desired rate limit per day.
Implement the Chat interface
In the process of creating the UI, we chose libraries and frameworks as follows:
Next.js: Start a new project using the command: npx create-next-app my-project
.
npx create-next-app my-project
cd my-project
React Hook Form: Install with: yarn add react-hook-form @hookform/resolvers zod
. We use Zod for input form validation.
yarn add react-hook-form @hookform/resolvers zod
TanStack Query: Install TanStack Query (React Query) in the project: yarn add react-query
.
yarn add react-query
Connect the frontend with the backend
Now, we will implement the Chat Interface with a Streamed Text approach to receive real-time text data from the API and display it.
codeuseEffect(() => {
const message = { query: watch('query') };
const getData = async () => {
try {
setValue('query', '');
const response = await fetch(
`${API_BOT}/query?uuid=${localStorage?.session}&message=${message.query}`,
{
method: 'GET',
headers: {
Accept: 'text/event-stream',
'x-api-key': localStorage?.apiKey, // API key
},
}
);
const reader = response.body!.getReader();
let result = '';
while (true) {
const { done, value } = await reader?.read();
if (done) {
setStreamText('');
setAnswer((prevState) => [
...prevState,
{
id: (prevState.length + 1).toString(),
role: 'ai',
message: result,
},
]);
break;
}
result += new TextDecoder().decode(value);
setStreamText(
(prevData) => prevData + new TextDecoder().decode(value)
);
}
} catch (error: any) {
console.error(error);
setError('bot', {
message: error?.response?.data?.message ?? 'Something went wrong',
});
}
};
if (isSubmitSuccessful) {
getData();
}
}, [submitCount, isSubmitSuccessful, setValue, watch, setError]);
This code snippet fetches data from the API using the Streamed Text approach, allowing real-time updates of chat messages.
Implementing API key generation
After setting up the UI interface, we connect it to the backend for API key generation.
// API Plan
// GET {endpoint}/plan
[
{
"name": "20RequestPerDay"
},
{
"name": "30RequestPerDay"
},
{
"name": "10RequestPerDay"
}
]
// API Create Key
// POST {endpoint}/create_key
// Body
{
"plan_name": "30RequestPerDay",
"user": "example@email.com"
}
// Response 200 (OK)
{
"value": "7wuOmY7Osz721X1bQUkAP2aGUF5oOVw28EJx7MVS"
}
After obtaining the API key, we use it in other projects by adding the header:
{
"x-api-key": "7wuOmY7Osz721X1bQUkAP2aGUF5oOVw28EJx7MVS"
}
Implementing rate limit notification
We handle input validation errors and errors from API usage in the Stream. When the API usage exceeds the limit, the API will return a status code of 429. In this case, we set an error message to notify the user.
if (response.status === 429) {
setError('bot', {
message: 'Limit Exceeded',
});
return;
}
This ensures that the UI displays an error message when the user exceeds the API usage limit.
Begin by creating the usage plans that you want. Create three plans: 10RequestPerDay, 20RequestPerDay, and 30RequestPerDay. Map these plans to the API Gateway you created, indicating which plans should be used by which API Gateway.
Configure each method in API Gateway to require an API key. Edit the method request in the desired resource and check the "API Key required" option.
Install the necessary Python libraries for FastAPI:
pip install fastapi
pip install "uvicorn[standard]"
Create a file named app.py
and initialize the FastAPI application:
pythonCopy codefrom fastapi import FastAPI
from fastapi.encoders import jsonable_encoder
from fastapi.responses import JSONResponse
from fastapi.middleware.cors import CORSMiddleware
app = FastAPI()
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.get("/helloworld")
async def helloworld():
return {"message": "Hello World"}
This code initializes a FastAPI project, enables CORS for frontend communication, and sets up a simple endpoint. Run the server using:
uvicorn app:app
Use the boto3 library to interact with AWS services. Make sure that the EC2 instance has sufficient permissions for the intended actions.
pip install boto3
Create a boto3 client for API Gateway:
import boto3
client = boto3.client(
'apigateway',
region_name='ap-southeast-1'
)
Design
Create two API endpoints:
To view available usage plans:
def get_plan():
res = client.get_usage_plans()
plan_name_list = []
for i in res['items']:
plan_name_list.append({
"name": i['name']
})
return JSONResponse(content=plan_name_list, status_code=200)
To create an API key and associate it with a usage plan:
from fastapi import HTTPException
@app.post("/create_key")
async def create_key(create_request: create_request):
res = module.check_user_exist_in_plan(create_request.user, create_request.plan_name)
if res:
raise HTTPException(status_code=409, detail="Key already exists")
else:
response = client.create_api_key(
name=create_request.user,
description='for use api',
enabled=True,
generateDistinctId=True
)
api_key = response['value']
api_id = response['id']
try:
module.add_key_to_plan(api_id, create_request.plan_name)
result = {
"value": api_key
}
return JSONResponse(content=result, status_code=200)
except Exception as e:
client.delete_api_key(
apiKey=api_id
)
result = {
"message": "create failed"
}
return JSONResponse(content=result, status_code=500)
This endpoint creates an API key, associates it with the specified usage plan, and returns the generated API key.
Use the uvicorn
command to run the FastAPI server:
uvicorn app:app --host 0.0.0.0
This command runs the server on port 8000.
To test API calls with an API key through AWS API Gateway, include the key in the header as x-api-key
:
{
"x-api-key": "7wuOmY7Osz721X1bQUkAP2aGUFxxxxxxxx"
}
This key will be read by API Gateway and sent to the backend, which you configured earlier.