Server mode

Get Endpoint via Float16

This tutorial guides you through deploying a simple FastAPI "Hello World" application using Float16's deployment mode.

• Float16 CLI installed

• Logged into Float16 account

• Terminal environment (VSCode or preferred compiler recommended)

Step 1 : Prepare Your Script

(main.py)

from fastapi import FastAPI
import uvicorn
import time

app = FastAPI(
    title="Hello World API",
    description="A simple FastAPI application",
    version="1.0.0"
)

@app.get("/hello")
async def root():
    return {"message": "Hello World!"}

# Add a health check endpoint
@app.get("/health")
async def health_check():
    return {"status": "healthy"}

if __name__ == "__main__":
    # Configure and start the server
    uvicorn.run(
        "main:app",
        host="0.0.0.0",  # Makes the server externally accessible
        port=8000,
        reload=True  # Enable auto-reload on code changes
    )
```

• Save the script in a selected folder

• Navigate to the folder in your terminal

• Ensure the port is set to 8000

Step 2 : Create project

float16 project create --instance l4

Resulting Files

• float16.conf: Contains your project ID

• requirements.txt: Initially empty

If you cannot create new project, learn more

Step 3 : Update Dependencies

Update requirements.txt with necessary libraries:

uvicorn
fastapi

if you don't know or don't specify the exact version, you can just input the library name without version.

Install dependencies:

float16 project install

Step 4 : Deploy Script

float16 deploy main.py

After successful deployment, you'll receive:

• Function Endpoint

• Server Endpoint

• API Key

Example:

Function Endpoint: http://api.float16.cloud:8000/task/run/function/x7x2DFl8zU   
Server Endpoint: http://api.float16.cloud:8000/task/run/server/x7x2DFl8zU       
API Key: float16-r-QoZU7uNlgDIFJ5IMrBtOCjuzVBlC

Step 5 : Endpoint Request

Use the provided endpoints with the API key (bearer token) to make requests.

Endpoint Request Example:

• Path: /hello

• Expected Response: {"message": "Hello World!"}

To understand the differences between function and server modes, refer to the dedicated section.

Congratulations! You've successfully use your first server mode on Float16's serverless GPU platform.

Explore More

Learn how to use Float16 CLI for various use cases in our tutorials.

Hello World

Launch your first serverless GPU function and kickstart your journey.

List all library

Explore the complete set of libraries available in your container.

Install new library

Enhance your toolkit by adding new libraries tailored to your project needs.

Pre-load model

Accelerate model access using remote storage for improved performance.

Copy output from remote

Efficiently transfer computation results from remote to your local storage.

Happy coding with Float16 Serverless GPU!