📚
Docs - Float16
homeapp
  • 🚀GETTING STARTED
    • Introduction
    • Account
      • Dashboard
      • Profile
      • Payment
      • Workspace
    • One Click Deploy
      • Quick Start
        • Instance Detail
        • Re-generate API Key
        • Terminate Instance
      • Features
        • OpenAI Compatible
        • Long context and Auto scheduler
        • Quantization
        • Context caching
      • Limitation
      • Validated model
      • Endpoint Specification
    • Serverless GPU
      • Quick Start
        • Mode
        • Task Status
        • App Features
          • Project Detail
      • Tutorials
        • Hello World
        • Install new library
        • Prepare model weight
        • S3 Copy output from remote
        • R2 Copy output from remote
        • Direct upload and download
        • Server mode
        • LLM Dynamic Batching
        • Train and Inference MNIST
        • Etc.
      • CLI References
      • ❓FAQ
    • Playground
      • FloatChat
      • FloatPrompt
      • Quantize by Float16
  • 📚Use Case
    • Q&A Bot (RAG)
    • Text-to-SQL
    • OpenAI with Rate Limit
    • OpenAI with Guardrail
    • Multiple Agents
    • Q&A Chatbots (RAG + Agents)
  • ✳️Journey
    • ✨The Beginner's LLM Development Journey
    • 📖Glossary
      • [English Version] LLM Glossary
      • [ภาษาไทย] LLM Glossary
    • 🧠How to install node
  • Prompting
    • 📚Variable
    • ⛓️Condition
    • 🔨Demonstration
    • ⌛Loop
    • 📙Formatting
    • 🐣Chat
    • 🔎Technical term (Retrieve)
  • Privacy Policy
  • Terms & Conditions
Powered by GitBook
On this page
  • CLI Installation
  • macOS
  • Windows or Linux
  • Verification
  • Get your token
  1. GETTING STARTED
  2. Serverless GPU

Quick Start

Serverless GPU quick start

PreviousServerless GPUNextMode

Last updated 1 day ago

Our Serverless GPU is accessed through a Command Line Interface (CLI). Upon visiting the serverless GPU service page in our app, you'll find an onboarding section that guides you through first-time use. We recommend following these suggestions for a smooth start.

For those eager to dive in, you can explore our use cases in the next section. However, please familiarize yourself with the following essential information first

CLI Installation

Install the "Float16" CLI using one of the following methods, depending on your operating system:

macOS

Use Homebrew to install Float16 CLI

brew install float16-cloud/float16/cli

Windows or Linux

Install Float16 CLI globally using npm

npm install -g @float16/cli 

Ensure you have the appropriate package manager (Homebrew for macOS or npm for Windows) installed on your system before proceeding with the installation.

Verification

After installation, verify that Float16 CLI is correctly installed by running

float16 --version

This command should display the current version of Float16 CLI installed on your system.

Get your token

You can now easily access your token directly from the Float16 web interface.

  • Go to the Float16 Dashboard and click the “Serverless GPU” menu.

  • Click the “Token” tab, copy the token shown there for use with the CLI.

  • This token is required to authenticate any command using float16.

Notes:

  • Tokens are for individual use only and must not be shared.

  • If you're new to Float16, make sure to sign up and log in before accessing your token.

For any issues or if your token is missing, please contact our support team via Discord or the support page.

🚀
Learn more
Serverless authentication tokens