S3 Copy output from remote

get your output to S3

In the Float16 project, each project has its own remote storage that automatically stores Python files and output files generated during run or deploy commands. This tutorial will guide you through copying output files from remote storage to your personal AWS S3 storage.

An active Float16 project
AWS S3 storage credentials

Step 1 : Prepare Your Script

Create a Python script (e.g., test_output.py) that generates an output file

import os

class FileWriter:
    def __init__(self, folder_name="output_files"):
        self.output_dir = folder_name
        os.makedirs(self.output_dir, exist_ok=True)
    
    def get_file_path(self, filename):
        return os.path.join(self.output_dir, filename)
    
    def write_simple_text(self):
        filename = f"output.txt"
        with open(self.get_file_path(filename), 'w', encoding='utf-8') as file:
            file.write('Hi\n')
            file.write('This is Serverless GPU\n')
            file.write('from Float16')
        return filename

def main():
    writer = FileWriter("my_output_files")
    
    print("processing...")
    
    simple_file = writer.write_simple_text()
    print(f"- write {simple_file} successfully")
    
    print(f"\nOutput Path: {os.path.abspath(writer.output_dir)}")

if __name__ == '__main__':
    main()

Step 2 : Run your script

First, start your project and then run the script

float16 project start

float16 run test_output.py

you will got successfully response from CLI.

Step 3 : Check Remote Storage

Check the files in your remote storage

float16 storage ls

Step 4 : Copy Output to S3

Use the following command to copy your output to AWS S3

float16 storage copy-output \
    --path my_output_files/output.txt \
    --s3-uri <YOUR_S3_URI_DESTINATION> \
    --s3-access-key <YOUR_S3_ACCESS_KEY> \
    --s3-secret-key <YOUR_S3_SECRET_KEY> \
    --aws-region <YOUR_S3_REGION>

Replace the placeholders with your actual AWS S3 credentials:

<YOUR_S3_URI_DESTINATION>: Your S3 bucket destination path
<YOUR_S3_ACCESS_KEY>: Your AWS access key
<YOUR_S3_SECRET_KEY>: Your AWS secret key
<YOUR_S3_REGION>: Your S3 bucket region

After running the command, check your specified S3 destination path. You should find the output.txt file successfully copied.

Congratulations! You've learned how to copy output files from Float16's remote storage to your AWS S3 storage.

Explore More

Learn how to use Float16 CLI for various use cases in our tutorials.

Happy coding with Float16 Serverless GPU!

PreviousPrepare model weight NextR2 Copy output from remote

Last updated 1 month ago

Step 1 : Prepare Your Script

Create a Python script (e.g., test_output.py) that generates an output file

import os

class FileWriter:
    def __init__(self, folder_name="output_files"):
        self.output_dir = folder_name
        os.makedirs(self.output_dir, exist_ok=True)
    
    def get_file_path(self, filename):
        return os.path.join(self.output_dir, filename)
    
    def write_simple_text(self):
        filename = f"output.txt"
        with open(self.get_file_path(filename), 'w', encoding='utf-8') as file:
            file.write('Hi\n')
            file.write('This is Serverless GPU\n')
            file.write('from Float16')
        return filename

def main():
    writer = FileWriter("my_output_files")
    
    print("processing...")
    
    simple_file = writer.write_simple_text()
    print(f"- write {simple_file} successfully")
    
    print(f"\nOutput Path: {os.path.abspath(writer.output_dir)}")

if __name__ == '__main__':
    main()

Step 4 : Copy Output to S3

Use the following command to copy your output to AWS S3

float16 storage copy-output \
    --path my_output_files/output.txt \
    --s3-uri <YOUR_S3_URI_DESTINATION> \
    --s3-access-key <YOUR_S3_ACCESS_KEY> \
    --s3-secret-key <YOUR_S3_SECRET_KEY> \
    --aws-region <YOUR_S3_REGION>

Replace the placeholders with your actual AWS S3 credentials:

<YOUR_S3_URI_DESTINATION>: Your S3 bucket destination path

<YOUR_S3_ACCESS_KEY>: Your AWS access key

<YOUR_S3_SECRET_KEY>: Your AWS secret key

<YOUR_S3_REGION>: Your S3 bucket region

After running the command, check your specified S3 destination path. You should find the output.txt file successfully copied.

Congratulations! You've learned how to copy output files from Float16's remote storage to your AWS S3 storage.