Adding SFTP abilities to Namespace.so's ephemeral instances

Update (February 2026): Namespace has since added native file upload support to their CLI. You can now use nsc instance upload to copy files directly to an instance, making the shell scripts in this post unnecessary. This post is kept for historical purposes.

Namespace.so provides ephemeral Linux and macOS instances, primarily for CI/CD workflows. Like Fly.io, it lets me avoid managing servers.

I use it for some of my CI/CD workflows and wanted to try it for other tasks. One idea is an on-demand nixpkgs builder, similar to my Fly.io setup. With Fly.io I manage the instances myself, while Namespace terminates them after a set period. That reduces the monthly hosting spend.

At the time I wrote this, Namespace did not support uploading files directly to instances. I wanted something I could use immediately, which turned into a small side project.

I asked in the Namespace Discord and was pointed to croc, which others had used. I wanted an option that did not require another tool on the remote server as well as locally. There are plenty of other ways to move the files, but I like making little Bash tools for myself, and this was a good excuse to play around.

Unlike Fly.io, Namespace does not provide direct SSH access, so I needed a way to send files through its existing command interface. The files might be configuration, binaries, or other inputs for whatever I was running.

The nsc client supports pseudo-shell sessions and arbitrary commands. My first pass copied a small file with nsc ssh $machine_id 'echo "hi" > /root/test.txt', but that does not work well for binary files or large text files. Terminal length limits also meant I had to split files into chunks and reassemble them on the remote machine. Each chunk establishes a new connection, so the transfer is slower than a direct copy. Parallelizing those connections can reduce the transfer time, but that was outside the initial scope.

#!/bin/bash

# help text
usage() {
    echo "Usage: $0 -l <local_file> -r <remote_file> -m <machine_id> [-c <chunk_size>]"
    echo ""
    echo "This Script uses the Namespace Client 'nsc' to copy files to your instance"
    echo "Note: You'll need to ensure you've logged in with 'nsc login' first"
    echo ""
    echo "Options:"
    echo "  -l <local_file>   Path to the local file to be transferred"
    echo "  -r <remote_file>  Path to the remote file to be created"
    echo "  -m <machine_id>   Machine ID for the nsc ssh command"
    echo "  -c <chunk_size>   Size of the chunks for splitting the base64 encoded file (default: 1k)"
    echo "  -h, --help        Show this help message and exit"
    exit 1
}

chunk_size="1k" # default chunk size

# parse args
while getopts ":l:r:m:c:h" opt; do
    case ${opt} in
        l )
            local_file=$OPTARG
            ;;
        r )
            remote_file=$OPTARG
            ;;
        m )
            machine_id=$OPTARG
            ;;
        c )
            chunk_size=$OPTARG
            ;;
        h )
            usage
            ;;
        \? )
            usage
            ;;
    esac
done

# check for (full) help flag
for arg in "$@"; do
    if [ "$arg" == "--help" ]; then
        usage
    fi
done

# make sure all args are set
if [ -z "${local_file}" ] || [ -z "${remote_file}" ] || [ -z "${machine_id}" ]; then
    usage
fi

#  base64 encode file and split into chunks
cat $local_file | base64 > /tmp/local_file.b64
split -l $chunk_size /tmp/local_file.b64 /tmp/chunk_

# init remote file
nsc ssh $machine_id "echo -n '' > /tmp/remote_file.b64"

# FIXME: transfer chunks individually, then reassemble them on server (this would allow for parallelization)
# loop over chunks and send each one
for chunk in /tmp/chunk_*; do
    chunk_content=$(cat $chunk)
    nsc ssh $machine_id "echo -n '$chunk_content' >> /tmp/remote_file.b64"
done

# decode base64 file
nsc ssh $machine_id "base64 -d /tmp/remote_file.b64 > $remote_file"

# clean up local temp files
rm /tmp/local_file.b64 /tmp/chunk_*

echo "File transferred successfully."

Run it like this:

# ensure you are logged into namespace.so
nsc login

# create a new ephemeral instance (4 cores, 8gb ram)
machine_id=$(nsc create --machine_type 4x8 --bare --output json | jq -r .cluster_id)

# copy file up to server (assuming you have chmod +x the script already)
./transfer_file.sh -l /home/tklk/Photos/nyan_cat.gif -r /root/nyan.gif -m $machine_id

Another option is to install Tailscale in the ephemeral instance and use tailscale file cp. I left that for another day.

Update:

After writing this post, I added parallelization with GNU parallel to send the chunks to the remote machine. This reduced the transfer time. You need parallel installed on the local machine because several common operating systems do not include it by default. Replace the transfer section of the script above with this version:

nsc ssh $machine_id "mkdir -p /tmp/chunks && echo -n '' > /tmp/remote_file.b64"
send_chunk() {
    chunk=$1
    chunk_name=$(basename $chunk)
    chunk_content=$(cat $chunk)
    nsc ssh $machine_id "echo -n '$chunk_content' > /tmp/chunks/$chunk_name"
}
export -f send_chunk
export machine_id

# use gnu parallel to transfer files
# FIXME: accept -j as an argument to be able to adjust the hardcoded number
#        left as an exercise for the reader
find /tmp/chunk_* | parallel -j 4 send_chunk

# reassemble chunks
nsc ssh $machine_id "cat /tmp/chunks/* > /tmp/remote_file.b64 && rm -r /tmp/chunks"

# based64 decode and write to destination path
nsc ssh $machine_id "base64 -d /tmp/remote_file.b64 > $remote_file && rm /tmp/remote_file.b64"