Run a project command

Start with live options

amics gpu options --gpus 1 --min-vram 24 --max-runtime 30m

The command returns independent options with exact GPU family, memory, location, live capacity depth, price, and quote expiry. Choose a family based on the workload; Amics does not rank unlike GPUs.

The table uses display labels. Remove spaces from the chosen label when passing its identifier to amics run (RTX 4090 becomes RTX4090).

All verified hosts are available by default, including community-operated machines. To restrict both discovery and acquisition to datacenter capacity, pass --host datacenter to both commands:

amics gpu options --host datacenter --min-vram 24 --max-runtime 30m
amics run --host datacenter [OPTIONS] -- uv run train.py

Community-operated hosts may have weaker physical and operational security. Use datacenter for sensitive source or forwarded environment values.

Execute

Amics uses interruptible Spot instances. Runs and services can stop before their runtime limit; --max-runtime and --ttl are upper bounds, not guaranteed uptime. For long-running work, save checkpoints to external storage or enable file recovery, and make your command resume from saved progress. Machine-local files can be lost; process and GPU memory are not restored.

amics run \
  --project fine-tune \
  --source . \
  --gpu RTX4090 --gpus 1 --min-vram 24 \
  --max-hourly-price 1.50 --max-runtime 30m \
  --yes \
  -- uv run train.py

The project name creates a stable reuse identity. The source defaults to the current directory.

Install Python libraries

For a project dependency, add it to the project locally so its manifest and lockfile are included in every snapshot:

uv add transformers
amics run [OPTIONS] -- uv run train.py

For a library needed by only one run, let uv provide it without changing the project:

amics run [OPTIONS] -- uv run --with transformers train.py

The remote runtime includes Python and uv. Installing into the remote source copy does not update the local project or a later snapshot.

Literal arguments and shell behavior

Arguments after -- are passed as literal process arguments. Pipes, redirects, globs, expansion, and command chaining do not happen implicitly.

Use a shell explicitly when shell behavior is part of the command:

amics run [OPTIONS] -- sh -lc 'uv run train.py | tee remote.log'

Source snapshots

Inside Git, the snapshot includes tracked and untracked files after standard ignore rules and .amicsignore. Amics excludes .git, common secret files, unsafe links, and traversal paths; .amicsignore can exclude a symlink before it is selected. Outside Git, traversal rejects any symlink before .amicsignore can filter it, so remove or relocate the link or snapshot a Git worktree instead.

Each run receives an immutable snapshot. Remote changes do not overwrite the local source tree and are not used in the next run.

Environment values

Forward values explicitly by local name:

export DATA_TOKEN=...
amics run [OPTIONS] --env DATA_TOKEN -- python train.py

Repeat --env for multiple names. Do not put secrets in command arguments.

Output modes

Only one of --quiet, --json, or --jsonl may be selected. Detaching does not extend the server deadline.

Default output includes lifecycle stages followed by command output. Quiet and structured modes keep their existing byte-exact or machine-readable contracts.

Reuse compute briefly

amics run [OPTIONS] --keep-warm 10m -- uv run train.py

A warm lease lets later compatible commands with the same project reuse the machine and its project cache. The lease expiry is calculated from each run's completion time, so repeating the full --keep-warm duration can extend the approved billing window. For multiple experiments, request the warm lease on the first run, capture its compute_id, and use that ID without --keep-warm on later runs:

amics run --compute COMPUTE_ID --max-runtime 10m -- uv run train.py

Keep --host datacenter on later runs when the workload requires that constraint.

It is bounded and can still end early if Spot capacity is interrupted. Without --keep-warm, auto-acquired compute stops after the command.

Use --compute COMPUTE_ID to run on compute you already control. Amics will not stop user-supplied compute implicitly.

Choose disk size

Use --storage GB for the attached disk on each machine. The default is 50 GB; you can request 1–10,000 GB, subject to live capacity. Include space for model weights, encoders, installed dependencies, temporary downloads, and outputs.

amics gpu options --min-vram 24 --storage 200
amics run --gpu RTX4090 --storage 200 --max-hourly-price 1.50 --max-runtime 30m -- ./generate.sh

Use the same size when checking prices and launching: the hourly quote and price ceiling include storage. amics serve also accepts --storage. A warm machine is reused only if its disk meets the request. This selects the size for a new machine; it does not resize an existing machine or make its files persistent. Transfer outputs before stopping compute.

Recover saved files after an interruption

When recovery is enabled on your platform, select directories and a saved-data cap. These options also work with amics serve (use --ttl for its runtime).

amics run --max-hourly-price 1.50 --max-runtime 2h --storage 100 \
  --save-path models --save-path state --max-saved-gb 50 \
  --save-every 5m --save-retention 24h -- python job.py

Amics saves changed file data and restores the last committed snapshot before restarting the original command on a replacement machine. Existing same-run files on a surviving machine are kept, including changes newer than the last backup. Write the command so it reuses completed downloads and reads its own progress. This can shorten recovery when restoring saved data costs less time than repeating downloads or work; it does not shorten machine provisioning.

Paths are source-relative directories. Add --save-workspace to preserve AMICS_WORKSPACE too. Only regular files and directories are supported; saved paths cannot overlap or contain symbolic links or special files. System packages, process memory, and GPU memory are not saved. Recovery requires auto-acquired compute without --compute or --keep-warm.

The default interval is 5 minutes (minimum 1 minute). Two completed snapshots are kept, with storage removed 24 hours after the run ends by default. Retention can be 1 hour to 7 days. --max-saved-gb caps encrypted saved data, including older unreferenced data retained until expiry, and the uncompressed files in any one snapshot. It is separate from --storage, the machine's disk size. When the cap is reached, backups stop succeeding and the previous committed snapshot remains available. Inspect amics runs get RUN_ID for recovery status, saved bytes, snapshot time, and errors.

Automatic backups copy live files; they are not application transactions. For a consistent checkpoint, pause writers, flush state to disk, then call the Python SDK from inside the job and keep writers paused until it returns:

from amics import checkpoint

# Pause writers and flush your application's checkpoint files first.
snapshot_id = checkpoint()  # Returns only after durable verification and commit.
# Writers may resume now.

The command receives AMICS_RESTORED=1 after a cold restore (0 otherwise), AMICS_RECOVERY_SNAPSHOT_ID, and a local AMICS_CHECKPOINT_SOCKET. Other languages can send save\n to that Unix socket and read one JSON line containing snapshot_id or error. Retries can repeat external effects such as API calls; make those operations idempotent. Runtime and service TTL limits are not extended by recovery, and files are not shared across separate runs.