Job Outputs

A job produces two kinds of output:

• Logs: stdout, stderr, and system events like job reassignment
• Artifacts: files written to /artifacts

Logs

Everything your code prints to stdout or stderr is captured while the job runs. The platform also generates system events for state changes like job start, completion, failure, and reassignment. Logs upload on a 600-second cadence, so live output is visible with a small delay rather than only at the end.

Artifacts

Artifacts are the output files a job produces. Anything written under /artifacts inside the container is captured and uploaded automatically as the job runs.

import os

output_path = "/artifacts/results.json"
with open(output_path, "w") as f:
    f.write('{"accuracy": 0.97}')

# Subdirectories are supported
os.makedirs("/artifacts/checkpoints")
torch.save(model.state_dict(), "/artifacts/checkpoints/epoch_10.pt")

Files upload as they're written, so you can view intermediate output via the Jobs page, cycle job pull, or cycle job tail. A file must stay unchanged for 500 milliseconds before it uploads, to avoid re-uploading half-written files. Once uploaded, it can take a few seconds to appear in the UI or be available for download.

Upload order

Artifacts upload one at a time to avoid splitting bandwidth across many files. By default, files upload in the order they were written (FIFO). This works well for most jobs.

If your job saves periodic checkpoints, the default ordering can cause problems: each checkpoint is large, and by the time one finishes uploading the next has already been written. The upload queue grows, and the latest checkpoint (the one you actually care about) sits behind older ones.

Set artifact_upload_order = "LIFO" in your manifest to upload the most recent file first:

[job]
artifact_upload_order = "LIFO"

With LIFO, the newest checkpoint uploads immediately and older ones fill in afterward using whatever bandwidth is left. Nothing is skipped; the order just changes.

If the upload queue stays non-empty for more than 600 seconds, a warning banner appears on the job page and in the CLI when you run job pull or job tail suggesting you switcch to LIFO.

Resuming from artifacts

If a job gets reassigned to a different machine, the new machine downloads everything already in /artifacts before starting the container. Your code sees the same files it wrote during the previous run.

This means you should structure your program to resume from /artifacts:

1. Save checkpoints to /artifacts periodically (model weights, optimizer state, step count)
2. On startup, check if a checkpoint already exists and load it instead of starting from scratch

import os
from pathlib import Path

CHECKPOINT_DIR = Path("/artifacts/checkpoints")
CHECKPOINT_DIR.mkdir(parents=True, exist_ok=True)

def find_latest_checkpoint():
    if not CHECKPOINT_DIR.exists():
        return None
    checkpoints = sorted(CHECKPOINT_DIR.glob("step-*.pt"))
    return checkpoints[-1] if checkpoints else None

# At startup
resume_from = find_latest_checkpoint()
if resume_from:
    state = torch.load(resume_from)
    model.load_state_dict(state["model"])
    optimizer.load_state_dict(state["optimizer"])
    start_step = state["step"]
else:
    start_step = 0

If you skip this, a reassigned job starts over from the beginning every time it moves to a new machine. See Adapting Your Code for more info on writing resumable jobs.

Viewing in the dashboard

On the Jobs page, click a job's name or the folder icon next to it to open the job browser. It has two tabs:

Logs shows stdout, stderr, and system events interleaved by timestamp. stderr lines appear in red; system events appear in gray italics with a [system] tag. Scroll up to load older log entries. The viewer starts at the bottom (most recent output) and fetches earlier pages on demand.

Artifacts is a file browser. You can navigate folders, sort by name or size, and download individual files. Files are available for download as they finish uploading from the container.

Viewing from the CLI

To stream live logs to stdout and download new artifacts as they appear:

cycle job tail my-job

To download all logs and artifacts at once:

cycle job pull my-job

Both commands save files to <project-root>/.cycleswap/jobs/<job-name>/, with logs in output.log and artifacts in artifacts/.