Jobs (CPU + GPU compute)

Jobs are deployed, named, async compute. Submit returns immediately with a run guid; the work runs on a sandbox container (CPU) or on Modal (GPU). Use jobs for anything that doesn't fit in a function (functions are 30 s / 128 MB; jobs go up to 60 min and 8 GB CPU / a full GPU).

Common shapes:

• Audio / video pipelines (ffmpeg, sox, whisper)
• ML inference that needs a GPU (Flux, LTX-Video, protein folding, transcription)
• Fine-tuning / training runs
• Batch processing (OCR, document conversion, embedding millions of items)
• Long ETL that exceeds function limits

Declaring a job

Add a jobs: phase to gipity.yaml:

version: 1
deploy:
  phases:
    - name: my-jobs
      type: jobs
      job_definitions:
        - name: transcribe
          handler: jobs/transcribe/main.py
          runtime: python-3.11
          compute: gpu-small        # L4 GPU
          timeout_ms: 300000        # 5 minutes
          deps: jobs/transcribe/requirements.txt
          retries: 1
          description: "Whisper transcription of an audio file"

Then drop the handler at jobs/transcribe/main.py. Deploy with gipity deploy dev.

Inline annotations (optional)

Instead of declaring everything in gipity.yaml, you can write # @gipity:job at the top of the handler:

# @gipity:job runtime=python-3.11 compute=gpu-small timeout=5m
import whisper
# ...

YAML wins on conflict; annotations seed defaults only.

Compute classes

Each class maps to a substrate + resource shape. Billed per second based on actual wall time of the run (deps install included).

timeout_ms caps the run inside the container (max 60 min). Modal kills the underlying call if our soft deadline is exceeded.

Runtimes & the fat image

Pick a runtime with runtime: (or the # @gipity:job runtime=... annotation). Valid values: python-3.11, node-24 (Node 24; node-20 is still accepted as a back-compat alias for the same runtime), and bash. Omit it and it's inferred from the handler extension (.py->python, .js->node-24, .sh->bash).

Node is CPU-only. The GPU image ships Python and Bash but no Node, so a node-* runtime must pair with a cpu-* compute class. Declaring runtime: node-24 (or node-20) with a gpu-* compute is rejected at deploy and at submit with a clear error - for GPU work, use python-3.11.

GPU runs execute in a Modal container preloaded with:

• System: ffmpeg, sox (incl. mp3/ogg/flac via libsox-fmt-all), imagemagick (PDF coder enabled), poppler-utils, tesseract, build-essential, git, git-lfs, libsndfile, libgl
• Python: torch, transformers, accelerate, diffusers, peft, datasets, safetensors, sentencepiece, tokenizers, librosa, soundfile, openai-whisper, demucs, opencv-python-headless, scikit-image, numpy, scipy, pandas, pillow, httpx, pydantic, requests, boto3, huggingface_hub
• Plus torchaudio (bundled in the pytorch base) - so torchaudio.pipelines.MMS_FA_BUNDLE works without pip install

If your job only needs the above, omit deps: entirely - the container starts faster.

CPU runs use the sandbox image (web-app-basics for the full list).

Persistent cache

Both substrates mount a project-shared volume at /cache. HF/torch model weights cache there by default (env: HF_HOME=/cache/hf, TORCH_HOME=/cache/torch). First job downloads the weights; subsequent jobs are instant.

Use /cache/<your-key>/ for anything you want persisted across runs (intermediate artifacts, downloaded datasets). Don't put per-run output there - use app-files to upload final results.

Handler contract

The handler is invoked with these env vars set:

Identical on CPU and GPU. Every var above - plus the gipity_ctx helper (gipity_ctx.py / gipity_ctx.js, see below) - is injected the same way on the Docker (CPU) and Modal (GPU) substrates, so set_output, progress, and the output channel work on a gpu-* job exactly as on a cpu-* one; don't gate any of this on compute class.

Job output contract

Set your result with ctx.set_output(...) - don't print it to stdout. stdout and stderr are captured as logs (streamed live via gipity job logs <runGuid>, and the final stdout is stored on output only as a fallback). The clean way to return a structured result is the dedicated output channel:

from gipity_ctx import ctx
ctx.set_output({"stems": stem_urls})   # this becomes the run's output

const { ctx } = require('./gipity_ctx');
await ctx.set_output({ stems: stemUrls });

set_output blocks until the platform has stored the result and throws if it can't, so a lost result fails loudly instead of vanishing. Whatever you pass is exactly what an on_complete hook receives as output - independent of stdout, so you can print()/console.log diagnostics and let libraries log freely without corrupting it. Call it once; the last call wins.

256 KB cap. The serialized output must be ≤256 KB; a larger payload is rejected (HTTP 413 RESULT_TOO_LARGE) and set_output throws. For a big result (large PDB/CSV/JSON), upload it via app-files and set_output the URL, or compress to fit (e.g. gzip+base64). Logs (stdout/stderr) share the same 256 KB cap; the live log stream is unbounded.

Before / after - the trap set_output removes:

# ❌ Before: result on stdout. One stray log line (yours or a library's) and the
# platform can't parse stdout as JSON, so output becomes {stdout: "...logs..."}
# and your on_complete hook sees output.stems as undefined.
print("loading model…")            # pollutes stdout
print(json.dumps({"stems": urls})) # intended result

# ✅ After: result on its own channel; logs go anywhere.
print("loading model…")            # just a log - harmless
ctx.set_output({"stems": urls})    # the result

If a job never calls set_output, the platform still falls back to parsing stdout - the whole stream as one JSON value, or a single JSON object/array at the very end - and otherwise stores the raw stream as {stdout: "..."}. Prefer set_output; the fallback is best-effort.

stderr is captured but not streamed live (yet); it surfaces in error_message on failure.

Progress callbacks

POST to $GIPITY_PROGRESS_URL with {"pct": 0.5, "message": "halfway"} - pct is 0-1 (not 0-100); a value >1 is rejected with HTTP 400. Updates progress_pct and progress_message on the run row. Best-effort: log any failure to stderr, never stdout (stdout is the result).

gipity_ctx is the blessed pattern - prefer it over hand-rolling these POSTs. The helper (gipity_ctx.py for python, gipity_ctx.js for node) is injected alongside the handler on both substrates, so use it on GPU jobs too:

from gipity_ctx import ctx
ctx.progress(0.5, "halfway")
ctx.set_output({"pdb_url": url})

(An older handler that hand-rolls the raw $GIPITY_PROGRESS_URL / $GIPITY_OUTPUT_URL POSTs predates SDK injection on Modal - not the pattern to copy.)

CLI surface

gipity job list                          # all jobs in current project
gipity job submit <name> [body]          # returns a run guid
gipity job status <runGuid>              # snapshot
gipity job runs <name> [--limit 20]      # recent runs of one job
gipity job logs <runGuid> [--follow]     # SSE stream by default
gipity job cancel <runGuid>              # queued or running -> cancelled
gipity job run-local <name>              # Docker-on-your-machine for dev iteration

Submit accepts --data '<json>' for input (or pass a JSON arg directly) and --idempotency-key <key> for replay.

Agent tools

The same six verbs are exposed as agent tools, with the same parameter shapes:

• job_list (VIEWER) - list jobs in current project
• job_submit (EDITOR) - submit a job; returns {run_guid, status, replayed}
• job_status (VIEWER) - short status snapshot
• job_runs (VIEWER) - recent runs of a named job
• job_logs (VIEWER) - captured stdout / stderr / output, truncated at 5 KB
• job_cancel (EDITOR) - flip a queued/running run to cancelled

Agents and the CLI hit the same /projects/{guid}/jobs/... endpoints; their behavior matches.

REST endpoints (project-scoped, JWT auth)

Submit body: { "input": {...}, "idempotency_key": "..." }.

In-function jobs API

A deployed function handler submits and polls jobs through ctx.jobs - no HTTP layer, same concurrency caps / GPU gating / idempotency replay as the CLI. All three throw on error:

const { run_guid, status, replayed } = await ctx.jobs.submit('fold', { seq }, { idempotency_key: 'opt' });
const s = await ctx.jobs.status(run_guid);
// s -> { run_guid, status, progress_pct, progress_message, attempt, started_at, completed_at, duration_ms, error, output }
//   output is populated once status === 'success' - the value the handler passed to set_output.
await ctx.jobs.cancel(run_guid);   // -> { status }

A function returns in ≤30 s - don't poll a long job to completion inside it. Submit and return run_guid, or set on_complete (below) to fire a function when the run finishes.

State machine

queued → running → success | failed | cancelled

Status writes are atomic - only one transition wins. Retries (retries: N in the YAML) re-claim a failed row back to running for the next attempt, with exponential backoff. Deps-install failures are terminal - no retries (the user's setup is wrong; retrying won't help).

Failed and cancelled runs are billed for the time they actually ran (Modal / Docker charged us). Successful runs are billed for the same.

Billing

Per-second metering by compute class. Underlying USD/sec rates track Modal's published prices; 100 % platform margin applied. Both lines visible on the credits Pricing tab (Jobs category).

A 30-second L4 (gpu-small) job at 100 % margin ≈ $0.013 in credits (Modal's $0.80 / hr × 30 s × 2).

Auto-scaling: parallel jobs run in parallel

Submit N jobs concurrently and they execute concurrently. Modal's worker fleet spins up a fresh container per submission within seconds (cached image), runs your handler, and tears down - no per-job queue, each submission gets its own container.

The only ceiling is the project-level cap of 10 in-flight jobs (queued + running). If you submit an 11th while 10 are already in flight you get RATE_LIMITED (HTTP 429); wait for some to finish. This cap is per project, not per compute class - 5 L4s and 5 H100s in flight = cap reached.

Practical implications:

• Batch workloads: submit 10 jobs in a tight loop and watch them all start. Modal handles the fan-out at the substrate layer.
• Warm-pool freebie: containers are kept alive ~60 s after a job finishes, so a second submission within that window starts in <5 s instead of paying the ~10-30 s cold-start.
• Image is cached fleet-wide: the 15GB fat image is pulled to each Modal worker once. Steady-state cold starts are bounded by Python interpreter + your handler's imports.
• What we don't expose yet: fn.map() fan-out (one submission → N parallel runs), min_containers (keep N warm at idle cost), per-container concurrent_inputs for IO-bound serving. All are Modal-native and would land in future PRs if you hit the use case.

Limits

• Max concurrent (queued + running) jobs per project: 10
• Max timeout_ms: 60 min (3,600,000)
• Max retries: 10
• Default 5-min deps install timeout
• Result stdout / stderr capped at 256 KB (truncation marker preserved). Live log stream is unbounded - store large outputs via app-files.

Common patterns

Audio job that reads input + uploads result

import json, os, urllib.request

run_input = json.loads(os.environ['GIPITY_RUN_INPUT'])
input_url = run_input['audio_url']

# Download to /tmp, process with ffmpeg or whisper or whatever
# ...

# Upload result via the app's file service (project-scoped JWT)
app_token = os.environ['GIPITY_APP_TOKEN']
app_guid  = os.environ['GIPITY_APP_GUID']
api_base  = os.environ['GIPITY_API_BASE']
# (use app-files skill for the upload flow)

print(json.dumps({"transcript_url": "..."}))

Fan-out via on_complete

- name: align
  handler: jobs/align/main.py
  compute: gpu-small
  on_complete: render-preview     # name of a deployed function

When the align job hits a terminal state the platform fires the named function with { run_guid, status, output, error }. The render function can then submit a new render job.

Testing a job end-to-end

A job needs a public input URL. Make one without any external host: build a fixture in the sandbox (it syncs back to project files), publish it via gipity deploy dev, then submit and watch.

gipity sandbox run bash 'ffmpeg -f lavfi -i sine=frequency=440:duration=5 /work/test.wav'  # fixture -> project files
gipity deploy dev                              # served at https://dev.gipity.ai/<acct>/<proj>/test.wav
gipity job submit <name> --data '{"audio_url":"https://dev.gipity.ai/<acct>/<proj>/test.wav"}'
# or, if a function starts the job: gipity fn call <trigger-fn> '{"audio_url":"…"}'
gipity job status <runGuid>
gipity job logs <runGuid> --follow

First GPU run downloads model weights to /cache (tens of seconds to minutes) - not a hang; subsequent runs are instant.

Common gotchas

• GPU without Modal config - submit returns MODAL_NOT_CONFIGURED (503). Operator must set MODAL_APP_URL + MODAL_INVOKE_SECRET on the server.
• Deps that conflict with the fat image - e.g. pinning torch==2.3 when the image ships a different minor. requirements.txt wins via pip override, but CUDA/torch mismatches may produce subtle bugs. Prefer no deps: if the fat image already has what you need.
• Forgetting set -e in bash - without it, only the LAST command's exit code propagates.
• output is whatever you set_output - for an agent or function calling job_submit, the run's output (and what an on_complete hook receives) is the value the handler passed to ctx.set_output(...). See "Job output contract" above.

See also

• deploy - how the jobs phase fits into the rest of the deploy pipeline
• app-development - functions vs jobs, when to use which
• app-files - uploading artifacts the job produces
• app-debugging - page inspect, function logs, run logs