# Jobs - long-running CPU & GPU compute Jobs are deployed, named, async compute. Submit returns immediately with a run guid; the work runs on a sandbox container (CPU) or on Modal (GPU). Use jobs for anything that doesn't fit in a function (functions are 30 s / 128 MB; jobs go up to 60 min and 8 GB CPU / a full GPU). Common shapes: - **Audio / video pipelines** (ffmpeg, sox, whisper) - **ML inference** that needs a GPU (Flux, LTX-Video, protein folding, transcription) - **Fine-tuning / training** runs - **Batch processing** (OCR, document conversion, embedding millions of items) - **Long ETL** that exceeds function limits ## Declaring a job Add a `jobs:` phase to `gipity.yaml`: ```yaml version: 1 deploy: phases: - name: my-jobs type: jobs job_definitions: - name: transcribe handler: jobs/transcribe/main.py runtime: python-3.11 compute: gpu-small # L4 GPU timeout_ms: 300000 # 5 minutes deps: jobs/transcribe/requirements.txt retries: 1 description: "Whisper transcription of an audio file" ``` Then drop the handler at `jobs/transcribe/main.py`. Deploy with `gipity deploy dev`. ### Inline annotations (optional) Instead of declaring everything in `gipity.yaml`, you can write `# @gipity:job` at the top of the handler: ```python # @gipity:job runtime=python-3.11 compute=gpu-small timeout=5m import whisper # ... ``` YAML wins on conflict; annotations seed defaults only. ## Compute classes Each class maps to a substrate + resource shape. Billed **per second** based on actual wall time of the run (deps install included). | Class | Substrate | RAM | Notes | |---|---|---|---| | `cpu-small` | Docker sandbox host | 1 GB | Default. Fine for ffmpeg / pandoc / pip-installed Python work | | `cpu-large` | Docker sandbox host | 8 GB | Heavy CPU work - large encodes, document batches, ETL | | `gpu-small` | Modal L4 (24 GB VRAM) | per Modal | Most inference; the cheapest GPU | | `gpu-medium` | Modal A10G (24 GB VRAM) | per Modal | Slightly faster than L4 for some workloads | | `gpu-large` | Modal A100-40GB | per Modal | Large model training / fine-tuning | | `gpu-huge` | Modal H100 | per Modal | Heaviest training, big batched inference | `timeout_ms` caps the run inside the container (max 60 min). Modal kills the underlying call if our soft deadline is exceeded. ## Runtimes & the fat image Pick a runtime with `runtime:` (or the `# @gipity:job runtime=...` annotation). Valid values: **`python-3.11`**, **`node-24`** (Node 24; `node-20` is still accepted as a back-compat alias for the same runtime), and **`bash`**. Omit it and it's inferred from the handler extension (`.py`->python, `.js`->node-24, `.sh`->bash). **Node is CPU-only.** The GPU image ships Python and Bash but no Node, so a `node-*` runtime must pair with a `cpu-*` compute class. Declaring `runtime: node-24` (or `node-20`) with a `gpu-*` compute is rejected at deploy and at submit with a clear error - for GPU work, use `python-3.11`. GPU runs execute in a Modal container preloaded with: - **System**: ffmpeg, sox (incl. mp3/ogg/flac via libsox-fmt-all), imagemagick (PDF coder enabled), poppler-utils, tesseract, build-essential, git, git-lfs, libsndfile, libgl - **Python**: torch, transformers, accelerate, diffusers, peft, datasets, safetensors, sentencepiece, tokenizers, librosa, soundfile, openai-whisper, **demucs**, opencv-python-headless, scikit-image, numpy, scipy, pandas, pillow, httpx, pydantic, requests, boto3, huggingface_hub - Plus torchaudio (bundled in the pytorch base) - so `torchaudio.pipelines.MMS_FA_BUNDLE` works without pip install If your job only needs the above, **omit `deps:`** entirely - the container starts faster. CPU runs use the sandbox image (`web-app-basics` for the full list). ## Persistent cache Both substrates mount a project-shared volume at `/cache`. HF/torch model weights cache there by default (env: `HF_HOME=/cache/hf`, `TORCH_HOME=/cache/torch`). First job downloads the weights; subsequent jobs are instant. Use `/cache//` for anything you want persisted across runs (intermediate artifacts, downloaded datasets). Don't put per-run output there - use `app-files` to upload final results. ## Handler contract The handler is invoked with these env vars set: | Var | Use | |---|---| | `GIPITY_RUN_GUID` | Pass to log/progress callbacks | | `GIPITY_RUN_INPUT` | JSON of the `input` arg from submit | | `GIPITY_APP_GUID` | Project short_guid | | `GIPITY_APP_TOKEN` | JWT for calling `/api/{appGuid}/services/*` | | `GIPITY_PROGRESS_URL` | POST progress updates here | | `GIPITY_OUTPUT_URL` | POST your structured result here (or use `ctx.set_output`) | | `GIPITY_PROGRESS_TOKEN` | Auth token for progress + output + log endpoints | | `GIPITY_API_BASE` | Base URL (e.g. `https://a.gipity.ai`) | **Identical on CPU and GPU.** Every var above - plus the `gipity_ctx` helper (`gipity_ctx.py` / `gipity_ctx.js`, see below) - is injected the same way on the Docker (CPU) and Modal (GPU) substrates, so `set_output`, `progress`, and the output channel work on a `gpu-*` job exactly as on a `cpu-*` one; don't gate any of this on compute class. ### Job output contract **Set your result with `ctx.set_output(...)` - don't print it to stdout.** stdout and stderr are captured as **logs** (streamed live via `gipity job logs `, and the final stdout is stored on `output` only as a fallback). The clean way to return a structured result is the dedicated output channel: ```python from gipity_ctx import ctx ctx.set_output({"stems": stem_urls}) # this becomes the run's output ``` ```js const { ctx } = require('./gipity_ctx'); await ctx.set_output({ stems: stemUrls }); ``` `set_output` blocks until the platform has stored the result and throws if it can't, so a lost result fails loudly instead of vanishing. Whatever you pass is exactly what an `on_complete` hook receives as `output` - **independent of stdout**, so you can `print()`/`console.log` diagnostics and let libraries log freely without corrupting it. Call it once; the last call wins. **256 KB cap.** The serialized output must be ≤256 KB; a larger payload is rejected (HTTP 413 `RESULT_TOO_LARGE`) and `set_output` throws. For a big result (large PDB/CSV/JSON), upload it via `app-files` and `set_output` the URL, or compress to fit (e.g. gzip+base64). Logs (stdout/stderr) share the same 256 KB cap; the live log stream is unbounded. **Before / after** - the trap `set_output` removes: ```python # ❌ Before: result on stdout. One stray log line (yours or a library's) and the # platform can't parse stdout as JSON, so output becomes {stdout: "...logs..."} # and your on_complete hook sees output.stems as undefined. print("loading model…") # pollutes stdout print(json.dumps({"stems": urls})) # intended result # ✅ After: result on its own channel; logs go anywhere. print("loading model…") # just a log - harmless ctx.set_output({"stems": urls}) # the result ``` If a job never calls `set_output`, the platform still falls back to parsing stdout - the whole stream as one JSON value, or a single JSON object/array at the very end - and otherwise stores the raw stream as `{stdout: "..."}`. Prefer `set_output`; the fallback is best-effort. stderr is captured but not streamed live (yet); it surfaces in `error_message` on failure. ### Progress callbacks POST to `$GIPITY_PROGRESS_URL` with `{"pct": 0.5, "message": "halfway"}` - `pct` is 0-1 (not 0-100); a value >1 is rejected with HTTP 400. Updates `progress_pct` and `progress_message` on the run row. Best-effort: log any failure to stderr, never stdout (stdout is the result). **`gipity_ctx` is the blessed pattern** - prefer it over hand-rolling these POSTs. The helper (`gipity_ctx.py` for python, `gipity_ctx.js` for node) is injected alongside the handler on **both** substrates, so use it on GPU jobs too: ```python from gipity_ctx import ctx ctx.progress(0.5, "halfway") ctx.set_output({"pdb_url": url}) ``` (An older handler that hand-rolls the raw `$GIPITY_PROGRESS_URL` / `$GIPITY_OUTPUT_URL` POSTs predates SDK injection on Modal - not the pattern to copy.) ## CLI surface ```bash gipity job list # all jobs in current project gipity job submit [body] # returns a run guid gipity job status # snapshot gipity job runs [--limit 20] # recent runs of one job gipity job logs [--follow] # SSE stream by default gipity job cancel # queued or running -> cancelled gipity job run-local # Docker-on-your-machine for dev iteration ``` Submit accepts `--data ''` for input (or pass a JSON arg directly) and `--idempotency-key ` for replay. ## Agent tools The same six verbs are exposed as agent tools, with the same parameter shapes: - `job_list` (VIEWER) - list jobs in current project - `job_submit` (EDITOR) - submit a job; returns `{run_guid, status, replayed}` - `job_status` (VIEWER) - short status snapshot - `job_runs` (VIEWER) - recent runs of a named job - `job_logs` (VIEWER) - captured stdout / stderr / output, truncated at 5 KB - `job_cancel` (EDITOR) - flip a queued/running run to cancelled Agents and the CLI hit the same `/projects/{guid}/jobs/...` endpoints; their behavior matches. ## REST endpoints (project-scoped, JWT auth) | Method | Path | Purpose | |---|---|---| | `GET` | `/projects/{guid}/jobs` | list jobs | | `POST` | `/projects/{guid}/jobs/{name}/submit` | submit (returns runGuid) | | `GET` | `/projects/{guid}/jobs/runs/{runGuid}` | one run's status | | `GET` | `/projects/{guid}/jobs/{name}/runs` | list runs of one job | | `GET` | `/projects/{guid}/jobs/runs/{runGuid}/logs/stream` | SSE log stream | | `POST` | `/projects/{guid}/jobs/runs/{runGuid}/cancel` | best-effort cancel | Submit body: `{ "input": {...}, "idempotency_key": "..." }`. ### In-function jobs API A deployed function handler submits and polls jobs through `ctx.jobs` - no HTTP layer, same concurrency caps / GPU gating / idempotency replay as the CLI. All three throw on error: ```js const { run_guid, status, replayed } = await ctx.jobs.submit('fold', { seq }, { idempotency_key: 'opt' }); const s = await ctx.jobs.status(run_guid); // s -> { run_guid, status, progress_pct, progress_message, attempt, started_at, completed_at, duration_ms, error, output } // output is populated once status === 'success' - the value the handler passed to set_output. await ctx.jobs.cancel(run_guid); // -> { status } ``` A function returns in ≤30 s - don't poll a long job to completion inside it. Submit and return `run_guid`, or set `on_complete` (below) to fire a function when the run finishes. ## State machine `queued → running → success | failed | cancelled` Status writes are atomic - only one transition wins. Retries (`retries: N` in the YAML) re-claim a `failed` row back to `running` for the next attempt, with exponential backoff. Deps-install failures are **terminal** - no retries (the user's setup is wrong; retrying won't help). Failed and cancelled runs are billed for the time they actually ran (Modal / Docker charged us). Successful runs are billed for the same. ## Billing Per-second metering by compute class. Underlying USD/sec rates track Modal's published prices; 100 % platform margin applied. Both lines visible on the credits Pricing tab (Jobs category). A 30-second L4 (`gpu-small`) job at 100 % margin ≈ $0.013 in credits (Modal's $0.80 / hr × 30 s × 2). ## Auto-scaling: parallel jobs run in parallel Submit N jobs concurrently and they execute concurrently. Modal's worker fleet spins up a fresh container per submission within seconds (cached image), runs your handler, and tears down - no per-job queue, each submission gets its own container. The only ceiling is the **project-level cap of 10 in-flight jobs** (queued + running). If you submit an 11th while 10 are already in flight you get `RATE_LIMITED` (HTTP 429); wait for some to finish. This cap is per project, not per compute class - 5 L4s and 5 H100s in flight = cap reached. Practical implications: - **Batch workloads**: submit 10 jobs in a tight loop and watch them all start. Modal handles the fan-out at the substrate layer. - **Warm-pool freebie**: containers are kept alive ~60 s after a job finishes, so a second submission within that window starts in <5 s instead of paying the ~10-30 s cold-start. - **Image is cached fleet-wide**: the 15GB fat image is pulled to each Modal worker once. Steady-state cold starts are bounded by Python interpreter + your handler's imports. - **What we don't expose yet**: `fn.map()` fan-out (one submission → N parallel runs), `min_containers` (keep N warm at idle cost), per-container `concurrent_inputs` for IO-bound serving. All are Modal-native and would land in future PRs if you hit the use case. ## Limits - Max concurrent (queued + running) jobs per project: 10 - Max `timeout_ms`: 60 min (3,600,000) - Max retries: 10 - Default 5-min deps install timeout - Result stdout / stderr capped at 256 KB (truncation marker preserved). Live log stream is unbounded - store large outputs via `app-files`. ## Common patterns ### Audio job that reads input + uploads result ```python import json, os, urllib.request run_input = json.loads(os.environ['GIPITY_RUN_INPUT']) input_url = run_input['audio_url'] # Download to /tmp, process with ffmpeg or whisper or whatever # ... # Upload result via the app's file service (project-scoped JWT) app_token = os.environ['GIPITY_APP_TOKEN'] app_guid = os.environ['GIPITY_APP_GUID'] api_base = os.environ['GIPITY_API_BASE'] # (use app-files skill for the upload flow) print(json.dumps({"transcript_url": "..."})) ``` ### Fan-out via on_complete ```yaml - name: align handler: jobs/align/main.py compute: gpu-small on_complete: render-preview # name of a deployed function ``` When the align job hits a terminal state the platform fires the named function with `{ run_guid, status, output, error }`. The render function can then submit a new render job. ## Testing a job end-to-end A job needs a public input URL. Make one without any external host: build a fixture in the sandbox (it syncs back to project files), publish it via `gipity deploy dev`, then submit and watch. ```bash gipity sandbox run bash 'ffmpeg -f lavfi -i sine=frequency=440:duration=5 /work/test.wav' # fixture -> project files gipity deploy dev # served at https://dev.gipity.ai///test.wav gipity job submit --data '{"audio_url":"https://dev.gipity.ai///test.wav"}' # or, if a function starts the job: gipity fn call '{"audio_url":"…"}' gipity job status gipity job logs --follow ``` First GPU run downloads model weights to `/cache` (tens of seconds to minutes) - not a hang; subsequent runs are instant. ## Common gotchas - **GPU without Modal config** - submit returns `MODAL_NOT_CONFIGURED` (503). Operator must set `MODAL_APP_URL` + `MODAL_INVOKE_SECRET` on the server. - **Deps that conflict with the fat image** - e.g. pinning `torch==2.3` when the image ships a different minor. `requirements.txt` wins via pip override, but CUDA/torch mismatches may produce subtle bugs. Prefer no `deps:` if the fat image already has what you need. - **Forgetting `set -e` in bash** - without it, only the LAST command's exit code propagates. - **`output` is whatever you `set_output`** - for an agent or function calling `job_submit`, the run's `output` (and what an `on_complete` hook receives) is the value the handler passed to `ctx.set_output(...)`. See "Job output contract" above. ## See also - [deploy](deploy.md) - how the jobs phase fits into the rest of the deploy pipeline - [app-development](app-development.md) - functions vs jobs, when to use which - [app-files](app-files.md) - uploading artifacts the job produces - [app-debugging](app-debugging.md) - `page inspect`, function logs, run logs