{
  "name": "jobs",
  "title": "Jobs (CPU + GPU compute)",
  "description": "Long-running async jobs - CPU or GPU via Gipity Jobs. Submit, poll, stream logs, retry, cancel. Per-second billing by compute class.",
  "guid": "sk_plat_jobs",
  "category": "App development",
  "requiredTools": [
    "job_list",
    "job_submit",
    "job_status",
    "job_runs",
    "job_logs",
    "job_cancel"
  ],
  "content": "# Jobs - long-running CPU & GPU compute\n\nJobs are deployed, named, async compute. Submit returns immediately with a run guid; the work runs on a sandbox container (CPU) or on Modal (GPU). Use jobs for anything that doesn't fit in a function (functions are 30 s / 128 MB; jobs go up to 60 min and 8 GB CPU / a full GPU).\n\nCommon shapes:\n- **Audio / video pipelines** (ffmpeg, sox, whisper)\n- **Image-to-video** - animate/\"bring to life\" a user's uploaded photo (LTX-Video, Stable Video Diffusion). This is the GPU-job path for image-conditioned motion; the [app-video](app-video.md) service is text-to-prompt only and can't preserve a specific image.\n- **ML inference** that needs a GPU (Flux, LTX-Video, protein folding, transcription)\n- **Fine-tuning / training** runs\n- **Batch processing** (OCR, document conversion, embedding millions of items)\n- **Long ETL** that exceeds function limits\n\n## Declaring a job\n\nAdd a `jobs:` phase to `gipity.yaml`:\n\n```yaml\nversion: 1\ndeploy:\n  phases:\n    - name: my-jobs\n      type: jobs\n      job_definitions:\n        - name: transcribe\n          handler: jobs/transcribe/main.py\n          runtime: python-3.11\n          compute: gpu-small        # L4 GPU\n          timeout_ms: 300000        # 5 minutes\n          deps: jobs/transcribe/requirements.txt\n          retries: 1\n          description: \"Whisper transcription of an audio file\"\n```\n\nThen drop the handler at `jobs/transcribe/main.py`. Deploy with `gipity deploy dev`.\n\n### Inline annotations (optional)\n\nInstead of declaring everything in `gipity.yaml`, you can write `# @gipity:job` at the top of the handler:\n\n```python\n# @gipity:job runtime=python-3.11 compute=gpu-small timeout=5m\nimport whisper\n# ...\n```\n\nYAML wins on conflict; annotations seed defaults only.\n\n## Compute classes\n\nEach class maps to a substrate + resource shape. Billed **per second** based on actual wall time of the run (deps install included).\n\n| Class | Substrate | RAM | Notes |\n|---|---|---|---|\n| `cpu-small` | Docker sandbox host | 1 GB | Default. Fine for ffmpeg / pandoc / pip-installed Python work |\n| `cpu-large` | Docker sandbox host | 8 GB | Heavy CPU work - large encodes, document batches, ETL |\n| `gpu-small` | Modal L4 (24 GB VRAM) | per Modal | Most inference; the cheapest GPU |\n| `gpu-medium` | Modal A10G (24 GB VRAM) | per Modal | Slightly faster than L4 for some workloads |\n| `gpu-large` | Modal A100-40GB | per Modal | Large model training / fine-tuning |\n| `gpu-huge` | Modal H100 | per Modal | Heaviest training, big batched inference |\n\n`timeout_ms` caps the run inside the container (max 60 min). Modal kills the underlying call if our soft deadline is exceeded.\n\n## Runtimes & the fat image\n\nPick a runtime with `runtime:` (or the `# @gipity:job runtime=...` annotation). Valid values: **`python-3.11`**, **`node-24`** (Node 24; `node-20` is still accepted as a back-compat alias for the same runtime), and **`bash`**. Omit it and it's inferred from the handler extension (`.py`->python, `.js`->node-24, `.sh`->bash).\n\n**Node is CPU-only.** The GPU image ships Python and Bash but no Node, so a `node-*` runtime must pair with a `cpu-*` compute class. Declaring `runtime: node-24` (or `node-20`) with a `gpu-*` compute is rejected at deploy and at submit with a clear error - for GPU work, use `python-3.11`.\n\nGPU runs execute in a Modal container preloaded with:\n- **System**: ffmpeg, sox (incl. mp3/ogg/flac via libsox-fmt-all), imagemagick (PDF coder enabled), poppler-utils, tesseract, build-essential, git, git-lfs, libsndfile, libgl\n- **Python**: torch, transformers, accelerate, diffusers (+ imageio-ffmpeg, so `diffusers.utils.export_to_video` works for image-to-video jobs), peft, datasets, safetensors, sentencepiece, tokenizers, librosa, soundfile, openai-whisper, **demucs** (split via the Python API + soundfile, not the CLI - see gotchas), opencv-python-headless, scikit-image, numpy, scipy, pandas, pillow, httpx, pydantic, requests, boto3, huggingface_hub\n- Plus torchaudio (explicitly baked into the image) - so `torchaudio.pipelines.MMS_FA` forced alignment works without pip install\n\nIf your job only needs the above, **omit `deps:`** entirely - the container starts faster.\n\nCPU runs use the **same image as `gipity sandbox run`** - so its toolkit is exactly the `sandbox-tools` skill's list (ffmpeg, ImageMagick, poppler's `pdftotext`/`pdfinfo`/`pdftoppm`, tesseract, LibreOffice, pandas/numpy/scipy, etc.). Read `sandbox-tools` to see what's already there instead of probing for a binary. Note it's the CPU image only; the GPU (Modal) image is the separate list above.\n\n## Persistent cache\n\nBoth substrates mount a project-shared volume at `/cache`. HF/torch model weights cache there by default (env: `HF_HOME=/cache/hf`, `TORCH_HOME=/cache/torch`). First job downloads the weights; subsequent jobs are instant.\n\nUse `/cache/<your-key>/` for anything you want persisted across runs (intermediate artifacts, downloaded datasets). Don't put per-run output there - use `app-files` to upload final results.\n\n## Handler contract\n\nThe handler is invoked with these env vars set:\n\n| Var | Use |\n|---|---|\n| `GIPITY_RUN_GUID` | Pass to log/progress callbacks |\n| `GIPITY_RUN_INPUT` | JSON of the `input` arg from submit |\n| `GIPITY_APP_GUID` | Project short_guid |\n| `GIPITY_APP_TOKEN` | JWT for calling `/api/{appGuid}/services/*` |\n| `GIPITY_PROGRESS_URL` | POST progress updates here |\n| `GIPITY_OUTPUT_URL` | POST your structured result here (or use `ctx.set_output`) |\n| `GIPITY_PROGRESS_TOKEN` | Auth token for progress + output + log endpoints |\n| `GIPITY_API_BASE` | Base URL (e.g. `https://a.gipity.ai`) |\n\n**Identical on CPU and GPU.** Every var above - plus the `gipity_ctx` helper (`gipity_ctx.py` / `gipity_ctx.js`, see below) - is injected the same way on the Docker (CPU) and Modal (GPU) substrates, so `set_output`, `progress`, and the output channel work on a `gpu-*` job exactly as on a `cpu-*` one; don't gate any of this on compute class.\n\n### Job output contract\n\n**Set your result with `ctx.set_output(...)` - don't print it to stdout.** stdout and stderr are captured as **logs** (streamed live via `gipity job logs <runGuid>`, and the final stdout is stored on `output` only as a fallback). The clean way to return a structured result is the dedicated output channel:\n\n```python\nfrom gipity_ctx import ctx\nctx.set_output({\"stems\": stem_urls})   # this becomes the run's output\n```\n\n```js\nconst { ctx } = require('./gipity_ctx');\nawait ctx.set_output({ stems: stemUrls });\n```\n\n`set_output` blocks until the platform has stored the result and throws if it can't, so a lost result fails loudly instead of vanishing. Whatever you pass is exactly what an `on_complete` hook receives as `output` - **independent of stdout**, so you can `print()`/`console.log` diagnostics and let libraries log freely without corrupting it. Call it once; the last call wins.\n\n**256 KB cap.** The serialized output must be ≤256 KB; a larger payload is rejected (HTTP 413 `RESULT_TOO_LARGE`) and `set_output` throws. For a big result (a rendered file, large PDB/CSV/JSON), upload it with `ctx.upload_file(path)` (see \"uploads the files it produces\" below) and `set_output` the returned URL, or compress to fit (e.g. gzip+base64). Logs (stdout/stderr) share the same 256 KB cap; the live log stream is unbounded.\n\n**Before / after** - the trap `set_output` removes:\n\n```python\n# ❌ Before: result on stdout. One stray log line (yours or a library's) and the\n# platform can't parse stdout as JSON, so output becomes {stdout: \"...logs...\"}\n# and your on_complete hook sees output.stems as undefined.\nprint(\"loading model…\")            # pollutes stdout\nprint(json.dumps({\"stems\": urls})) # intended result\n\n# ✅ After: result on its own channel; logs go anywhere.\nprint(\"loading model…\")            # just a log - harmless\nctx.set_output({\"stems\": urls})    # the result\n```\n\nIf a job never calls `set_output`, the platform still falls back to parsing stdout - the whole stream as one JSON value, or a single JSON object/array at the very end - and otherwise stores the raw stream as `{stdout: \"...\"}`. Prefer `set_output`; the fallback is best-effort.\n\nstderr is captured but not streamed live (yet); it surfaces in `error_message` on failure.\n\n### Progress callbacks\n\nPOST to `$GIPITY_PROGRESS_URL` with `{\"pct\": 0.5, \"message\": \"halfway\"}` - `pct` is 0-1 (not 0-100); a value >1 is rejected with HTTP 400. Updates `progress_pct` and `progress_message` on the run row. Best-effort: log any failure to stderr, never stdout (stdout is the result).\n\n**`gipity_ctx` is the blessed pattern** - prefer it over hand-rolling these POSTs. The helper (`gipity_ctx.py` for python, `gipity_ctx.js` for node) is injected alongside the handler on **both** substrates, so use it on GPU jobs too:\n\n```python\nfrom gipity_ctx import ctx\nctx.progress(0.5, \"halfway\")\nurl = ctx.upload_file(\"/work/result.pdb\", public=True)[\"url\"]   # return a produced file\nctx.set_output({\"pdb_url\": url})\n```\n\nThe helper's surface: `ctx.input`, `ctx.progress()`, `ctx.set_output()`, `ctx.upload_file()`, and `ctx.fetch(path)` (an authenticated call to your app's own `/api/{appGuid}/<path>` - services, records, files, **and your own deployed functions via `fn/<name>`**). All authenticate with the job's app token for you, so you never assemble a raw URL or reason about job networking - `ctx.fetch(\"fn/save-page\", method=\"POST\", body={...})` reaches a public deployed function from inside a running job.\n\n(An older handler that hand-rolls the raw `$GIPITY_PROGRESS_URL` / `$GIPITY_OUTPUT_URL` POSTs predates SDK injection on Modal - not the pattern to copy.)\n\n## CLI surface\n\n```bash\ngipity job list                          # all jobs in current project\ngipity job submit <name> [body]          # returns a run guid\ngipity job wait <runGuid> [--timeout 90] # block until the run finishes; the way to await a job\ngipity job status <runGuid>              # one instant snapshot (no blocking)\ngipity job runs <name> [--limit 20]      # recent runs of one job\ngipity job logs <runGuid> [--follow]     # SSE stream by default; --no-follow = instant snapshot incl. recent stdout/stderr tail\ngipity job cancel <runGuid>              # queued or running -> cancelled\ngipity job run-local <name>              # Docker-on-your-machine for dev iteration\n```\n\nSubmit accepts `--data '<json>'` for input (or pass a JSON arg directly) and `--idempotency-key <key>` for replay.\n\n**Await a run with `gipity job wait` - don't hand-roll a status-poll loop.** It blocks until the run reaches a terminal state and prints its `output`, or, if `--timeout` (seconds, default 90) elapses first, prints current progress and exits non-zero. Exit codes are distinct so a script can branch: **0** = success, **1** = the run failed/cancelled, **2** = still running at timeout (re-run `job wait` to keep waiting). So one call can never hang your shell - it either finishes or tells you to poll again - and you never write a `sleep`+`job status` loop or block on `job logs --follow`. Use `job status` only when you want a single instant snapshot without waiting.\n\n## Agent tools\n\nThe same six verbs are exposed as agent tools, with the same parameter shapes:\n\n- `job_list` (VIEWER) - list jobs in current project\n- `job_submit` (EDITOR) - submit a job; returns `{run_guid, status, replayed}`\n- `job_status` (VIEWER) - short status snapshot\n- `job_runs` (VIEWER) - recent runs of a named job\n- `job_logs` (VIEWER) - captured stdout / stderr / output, truncated at 5 KB\n- `job_cancel` (EDITOR) - flip a queued/running run to cancelled\n\nAgents and the CLI hit the same `/projects/{guid}/jobs/...` endpoints; their behavior matches.\n\n## REST endpoints (project-scoped, JWT auth)\n\n| Method | Path | Purpose |\n|---|---|---|\n| `GET`  | `/projects/{guid}/jobs` | list jobs |\n| `POST` | `/projects/{guid}/jobs/{name}/submit` | submit (returns runGuid) |\n| `GET`  | `/projects/{guid}/jobs/runs/{runGuid}` | one run's status |\n| `GET`  | `/projects/{guid}/jobs/{name}/runs` | list runs of one job |\n| `GET`  | `/projects/{guid}/jobs/runs/{runGuid}/logs/stream` | SSE log stream |\n| `POST` | `/projects/{guid}/jobs/runs/{runGuid}/cancel` | best-effort cancel |\n\nSubmit body: `{ \"input\": {...}, \"idempotency_key\": \"...\" }`.\n\n### In-function jobs API\n\nA deployed function submits and polls jobs through `jobs` - a capability on the handler's **second argument**, a peer of `db`/`storage` (NOT `ctx.jobs` - `ctx` only carries `body`/`query`/`headers`/`method`/`auth`). Like `db`, it needs no `services:` declaration. No HTTP layer, same concurrency caps / GPU gating / idempotency replay as the CLI. All three throw on error:\n\n```js\nexport default async function startFold(ctx, { jobs }) {\n  const { run_guid, status, replayed } = await jobs.submit('fold', { seq }, { idempotency_key: 'opt' });\n  const s = await jobs.status(run_guid);\n  // s -> { run_guid, status, progress_pct, progress_message, attempt, started_at, completed_at, duration_ms, error, output }\n  //   output is populated once status === 'success' - the value the handler passed to set_output.\n  await jobs.cancel(run_guid);   // -> { status }\n  return { run_guid };\n}\n```\n\nA function returns in ≤30 s - don't poll a long job to completion inside it. Submit and return `run_guid`, or set `on_complete` (below) to fire a function when the run finishes.\n\n## State machine\n\n`queued → running → success | failed | cancelled`\n\nStatus writes are atomic - only one transition wins. Retries (`retries: N` in the YAML) re-claim a `failed` row back to `running` for the next attempt, with exponential backoff. Deps-install failures are **terminal** - no retries (the user's setup is wrong; retrying won't help).\n\nFailed and cancelled runs are billed for the time they actually ran (Modal / Docker charged us). Successful runs are billed for the same.\n\n## Billing\n\nPer-second metering by compute class. Underlying USD/sec rates track Modal's published prices; 100 % platform margin applied. Both lines visible on the credits Pricing tab (Jobs category).\n\nA 30-second L4 (`gpu-small`) job at 100 % margin ≈ $0.013 in credits (Modal's $0.80 / hr × 30 s × 2).\n\n## Auto-scaling: parallel jobs run in parallel\n\nSubmit N jobs concurrently and they execute concurrently. Modal's worker fleet spins up a fresh container per submission within seconds (cached image), runs your handler, and tears down - no per-job queue, each submission gets its own container.\n\nThe only ceiling is the **project-level cap of 10 in-flight jobs** (queued + running). If you submit an 11th while 10 are already in flight you get `RATE_LIMITED` (HTTP 429); wait for some to finish. This cap is per project, not per compute class - 5 L4s and 5 H100s in flight = cap reached.\n\nPractical implications:\n\n- **Batch workloads**: submit 10 jobs in a tight loop and watch them all start. Modal handles the fan-out at the substrate layer.\n- **Warm-pool freebie**: containers are kept alive ~60 s after a job finishes, so a second submission within that window starts in <5 s instead of paying the ~10-30 s cold-start.\n- **Image is cached fleet-wide**: the 15GB fat image is pulled to each Modal worker once. Steady-state cold starts are bounded by Python interpreter + your handler's imports.\n- **What we don't expose yet**: `fn.map()` fan-out (one submission → N parallel runs), `min_containers` (keep N warm at idle cost), per-container `concurrent_inputs` for IO-bound serving. All are Modal-native and would land in future PRs if you hit the use case.\n\n## Limits\n\n- Max concurrent (queued + running) jobs per project: 10\n- Max `timeout_ms`: 60 min (3,600,000)\n- Max retries: 10\n- Default 5-min deps install timeout\n- Result stdout / stderr capped at 256 KB (truncation marker preserved). Live log stream is unbounded - store large outputs via `app-files`.\n\n## Common patterns\n\n### Reads an input file + uploads the files it produces\n\nA job returns files with **`ctx.upload_file(path)`** - one call that stores the file in the app and hands back its `{guid, url, variants}`. That's the answer to \"how does a GPU job get its output files back\": don't hand-roll the upload endpoints, and don't try to cram bytes into `set_output` (that channel is ≤256 KB of JSON). Upload each file, then `set_output` the urls.\n\n**Feeding a job a real input file when testing: `gipity upload`.** `gipity upload <file>` uploads a local file and prints a durable, worker-reachable URL the instant it lands - no `gipity deploy` needed (a file pushed into the project tree with `gipity push` has no public URL until you deploy; this does). Pass that URL straight into the job input:\n\n```bash\ngipity upload song.mp3\ngipity job submit split-stems --data '{\"audio_url\":\"<printed url>\"}'\n```\n\nBy default the file is PUBLIC (a plain `media.gipity.ai` CDN url that resolves from anywhere, so a cloud worker can always fetch it); `--private` mints a token-signed serve url instead, which workers can still fetch (see below).\n\n```python\nfrom gipity_ctx import ctx\n\n# Download the input. The URL must be reachable from the cloud worker: pass what\n# storage.fileUrl(guid) returns for the uploaded file. This is reachable whether the\n# file is PUBLIC (a media.gipity.ai CDN url) or PRIVATE (a tokenized a.gipity.ai serve\n# url) - a private file's serve url is a public https endpoint gated by a capability\n# token, so the worker fetches it fine. DON'T flip a user's upload to public just to\n# feed a job; keep private uploads private. A deployed https://dev.gipity.ai/… file\n# also works. Only an app-internal/localhost url won't resolve.\nimport urllib.request\nurllib.request.urlretrieve(ctx.input[\"audio_url\"], \"/work/song.mp3\")\n\n# ...process with ffmpeg / whisper / demucs, writing outputs under /work ...\n\nstems = {name: ctx.upload_file(f\"/work/{name}.wav\", public=True, content_type=\"audio/wav\")[\"url\"]\n         for name in (\"vocals\", \"drums\", \"bass\", \"other\")}\nctx.set_output({\"stems\": stems})   # the ≤256 KB JSON result: just the urls\n```\n\nNode: `await ctx.upload_file('/work/out.mp4', { public: true })` returns the same `{ guid, url, ... }`.\n\n### Fan-out via on_complete\n\n```yaml\n- name: align\n  handler: jobs/align/main.py\n  compute: gpu-small\n  on_complete: render-preview     # name of a deployed function\n```\n\nWhen the align job hits a terminal state the platform fires the named function with `{ run_guid, status, output, error }`. The render function can then submit a new render job.\n\n### Job persists structured results by calling a function\n\nThe canonical ingestion pipeline: a function submits a `cpu-*` job, the job does the heavy work, and posts results back to a deployed function that writes them to the database. Use `ctx.fetch(\"fn/<name>\", ...)` - it targets your own function with the app token attached, so you never build the raw URL or worry about whether the container can reach the API (it can; both substrates share the host network). This works to stream partial results *during* the run, not just at the end like `on_complete`.\n\n```python\nfrom gipity_ctx import ctx\nimport urllib.request, subprocess\n\nurllib.request.urlretrieve(ctx.input[\"pdf_url\"], \"/work/doc.pdf\")\nn_pages = int(ctx.input[\"pages\"])\nfor page in range(1, n_pages + 1):\n    text = subprocess.run(\n        [\"pdftotext\", \"-f\", str(page), \"-l\", str(page), \"/work/doc.pdf\", \"-\"],\n        capture_output=True, text=True).stdout\n    # POST each page straight into a deployed function that inserts a DB row.\n    ctx.fetch(\"fn/save-page\", method=\"POST\",\n              body={\"doc_id\": ctx.input[\"doc_id\"], \"page\": page, \"text\": text})\n    ctx.progress(page / n_pages, f\"page {page}/{n_pages}\")\nctx.set_output({\"pages_indexed\": n_pages})\n```\n\nThe `save-page` function is an ordinary deployed function (`auth_level: public` so the job's app token reaches it) that does the insert. No webhook URL, no `X-App-Token` plumbing - `ctx.fetch` is the whole story.\n\n## Testing & gotchas\n\nTesting a job pipeline (submits are **suppressed** in a test run, so you drive the trigger fn and `on_complete` fn directly - plus the one-time real GPU smoke from the CLI) and the common gotchas (the Demucs 4-stem recipe, deps that conflict with the fat image, bash `set -e`, and the `output` channel) live in the [jobs-reference](jobs-reference.md) skill.\n\n## See also\n\n- [jobs-reference](jobs-reference.md) - testing a job pipeline and the common gotchas\n- [deploy](deploy.md) - how the jobs phase fits into the rest of the deploy pipeline\n- [app-development](app-development.md) - functions vs jobs, when to use which\n- [app-files](app-files.md) - uploading artifacts the job produces\n- [app-debugging](app-debugging.md) - `page inspect`, function logs, run logs\n"
}