# App API - File Uploads

File uploads for deployed apps use presigned upload URLs to Gipity Storage. The client uploads directly to storage (no proxy through the server), supporting files up to 30GB with progress tracking.

## How It Works

1. **Init** - App calls `POST /api/<APP_GUID>/uploads/init` with file metadata. Server returns a presigned upload URL.
2. **Upload** - Client PUTs the file directly to the presigned URL. No auth header needed - the URL is pre-authorized.
3. **Complete** - App calls `POST /api/<APP_GUID>/uploads/complete`. Server copies the file to permanent Gipity Storage, records it, and generates variants (thumbnails for images, text for PDFs, ID3 tags + cover-art thumbnail for audio). For audio it also returns the parsed tags on `data.audio`.

## Client Helper Library

The client SDK (`gipity.js`, auto-injected into every template via `{{ANALYTICS_SCRIPT}}`) exposes `Gipity.upload` for automatic progress tracking, multipart handling, and retries. No extra script tag is needed.

```js
const result = await Gipity.upload(fileInput.files[0], {
  onProgress: (pct) => progressBar.style.width = pct + '%',
  public: false,
  table: 'attachments',
  recordId: 'rec_123',
});
console.log(result.guid, result.url);
```

The SDK auto-supplies `appGuid` from the script tag's `data-app` attribute and `appToken` from `Gipity.auth` (when the user is signed in). Pass them explicitly via `{ appGuid, appToken }` only when calling from a page that doesn't load the SDK with `data-app` set.

Single-part uploads (< 5GB) and multipart uploads (5GB+) are handled transparently. Failed parts retry up to 3 times with exponential backoff.

**Verifying the upload flow end-to-end from the CLI.** To drive this signed-in path against a real file - upload a PDF, then assert whatever ingest/extract runs next - use `gipity page eval --auth --fixture ./sample.pdf`. `--fixture` hosts the local file ephemerally, injects a `fetch`-able `fixtureUrl` into the eval, and **deletes the hosted copy afterwards** - so you never bundle a test file into `src/` or run an extra `deploy` just to make it reachable. Fetch it in-page, wrap the bytes in a `File`, and hand it to `Gipity.upload`:

```bash
gipity page eval "https://dev.gipity.ai/me/app/" --auth --fixture ./sample.pdf \
  "(async()=>{ const b = await fetch(fixtureUrl).then(r=>r.blob()); const f = new File([b], 'sample.pdf', { type: 'application/pdf' }); const { guid } = await Gipity.upload(f); return { guid }; })()"
```

Repeat `--fixture` for several files (each also exposed under `fixtures['<basename>']`). Full reference: [app-debugging-reference](app-debugging-reference.md).

**Just need a durable URL for a local file?** `gipity upload <file>` uploads it into this same store and prints a worker-reachable URL immediately (no deploy) - public CDN by default, `--private` for a token-signed serve url. Handy for feeding a job or function a real input; see [jobs](jobs.md).

## Folder & multi-file uploads

`Gipity.upload` takes one `File`. For a **folder drop** or a multi-file picker, the browser makes you do fiddly, easy-to-get-wrong traversal first: a dropped folder leaves `dataTransfer.files` empty (or a zero-byte stub), so you must read the non-standard entry tree - capturing `webkitGetAsEntry()` *synchronously* in the drop handler (the items list goes stale the instant it returns) and looping `readEntries()`, which pages at 100 entries per call. The SDK owns that for you.

`Gipity.collectFiles(source)` normalizes a drop event, a `DataTransfer`, an `<input webkitdirectory>`, a plain `<input multiple>`, a `FileList`, or a `File[]` into a flat list. Each entry is `{ file, relativePath }`, where `relativePath` is the path *within the drop* (`"Album/Disc 1/track01.mp3"`) - just the filename for a loose file. It's traversal only: loop and call `Gipity.upload` yourself, or hand the list to `uploadAll`.

```js
// In the drop handler - call collectFiles SYNCHRONOUSLY (don't await anything first).
dropZone.addEventListener('drop', async (e) => {
  e.preventDefault();
  const items = await Gipity.collectFiles(e);          // [{ file, relativePath }, ...]
  for (const { file, relativePath } of items) {
    await Gipity.upload(file, { path: relativePath.split('/').slice(0, -1).join('/') });
  }
});

// Or from a folder picker:
//   <input type="file" webkitdirectory multiple>
input.addEventListener('change', () => Gipity.collectFiles(input).then(...));
```

`Gipity.uploadAll(items, opts?)` uploads a whole batch and mirrors the folder structure for you - each file's `relativePath` directory is appended to `opts.path`, so the tree lands the same way in storage. It accepts the array from `collectFiles`, or a raw drop/input it collects for you. Uploads run `opts.concurrency` at a time (default 3) and **one file failing does not abort the batch** - you get a per-item report. Pass `{ stopOnError: true }` to reject on the first failure instead.

```js
const report = await Gipity.uploadAll(e, {       // pass the drop event directly
  path: 'locker',                                // base path; folders nest under it
  concurrency: 4,
  onProgress: (overall, info) => bar.style.width = overall + '%',  // 0-100 across the whole batch
});
// report.succeeded, report.failed, report.results: [{ relativePath, file, ok, value?, error? }, ...]
const failures = report.results.filter((r) => !r.ok);
```

`opts` also forwards `appGuid`, `appToken`, `apiKey`, `public`, `table`, `recordId`, and `signal` to each `Gipity.upload` call.

## Client SDK surface

Browser globals from `gipity.js`: `Gipity.upload`, `Gipity.collectFiles`, `Gipity.uploadAll`, `Gipity.fileUrl`, `Gipity.fn`, `Gipity.auth`, `Gipity.captureError`, `Gipity.version`. `upload` stores one file; `collectFiles` flattens a folder drop / directory picker into `{ file, relativePath }[]`; `uploadAll` uploads such a batch (mirroring folders under a base path); `fileUrl` gets an embeddable URL to display one (see Displaying files). To list or delete files, call your own `auth: user` functions via `Gipity.fn` (see Deleting a file). `uploads/init` always requires an authenticated user; anonymous init returns 401 UNAUTHORIZED, so personal-file apps need sign-in.

## Endpoints

### POST /api/<APP_GUID>/uploads/init

Get a presigned upload URL.

**Auth:** App token (`X-App-Token`), API key (`X-Api-Key`), or session cookie.

**Request:**
```json
{
  "filename": "photo.jpg",
  "content_type": "image/jpeg",
  "size": 2048576,
  "public": false,
  "table": "incidents",
  "record_id": "42",
  "path": "uploads"
}
```

- `filename` (required) - original filename
- `content_type` (required) - MIME type
- `size` (required) - file size in bytes (max 30GB)
- `public` (optional, default false) - public CDN URL or signed URL
- `table` (optional) - associate with a record table
- `record_id` (optional) - associate with a specific record
- `path` (optional, default "uploads") - virtual directory path (no leading slash; any leading/duplicate slashes are normalized away)

**Response (single-part, < 5GB):**
```json
{
  "data": {
    "upload_guid": "fl_Xk7mNp2R",
    "method": "PUT",
    "url": "https://muda.gipity.ai.s3...",
    "headers": { "Content-Type": "image/jpeg" },
    "expires_in": 600
  }
}
```

**Response (multipart, >= 5GB):**
```json
{
  "data": {
    "upload_guid": "fl_Xk7mNp2R",
    "method": "multipart",
    "upload_id": "abc123...",
    "part_size": 104857600,
    "parts": [
      { "partNumber": 1, "url": "https://..." },
      { "partNumber": 2, "url": "https://..." }
    ],
    "expires_in": 600
  }
}
```

### POST /api/<APP_GUID>/uploads/complete

Finalize the upload after the client has PUT the file to S3.

**Request:**
```json
{
  "upload_guid": "fl_Xk7mNp2R",
  "parts": [
    { "part_number": 1, "etag": "\"abc123\"" },
    { "part_number": 2, "etag": "\"def456\"" }
  ]
}
```

- `upload_guid` (required) - from the init response
- `parts` (required for multipart only) - part numbers and ETags from each S3 PUT response

**Response:**
```json
{
  "data": {
    "guid": "fl_Xk7mNp2R",
    "name": "photo.jpg",
    "size": 2048576,
    "content_type": "image/jpeg",
    "width": 1920,
    "height": 1080,
    "audio": null,
    "url": "https://media.gipity.ai/app-files/...",
    "is_public": false,
    "table": "incidents",
    "record_id": "42",
    "variants": [
      { "type": "thumbnail", "url": "...", "content_type": "image/jpeg", "status": "complete" }
    ]
  }
}
```

### GET /api/<APP_GUID>/files

List uploaded files. Supports filtering.

**Query params:** `table`, `record_id`, `path`, `limit` (default 50, max 200), `offset`

### GET /api/<APP_GUID>/files/:guid

Get file metadata + variant list.

### GET /api/<APP_GUID>/files/:guid/content

Download file (auth-gated). 302-redirects to the underlying storage. Convenience endpoint; for an embeddable/persistable URL use the durable serve URL below.

### GET /api/<APP_GUID>/files/:guid/serve?t=<token>

**The durable, embeddable access URL for a file.** Auth is the capability token in `?t=`, *not* a header — so it drops straight into `<audio src>`, `<a download>`, or an anonymous share page, and resolves for signed-out viewers. The server re-signs storage on every hit and 302-redirects (the browser follows it with its `Range` header, so audio/video seeking works), so the URL never expires on its own. You don't build this URL by hand — `Gipity.fileUrl` (client) and `storage.fileUrl` (functions) return it. Add `&variant=thumbnail` for a variant, `&download=1` to force a download.

Serve URLs work everywhere: element sources (`<img>`, `<audio>`, `<video>`, `<a download>`), navigations, cross-origin `fetch()`/XHR from page JS (CORS-enabled — blob downloads, hashing, canvas export all work), and `curl`/`curl -I`.

### GET /api/<APP_GUID>/files/:guid/variants/:type

Get a specific variant (e.g., thumbnail), auth-gated. Redirects to storage.

## Displaying files (`<img src>`, `<audio>`, downloads)

The `url` you get back is **durable and safe to persist** — store it on your record, in a share link, or render it straight into the page. It does not expire on its own and never exposes a storage hostname:
- **public** files → a permanent CDN URL (`media.gipity.ai/...`)
- **private** files → a branded serve URL that re-signs storage on every request

> The metadata/content endpoints are auth-gated (`X-App-Token`), so a bare `<img src="https://a.gipity.ai/api/<APP_GUID>/files/<guid>/content">` (cookie only, no header) is rejected. **That is the one wrong way to show a private file** — use the durable `url` / `Gipity.fileUrl` instead.

```js
img.src = await Gipity.fileUrl(fileGuid);                          // full file
thumb.src = await Gipity.fileUrl(fileGuid, { variant: 'thumbnail' }); // 200x200 image thumb
```

For `public: true` files, `Gipity.upload(...)` already returns a usable `result.url`. `Gipity.fileUrl` is for private files or a variant URL.

**Don't persist the raw upload result for a private file and assume it streams forever — persist the file `guid`, the durable `url`, or both.** (The older pattern handed back a 1-hour signed storage URL; apps that stored *that* served it stale once it expired — the "Request has expired" bug. The serve URL above replaces it. If you have old rows holding an `s3.amazonaws.com` or `...X-Amz-...` URL, re-resolve them from the `guid` via `Gipity.fileUrl` / `storage.fileUrl`.)

### Serving private files to anonymous viewers (share links)

A `public`-auth function (e.g. a share-link or delivery resolver) can mint a durable, header-free URL for a privately-stored file with the `storage.fileUrl` helper — no sign-in needed by the viewer:

```js
// functions/share-resolve.js  (auth: public)
export default async function (ctx, { db, storage }) {
  const link = await db.findOne('share_links', { short_name: ctx.query.s });
  if (!link) return { error: 'not found' };
  const file = await db.findOne('files', { short_guid: link.file_guid });
  return {
    name: file.filename,
    url: await storage.fileUrl(file.file_guid),                 // streamable, durable
    download_url: await storage.fileUrl(file.file_guid, { download: true }),
  };
}
```

The returned URL is permanent until you delete the file or call `storage.revokeFileUrl(fileGuid)` (which bumps a per-file epoch and invalidates every URL previously minted for it — call `storage.fileUrl` again for a fresh one). Store the file `guid`, never a baked-in signed URL.

> **Testing it:** to assert a function resolves a real serve URL, make a real file in the test with `ctx.upload(name, opts?)` (returns `{ guid, url, ... }`) and pass its `guid` — a fabricated `storage_guid` won't resolve. See [app-testing → Testing file serving](app-testing.md#testing-file-serving-with-ctxupload).

## Variants

Variants are generated **synchronously at upload-complete** for files up to 10MB - they're in the upload result's `variants` array immediately, or never (nothing materializes later):
- **thumbnail** - Images resized to 200x200px (JPEG), any dimensions. Audio files with embedded cover art (ID3 APIC) also get a `thumbnail` variant from that art.
- **text** - PDFs get their whole-document text extracted to plain text.
- **text_pages** - PDFs also get a JSON array of per-page text: `[{ "page": 1, "text": "..." }, ...]`, 1-indexed. This is what a document-Q&A / RAG app indexes so an answer can cite **which file and page** it came from. **Read it server-side in a function with `storage.readText(fileGuid)`** (see below) - don't fetch the variant URL (functions can't reach `*.gipity.ai`) or relay it through the browser. Chunk each page's text, store it with the page number, and search. No need to roll your own per-page extraction job. A PDF with no text layer (a scan) gets **neither** text variant, so `readText` returns empty `pages` (and the upload `warnings` say to OCR it, e.g. a `tesseract` CPU job).

**A file can legitimately have no thumbnail** - over 10MB, undecodable bytes, non-image, audio without cover art. When generation is skipped or fails, the upload result carries a `warnings` array saying why. So check `result.variants` before resolving: `Gipity.fileUrl` and `storage.fileUrl` with `{ variant: 'thumbnail' }` **throw** when the variant doesn't exist - fall back to the original file or a placeholder in your grid/list rendering.

For audio uploads, the complete-response also carries a `data.audio` object with the parsed ID3/container tags (`null` when the file isn't audio or carries no readable tags):
```json
"audio": {
  "artist": "Daft Punk", "album": "Discovery", "genre": "House",
  "title": "One More Time", "track_number": 1,
  "track_seconds": 320, "duration_ms": 320373, "bitrate": 320, "year": 2001
}
```
Persist these onto your own record if you want a searchable/sortable music library — the platform returns them but does not store them on your app's tables for you.

## Deleting a file

There is no client-side or HTTP delete - removing a file is a privileged operation, so it lives in **function code**. Every deployed function gets a `storage` helper in its `services` argument (always available, like `db` - no `gipity.yaml` declaration needed):

```js
await storage.delete(fileGuid);              // → { deleted: true }  (or false if not found)
await storage.fileUrl(fileGuid, opts?);      // → durable access URL (see "share links" above)
await storage.revokeFileUrl(fileGuid);       // → { revoked: true }  (invalidate minted URLs)
await storage.readText(fileGuid);            // → { text, pages: [{ page, text }], pageCount }  (extracted PDF text)
```

**`storage.readText(fileGuid)`** reads a PDF's server-extracted text for document-Q&A / RAG. The platform auto-extracts per-page text at upload; this pulls it straight from storage inside your function, so you index it server-side without a browser relay. Returns `{ text, pages: [{ page, text }], pageCount }` — `text` is the whole document, `pages` is per-page (1-indexed) for **file + page** citations. A file with no text layer (a scanned PDF, or a non-PDF) returns empty `text`/`pages` and `pageCount: 0`, so an ingest loop can skip it (OCR it separately). The canonical ingest: user uploads a PDF → an `ingest` function calls `storage.readText`, chunks each page, and INSERTs rows with a full-text index → a `query` function searches those chunks and returns answers with their file/page.

This permanently removes the stored bytes - the file **and** any thumbnail/text variants - from Gipity Storage, and marks the file deleted. It is scoped to your app (a function can only delete its own app's files) and idempotent: deleting an unknown or already-deleted guid returns `{ deleted: false }` instead of throwing.

**Delete the bytes whenever you delete the record that points at them** - otherwise the file lingers in storage forever, leaking storage on every removal. The usual shape: the browser calls one of your functions, which removes both the record and the file.

```js
// functions/delete-receipt.js
export default async function (ctx, { db, storage }) {
  const { file_guid } = ctx.body;
  const row = await db.findOne('receipts', { file_guid, user_guid: ctx.auth.userGuid });
  if (!row) return { error: 'not found' };

  await db.delete('receipts', { id: row.id });   // your app-data row
  await storage.delete(file_guid);                // the actual bytes - reclaims storage
  return { ok: true };
}
```

```js
// client - call your function; never try to delete files straight from the browser
await Gipity.fn('delete-receipt', { file_guid });
```

`file_guid` is the `guid` returned from the upload (`Gipity.upload(...)` → `result.guid`) - the same value you stored on your record.

## Limits

- Max file size: 30GB
- Presigned URL expiry: 10 minutes
- Rate limit: 300 uploads per minute
- Multipart part size: 100MB

## Manual Upload (without helper library)

If you can't use the helper library, implement the three-step flow directly:

```js
// 1. Init
const init = await fetch(`https://a.gipity.ai/api/${appGuid}/uploads/init`, {
  method: 'POST',
  headers: { 'X-App-Token': token, 'Content-Type': 'application/json' },
  body: JSON.stringify({
    filename: file.name,
    content_type: file.type,
    size: file.size,
  }),
}).then(r => r.json());

// 2. Upload to S3
await fetch(init.data.url, {
  method: 'PUT',
  headers: { 'Content-Type': file.type },
  body: file,
});

// 3. Complete
const result = await fetch(`https://a.gipity.ai/api/${appGuid}/uploads/complete`, {
  method: 'POST',
  headers: { 'X-App-Token': token, 'Content-Type': 'application/json' },
  body: JSON.stringify({ upload_guid: init.data.upload_guid }),
}).then(r => r.json());

console.log(result.data.url); // permanent file URL
```

## Integration with Records

Files can be associated with records via `table` and `record_id` fields (soft reference). Example flow:
1. User uploads a screenshot with `table: 'incidents', record_id: '42'`
2. Frontend lists attachments via `GET /api/:app/files?table=incidents&record_id=42`
3. Each attachment shows the thumbnail variant as a preview