Definition

Bucket vs Workspace

A bucket is raw S3 — it exists the moment it’s created, whether or not Atlas knows about it. A Workspace is the governance counterpart: a bucket an admin has explicitly registered, which is what unlocks declaring documents, links, search, and role-based access on that bucket.

S3 Bucket: invoices

Registered by an admin

1:1 by name

governance Workspace

Declare, search, roles

S3 Bucket: scratch-uploads

Never registered

no Workspace

fallback Browsing only

Superuser-gated

There’s no separate table joined by an id — the bucket’s name is the join key everywhere, because a bucket can’t be renamed in S3 either, so the name is stable for the row’s whole lifetime. Raw object browsing (list, download, tags) stays open to any authenticated session on any bucket, registered or not — only the governance routes require a Workspace to exist first.

It’s a real bucket, not a shared bucket with a workspace/xyz/ prefix per team: versioning, Object Lock, lifecycle rules, and encryption are all configured per bucket in S3, not per prefix, and a real bucket gives every workspace a hard blast radius — deleting one workspace’s data can never leak into another’s storage or IAM scope.

Register a workspace from the admin UI or the API, either creating a new bucket or adopting an existing one. Granting roles on it is covered in Authorization.

Object, MetaObject & Document

The same file is visible through three different lenses, deliberately never merged into one table:

ground truth S3 Object

Read live, every request

scan

technical layer MetaObject

Cache of every version

ground truth S3 Object

Read live, every request

declare

business layer Document

Status, properties, links

A Link hangs off a Document the same way — source/target, pointing at another Document, never at an Object directly. A Document also keeps two drift flags compared live against its Object: orphan and stale, covered below.

• Object — S3’s own reality. Listing a workspace always reads the bucket live; nothing about “which files exist” is ever cached.
• MetaObject — one row per version of an object, mirrored into Postgres so bucket-wide stats (total size, object count) don’t require a live walk of the whole bucket on every request. Filled in lazily (any request that already fetched an object’s metadata for its own response updates it) and in bulk by the reconciliation scan.
• Document — the governance row. Created only by declaring, idempotent per file, so retrying a declaration is always safe. Carries a state (NORMAL / PROCESSING / SUCCESS / ERROR) and free-form business properties.
• Link — a hint that two Documents relate to each other (an invoice and its purchase order, say), framed relative to whichever document the caller is looking at.

No foreign key connects a MetaObject to a Document: a version can exist with no Document ever declared over it — most files, in practice — and a Document can be declared before any version has been scanned. Keeping them independent means either can be populated on its own schedule instead of one waiting on the other.

Orphan & Stale

Two drift flags live on a Document, compared against the live object: orphan (the object is gone) and stale (the object changed since declaration).

declared Document

orphan: false

stale: false

S3 read notices a mismatch

drifted Document

orphan and/or stale → true

Never flips back on its own

orphan/stale cost a live S3 call to check, so nothing forces that check on every read. They flip to true the moment any request that already touched S3 for that key notices a mismatch — browsing, opening a detail view, searching — plus a bulk sweep triggered manually (Sync, on a workspace or a bucket). They deliberately never flip back to false on their own: only a person explicitly accepting the current content as the new baseline clears them. Auto-clearing the moment content happens to match again would quietly erase real drift history from whoever’s meant to review it — the flag is a signal for a human, not a cache to keep fresh.

The audit trail follows the same instinct one level up: an entry records bucket/key directly rather than pointing at a Document row, because a bare S3 rename or delete can target a key that was never declared at all — the audit trail can’t assume governance data exists just because an action happened.