tl;dr
Put durable business truth in a database, recomputable hot reads in a cache, work awaiting execution in a task queue, and large opaque payloads in object storage. Break this rule only when the workload's durability, consistency, latency, retention, and delivery requirements justify the exception.
- A database answers questions about current durable truth. It is the default for users, permissions, orders, workflow state, and relationships.
- A cache makes a known read cheaper. Its contents should have an authority, staleness bound, invalidation rule, and miss path.
- A queue represents work or delivery. It needs explicit retry, acknowledgment, ordering, retention, and duplicate-execution semantics.
- Object storage holds objects. Keep identity, permissions, and workflow metadata in the database while storing file bytes under stable object keys.
- Start with fewer systems. Every additional state layer adds isolation, observability, backup, credential, and recovery obligations.
An agent can connect a database, Redis instance, queue, and storage bucket faster than a human can review the architecture. That speed makes the placement decision more important, not less.
The four systems do not differ only by product category. They answer different questions:
- What is true now?
- What value can be recomputed or fetched again?
- What work still needs to happen?
- Where do these bytes live?
When one system answers all four accidentally, failure behavior becomes difficult to explain.
the state-placement worksheet
Before choosing a layer, write down these properties:
| question | why it changes the choice |
|---|---|
| Must it survive process, zone, or provider failure? | Defines the durability and recovery contract |
| Is loss acceptable? For how long? | Separates durable truth from bounded-staleness cache |
| Is the payload relational, computed, executable, or opaque bytes? | Points toward database, cache, queue, or object storage |
| Does it change with other records atomically? | Favors one transactional boundary |
| Is it read by key, queried flexibly, or consumed once? | Defines the access model |
| Can it be regenerated? | Makes eviction or deletion safer |
| Must many consumers see it? | Distinguishes task delivery from streams and pub/sub |
| Can processing happen twice? | Determines idempotency and deduplication needs |
| How long should it remain? | Defines TTL, retention, archive, and deletion policy |
| Which environment owns it? | Prevents preview, development, and production collisions |
If the answers are unknown, adding a service does not resolve the uncertainty.
database: durable application truth
Use a database when the application needs to ask what is true and preserve the answer across compute replacement.
Common database records include:
- users and organizations;
- permissions;
- orders and invoices;
- workflow and job state;
- object metadata;
- audit events;
- configuration owned by the application;
- relationships among these records.
A relational database is especially valuable when several changes must commit together or invalid states should be rejected centrally. PostgreSQL transactions provide all-or-nothing updates, and constraints enforce rules such as uniqueness and valid references. The PostgreSQL documentation describes those guarantees directly.
a non-obvious database use: the first queue
A Postgres table can be the correct first task queue.
Suppose creating an import must also guarantee that processing will eventually be attempted. One transaction can insert the import record and its task row. A worker claims available rows with locking, records attempts, and updates terminal state.
This avoids the “database commit succeeded but queue publish failed” gap. It may be enough until throughput, retry scheduling, or independent scaling justify a dedicated queue.
The database is not always the most specialized tool. It is often the simplest correct boundary.
cache: replaceable acceleration
A cache stores a value because retrieving or computing it repeatedly is expensive.
Every cache entry should answer four questions:
- What is the authoritative source?
- How stale may this value become?
- What invalidates or versions it?
- What happens on a miss or cache outage?
If the application cannot answer those, the cache may have become an accidental database.
Redis cache-aside is a common pattern: look in Redis, load from the primary on a miss, write with a TTL, and invalidate when the primary changes. Redis documents TTL-bounded staleness and explicit invalidation in its cache-aside guidance.
a non-obvious cache decision: do nothing
Do not add Redis because a query takes 40 milliseconds in local testing.
First identify the production access pattern. An index, smaller query, precomputed column, connection-pool adjustment, or short in-process cache may solve the actual bottleneck without a second distributed state system.
The specific two-system decision is covered in Postgres vs Redis: when do you need each?.
queue: work waiting for execution
A queue holds messages or tasks because producers and consumers should not have to complete work in the same request or at the same rate.
But “queue” can mean several different systems:
| queue shape | delivery model | offline and replay behavior |
|---|---|---|
| Pub/sub | Broadcast to active subscribers | Offline subscribers may miss messages; usually no replay |
| Task queue | One worker or worker group owns each task attempt | Retries and duplicate delivery depend on the queue contract |
| Retained stream or log | Consumers track positions through ordered history | Supports replay and multiple independent consumer groups |
| Database-backed work table | Workers claim durable rows | Delivery and ordering are implemented through database transactions and worker logic |
Google Cloud Tasks uses at-least-once delivery, which means duplicate execution can occur. Its documentation requires handlers to avoid harmful repeated side effects.
Redis distinguishes fire-and-forget Pub/Sub, replayable Streams, and job queues where each task is claimed by one worker. Its streaming guide explains the different retention and consumer semantics.
Before choosing a queue, define:
- delivery guarantee;
- acknowledgment point;
- retry schedule and limit;
- visibility timeout or lease;
- ordering scope;
- retention;
- dead-letter or terminal failure behavior;
- idempotency key;
- payload schema evolution;
- observability for one exact task.
A queue that makes work invisible when it fails is not reliable infrastructure.
object storage: durable opaque payloads
Object storage fits files and blobs that are primarily written and retrieved as whole objects:
- user uploads;
- images and video;
- generated reports;
- exports and archives;
- build artifacts;
- backups;
- large model or dataset files.
An object has bytes, a key, and metadata. A bucket or equivalent container groups objects. Amazon S3, for example, documents objects as files plus metadata stored under keys in buckets, with strong read-after-write consistency for object PUT and DELETE operations. Those guarantees are specific to S3's documented model, not a universal promise about every provider or every bucket-configuration operation.
split bytes from business metadata
For the document-review application:
- Postgres stores the document identifier, organization, uploader, MIME type, processing state, retention policy, and object key.
- Object storage holds the uploaded bytes under that key.
The application authorizes access using database records, then reads or signs access to the object. A missing object is a state mismatch that operations should surface. Deleting the database row and deleting the object are separate state transitions with an explicit recovery order.
when a blob in Postgres is reasonable
Small objects can live in Postgres. This can simplify atomic writes, backup, and access control for an early application. Move to object storage when payload size, transfer patterns, retention, cost, or independent lifecycle makes the split worthwhile.
“Never store files in a database” is as unhelpful as “store every upload in a bucket on day one.” Use workload evidence.
place the reference application's state
The document-review application contains several state shapes:
| state | placement | reason |
|---|---|---|
| User and organization | Postgres | Durable relational truth |
| Document workflow status | Postgres | Queried, constrained, and updated with other records |
| Uploaded PDF bytes | Object storage | Large opaque payload with its own lifecycle |
| Document metadata and object key | Postgres | Authoritative identity and authorization context |
| Permission projection | Redis cache after measured need | Recomputable from Postgres with bounded staleness |
| OCR task | Postgres work table initially, dedicated queue later if earned | Durable work identity and eventual worker execution |
| Live progress broadcast | Pub/sub if transient loss is acceptable | Connected clients need current updates, not replayed history |
| Audit event | Postgres or retained event store | Must survive consumer disconnect and support later inspection |
| Weekly-report trigger | Cron schedule | Begins because of time, then creates traceable work |
No single product category is “best” for all of this state. The system is correct when each placement has a named authority and recovery contract.
common agent-generated placement failures
- storing user-visible workflow state only in process memory;
- writing uploads to local container disk;
- using Redis as authoritative state without choosing persistence, backup, or recovery behavior;
- caching authorization with no version or invalidation path;
- sending large binary payloads through a queue instead of passing an object reference;
- treating Pub/Sub as a durable task queue;
- claiming exactly-once processing without consequence-level deduplication;
- deleting a database record before confirming object cleanup or retention policy;
- sharing queue names, cache keys, or buckets across preview and production;
- retrying failed work without preserving attempt identity and evidence.
Each failure comes from confusing where data is convenient to write with who owns it after failure.
environment isolation is part of placement
State placement is incomplete without environment ownership.
A preview should not receive production database credentials, consume development tasks, use the same Redis key namespace, or write uploads into an unscoped production prefix. Each resource needs an owner that can be enumerated, reset, and removed without inference.
The isolation contract should block preview creation when supported managed state cannot be isolated. Silent fallback to shared credentials turns a development convenience into a data boundary violation.
This is the core argument behind production-shaped preview environments: the preview preserves the system shape while giving the proposed change its own state boundary.
where floo fits
floo organizes services and supported managed state around the application environment. Repository configuration holds reviewable topology and policy. Secret values remain outside git. Supported floo-managed Postgres, Redis, and storage participate in environment identity, preview isolation, resource provenance, and teardown rather than appearing as disconnected credentials.
Workers and cron share the application's deployment and observability model. floo does not currently claim that one generic queue primitive erases the need to choose delivery semantics. Applications can use Postgres, Redis patterns, or an external queue according to the contract they need.
Agents can inspect the resulting environment, deploys, logs, diagnostics, and managed attachments through structured interfaces. Humans retain review over high-impact state changes.
Use the worksheet on one real feature before adding infrastructure. If its correct state placement spans services, managed state, schedules, and isolated previews, request access to floo and run the whole shape as one application environment.