Data modeling: a schema from the access patterns
Build a schema in five steps: entities, access patterns, keys and indexes, normalise or denormalise, then choose the store. Two worked examples, a multi-tenant SaaS and a social feed, apply the method.
- 1List the access patterns first: each query, its rate and its latency target. Keys and indexes follow from that list.
- 2Normalise by default. Copy data into a read model only for a hot read that joins many rows.
- 3Put the owner of the data first in every key: tenant_id, user_id. Queries, security and later shards all use it.
- 4Choose types on purpose: time-ordered IDs, timestamptz, exact money, and JSONB only for attributes no query joins on.
- Step 2 drives the rest. Write each query as a sentence with a number: "the 50 newest posts for one reader, 2,300 a second, p99 under 100 ms".
- Step 3: one index range answers one query. The leading key columns are the columns the query names with "=".
- Step 4: copy data only when a hot read would join many rows. Every copy needs a writer that keeps it current.
- Step 5: one store holds the truth. Caches, search indexes and timelines are copies that you can rebuild.
I list the entities, then every query with its rate and latency target. Each query gets a key or an index, and only then do I decide what to copy and where it lives.
| model | stores | built for | status | example |
|---|---|---|---|---|
| Relational | Rows in tables, joined by keys | Any query, joins, constraints, transactions | Default | Postgres |
| Document | One nested record per key | Read or write a whole record | Whole-record access | MongoDB JSONB |
| Wide-column | Partitions of rows sorted by a clustering key | One partition, a range of its rows | Known queries, huge writes | Cassandra |
| Key-value | One value per key | Get and put by key | Lookups by key only | Redis DynamoDB |
| Graph | Nodes and edges | Traverse many hops from one node | Deep traversals | Neo4j |
| Entity-attribute-value rows | One row per attribute | Nothing well | Not approved | Use JSONB instead |
A document store and a wide-column store answer only the queries the model was built for. A new query often means a new copy of the data.
Relational is my default because it answers queries I did not plan for. I pick another model when one known query shape dominates and outgrows one machine.
- The partition key picks one node and one partition.
- The clustering key sorts rows inside it, so a range read is sequential.
- No joins: a second query gets a second table.
In a wide-column store I write one table per query. The partition key is what the query names; the clustering key is the order it reads.
| # | access pattern | per second | p99 | served by |
|---|---|---|---|---|
| Q1 | Sign in: user by email in a tenant | 200 | 50 ms | UQ (tenant_id, lower(email)) |
| Q2 | Check a user's roles, every request | 2,000 | 5 ms | PK of user_roles, prefix |
| Q3 | List the admins of a tenant | 50 | 20 ms | idx (tenant_id, role_id) |
| Q4 | Find users by an attribute | 10 | 100 ms | GIN on attrs |
| Q5 | Write an audit row per change | 500 | 20 ms | append to this month |
| Q6 | Audit page, newest first | 5 | 200 ms | PK (tenant_id, at, id) |
Rates and targets are example requirements for the interview, not measurements. Q2 at 2,000 a second is the hot path; cache the role set per session if it grows.
Every access pattern in this product names a tenant, so tenant_id leads every key. That one choice serves the queries, the security policy and a later shard.
-- A lookup table: the business edits plans, and a plan has attributes.
CREATE TABLE plans (
code text PRIMARY KEY,
seats int NOT NULL,
price_cents bigint NOT NULL CHECK (price_cents >= 0)1
);
CREATE TABLE tenants (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
slug text NOT NULL UNIQUE,
plan text NOT NULL REFERENCES plans (code),
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE users (
tenant_id bigint NOT NULL REFERENCES tenants (id),
id uuid NOT NULL,
email text NOT NULL,
attrs jsonb NOT NULL DEFAULT '{}',
created_at timestamptz NOT NULL DEFAULT now(),
deleted_at timestamptz,
PRIMARY KEY (tenant_id, id)2
);
-- One live account per email in a tenant. A deleted row does not block a new one.
CREATE UNIQUE INDEX users_email ON users (tenant_id, lower(email))
WHERE deleted_at IS NULL3;
CREATE INDEX users_attrs ON users USING gin (attrs jsonb_path_ops4);
CREATE TABLE roles (
tenant_id bigint NOT NULL REFERENCES tenants (id),
id bigint GENERATED ALWAYS AS IDENTITY,
name text NOT NULL,
PRIMARY KEY (tenant_id, id),
UNIQUE (tenant_id, name)
);
-- Many-to-many. Both keys carry tenant_id, so a link can never cross tenants.
CREATE TABLE user_roles (
tenant_id bigint NOT NULL,
user_id uuid NOT NULL,
role_id bigint NOT NULL,
PRIMARY KEY (tenant_id, user_id, role_id),
FOREIGN KEY (tenant_id, user_id) REFERENCES users (tenant_id, id),
FOREIGN KEY (tenant_id, role_id) REFERENCES roles (tenant_id, id)5
);
CREATE INDEX user_roles_role ON user_roles (tenant_id, role_id);
-- An enum: a short, fixed set that only a code change extends.
CREATE TYPE audit_action AS ENUM6 ('create', 'update', 'delete', 'login');
-- Append-only, one partition per month. Retention drops a partition.
CREATE TABLE audit_log (
tenant_id bigint NOT NULL,
at timestamptz NOT NULL DEFAULT now(),
id bigint GENERATED ALWAYS AS IDENTITY,
actor_id uuid,
action audit_action NOT NULL,
target text NOT NULL,
diff jsonb NOT NULL DEFAULT '{}',
PRIMARY KEY (tenant_id, at, id)
) PARTITION BY RANGE (at)7;
CREATE TABLE audit_log_2026_10 PARTITION OF audit_log
FOR VALUES FROM ('2026-10-01') TO ('2026-11-01');
CREATE TABLE audit_log_2026_11 PARTITION OF audit_log
FOR VALUES FROM ('2026-11-01') TO ('2026-12-01');- 1Money as an integer count of cents. Exact sums, and the database refuses a negative price.
- 2tenant_id first: every index range is one tenant, and other tables can reference the pair.
- 3A partial unique index. Only live rows must be unique, so a deleted user can sign up again.
- 4A GIN index for containment (@>). It answers "attrs contains this pair" without a scan.
- 5Both sides carry tenant_id. A link to another tenant's role cannot exist.
- 6An enum for a fixed set. Postgres can add a value later, but not remove one.
- 7One partition per month. Retention drops a partition; no row-by-row DELETE.
| attempt | result |
|---|---|
| Tenant 1 writes a row for tenant 2 | Refused, 42501 |
| A link joins a tenant 1 user to a tenant 2 role | Refused, 23503 |
| A second live account with the same email, other case | Refused, 23505 |
| The service changes an audit row | Refused, 42501 |
| A plan price below zero | Refused, 23514 |
| A request names no tenant | Sees 0 rows |
| A deleted user signs up again | Accepted |
-- Every tenant table filters by the tenant the transaction names. No tenant named: no rows.
ALTER TABLE users ENABLE ROW LEVEL SECURITY;
CREATE POLICY tenant ON users
USING (tenant_id = nullif(current_setting('app.tenant_id', true), '')1::bigint);
-- ... the same policy on roles, user_roles, audit_log
GRANT SELECT, INSERT, UPDATE, DELETE ON users, roles, user_roles TO modeling_app;
GRANT SELECT ON plans, tenants TO modeling_app;
-- The service can add audit rows and read them, never change them.
GRANT SELECT, INSERT ON audit_log TO modeling_app2;- 1The tenant comes from the transaction. Unset, it is NULL, and NULL matches no row.
- 2No UPDATE or DELETE: "permission denied for table audit_log".
The table owner bypasses these policies. The service connects as its own role.
Keys carry the tenant and the email index skips deleted rows. Row-level security filters every query by the tenant, so a query that forgets it sees nothing.
| # | access pattern | per second | served by |
|---|---|---|---|
| F1 | Home feed: 50 newest posts of everyone I follow | 2,300 | PK (user_id, created_at, post_id) |
| F2 | Publish a post | 60 | posts PK, then fan-out |
| F3 | Who follows this author (for fan-out) | 60 | idx (followee_id, follower_id) |
| F4 | One author's posts, newest first | 300 | idx (author_id, created_at) |
| F5 | Follow or unfollow | 20 | follows PK |
- reads
- 10 million daily users × 20 feed opens = 200 million a day ≈ 2,300 a second.
- writes
- 10 million users × 0.5 posts a day = 5 million a day ≈ 60 a second.
- ratio
- About 40 reads per post: do the work once at write time.
CREATE TABLE users (
id bigint PRIMARY KEY,
handle text NOT NULL UNIQUE
);
-- Many-to-many between users. The key answers "whom do I follow".
CREATE TABLE follows (
follower_id bigint NOT NULL REFERENCES users (id),
followee_id bigint NOT NULL REFERENCES users (id),
PRIMARY KEY (follower_id, followee_id)1
);
-- The reverse index answers "who follows me", which fan-out needs.
CREATE INDEX follows_followee ON follows (followee_id, follower_id);2
CREATE TABLE posts (
id bigint PRIMARY KEY,
author_id bigint NOT NULL REFERENCES users (id),
created_at timestamptz NOT NULL,
body text NOT NULL
);
CREATE INDEX posts_author_time ON posts (author_id, created_at DESC);3
-- The read model: one row per (reader, post). Written at post time, read in one range scan.
CREATE TABLE timeline (
user_id bigint NOT NULL,
created_at timestamptz NOT NULL,
post_id bigint NOT NULL,
author_id bigint NOT NULL,
PRIMARY KEY (user_id, created_at, post_id)4
);- 1A many-to-many link. The key answers "whom do I follow" with one range.
- 2The reverse direction. Fan-out reads it to find every follower of an author.
- 3One author's posts, newest first, for profiles and for the pull side of the hybrid.
- 4The read model. One range scan, newest first, returns a feed page. post_id breaks ties.
The hot query is the home feed: the 50 newest posts from everyone I follow. I give it its own table, keyed by reader and time.
post(author, body):
BEGIN
insert the post // the truth
FOR EACH follower OF author: // one INSERT ... SELECT1
add (follower, created_at, post) to timeline
COMMIT // rows written = followers2
read_feed(reader):
ids = 50 newest timeline rows of reader // one index range3
RETURN the posts of ids // 50 lookups by primary key
trim(reader, keep):
delete timeline rows of reader older than the newest keep4- 1The fan-out is one statement: read the followers from the reverse index, write one timeline row each.
- 2The cost moves to the writer. 10,001 followers took 41 ms; 100,001 took 497 ms.
- 3Constant work: 100 rows read, whether the reader follows 10 authors or 5,000.
- 4A cap keeps each timeline bounded. Older pages fall back to the join, which is rare.
Tested source Go: post and fan out · SQL: fan-out, read, trim
// Post writes a post and fans it out to every follower's timeline, in one transaction. It
// returns the number of timeline rows written.
func Post(ctx context.Context, db *pgxpool.Pool, id, author int64, at time.Time, body string) (int64, error) {
tx, err := db.Begin(ctx)
if err != nil {
return 0, fmt.Errorf("begin: %w", err)
}
if _, err := tx.Exec(ctx, Q("feed", "post"), id, author, at, body); err != nil {
return 0, fmt.Errorf("insert post: %w", rollback(ctx, tx, err))
}
tag, err := tx.Exec(ctx, Q("feed", "fan_out"), id, author, at)
if err != nil {
return 0, fmt.Errorf("fan out: %w", rollback(ctx, tx, err))
}
if err := tx.Commit(ctx); err != nil {
return 0, fmt.Errorf("commit: %w", err)
}
return tag.RowsAffected(), nil
}
-- One row per follower of the author. The cost grows with the follower count.
INSERT INTO timeline (user_id, created_at, post_id, author_id)
SELECT f.follower_id, $3, $1, $2 FROM follows f WHERE f.followee_id = $2;
-- Denormalised: one range scan of the reader's timeline, then 50 lookups by primary key.
SELECT p.id, p.author_id, p.created_at, p.body
FROM timeline t
JOIN posts p ON p.id = t.post_id
WHERE t.user_id = $1
ORDER BY t.created_at DESC, t.post_id DESC
LIMIT 50;
-- Keep the newest $2 rows per reader, so a timeline never grows without bound.
DELETE FROM timeline t
WHERE t.user_id = $1
AND (t.created_at, t.post_id) < (
SELECT created_at, post_id FROM timeline
WHERE user_id = $1
ORDER BY created_at DESC, post_id DESC
OFFSET $2 - 1 LIMIT 1);| design | status | cost |
|---|---|---|
| Join on every read | Few follows, low rate | At 1,000 follows: 202,000 rows and 25 ms per read. |
| Fan-out on write to a timeline | Approved | One row per follower per post. Reads cost 0.20 ms. |
| Hybrid: push most authors, pull authors with very many followers | Large audiences | A read merges the timeline with a few authors' newest posts. |
| Copy the post body into each timeline row | Not approved | An edit must rewrite every copy. Store the post ID only. |
| Timeline with no cap | Not approved | Grows by one row per followed post, forever. |
I write a timeline row per follower when the post is made. Each read is then one range scan; in the lab that was 127 times faster than the join at 1,000 follows.
Following 1,000 authors, the join read 202,000 rows and took 25 ms. The timeline read 100 rows and took 0.20 ms. The join was 127 times slower.
5,000 authors with 40 posts each. Median of 200 runs from one client. With 32 clients at 1,000 follows: 199 joins a second against 29,703 timeline reads.
Postgres 16.14 on one 16-thread laptop, shared with other runs. Read the numbers as ratios and orders of magnitude.
The join grows with the number of follows and the timeline does not. Random UUIDs cost WAL and index space, and a closure table trades storage and move cost for fast subtree reads.
| tool | capability | what it gives this design | also used for |
|---|---|---|---|
| Postgres | Composite primary keys and B-tree indexes | The leading columns pick one range: one tenant, one reader, newest first. | Every access pattern |
| Postgres | Composite foreign keys | A link must match the tenant on both sides. | Any owned child row |
| Postgres | Partial and expression indexes | Unique email per tenant, ignoring case and deleted rows. | Soft deletes, "one active per user" |
| Postgres | Row-level security policies | Every query filters by the transaction's tenant, even one with a bug. | Per-user data, shared tables |
| Postgres | GRANT per table and role | The service can append audit rows and never change them. | Ledgers, event logs |
| Postgres | JSONB with a GIN index | Flexible attributes that a containment query can still find. | Settings, product attributes |
| Postgres | Declarative partitioning by range | Audit retention by dropping a month. | Logs, events, time series |
| Postgres | WITH RECURSIVE | Walk an adjacency list up or down in one query. | Org charts, threads, bills of materials |
| Postgres | INSERT ... SELECT | Fan-out to every follower in one statement. | Backfills, copies between tables |
| Postgres | numeric, timestamptz | Exact money; instants that do not depend on the session time zone. | Billing, scheduling |
| Postgres | EXPLAIN (ANALYZE, BUFFERS, WAL) | Rows, pages and WAL bytes of one query: the numbers on this sheet. | Index reviews, capacity checks |
| Cassandra | Partition key plus clustering key | One table per query; rows sorted inside a partition. | Messages, events per device |
| DynamoDB | Partition key plus sort key; global secondary indexes | The same model, managed. Reads from a global secondary index are always eventually consistent. | Carts, sessions at large scale |
Postgres gives me composite keys, partial and expression indexes, row-level security, JSONB with GIN, partitions and recursive queries. That covers most schemas before any second store.
| model | read a subtree | move a subtree | storage | fits |
|---|---|---|---|---|
| Adjacency list | Step per level 117 ms | 1 row 0.94 ms | 26.4 MB | Frequent moves, shallow reads |
| Materialised path | One range 24 ms | 4,681 rows 54 ms | 53.3 MB | Categories, folders, breadcrumbs |
| Closure table | One range 11 ms | 18,724 rows 310 ms | 235.2 MB | Read-heavy trees, depth queries |
subtree(n), adjacency list:
rows = {n}
REPEAT: rows += children of the last level // one index scan per level1
UNTIL a level adds nothing
subtree(n), materialised path:
p = path of n // '/1/2/'
RETURN rows WHERE p <= path < p + '~'2 // one index range
subtree(n), closure table:
RETURN descendants WHERE ancestor = n3 // one index range, too
move(n, under m):
adjacency: set parent of n to m // 1 row
path: rewrite the prefix of every row in the subtree4
closure: delete (old ancestor, node) pairs
insert (new ancestor, node) pairs5- 1A recursive query: one round per level, each an index scan on parent_id.
- 2In byte order, every path with this prefix sits in one contiguous index range.
- 3The table stores every ancestor pair, so the subtree is already listed.
- 44,681 rows for a subtree of 4,681 nodes.
- 5Subtree size × (old + new ancestors): 18,724 rows in the lab.
Tested source SQL: subtree, three ways · SQL: move, three ways
-- adjacency list
-- Walk down one level per step.
WITH RECURSIVE sub AS (
SELECT id FROM cat_adjacency WHERE id = $1
UNION ALL
SELECT c.id FROM cat_adjacency c JOIN sub ON c.parent_id = sub.id
)
SELECT id FROM sub ORDER BY id;
-- materialised path
-- One range scan: every path that starts with the node's path. '~' sorts after '/' and digits.
SELECT c.id
FROM cat_path c, (SELECT path FROM cat_path WHERE id = $1) s
WHERE c.path >= s.path AND c.path < s.path || '~'
ORDER BY c.id;
-- closure table
-- One index range: every descendant of $1.
SELECT descendant_id AS id FROM cat_closure WHERE ancestor_id = $1 ORDER BY id;-- Move node $1 under $2: one row changes.
UPDATE cat_adjacency SET parent_id = $2 WHERE id = $1;
-- Every row in the subtree gets a new prefix.
UPDATE cat_path c
SET path = (SELECT path FROM cat_path WHERE id = $2) || substr(c.path, length(m.parent_path) + 1)
FROM (SELECT path, left(path, length(path) - length(id::text) - 1) AS parent_path
FROM cat_path WHERE id = $1) m
WHERE c.path >= m.path AND c.path < m.path || '~';
-- Remove the links from the old ancestors to every node in the subtree.
DELETE FROM cat_closure c
USING cat_closure sub, cat_closure up
WHERE sub.ancestor_id = $1
AND up.descendant_id = $1 AND up.ancestor_id <> $1
AND c.ancestor_id = up.ancestor_id AND c.descendant_id = sub.descendant_id;
-- Link every new ancestor to every node in the subtree.
INSERT INTO cat_closure (ancestor_id, descendant_id, depth)
SELECT up.ancestor_id, sub.descendant_id, up.depth + sub.depth + 1
FROM cat_closure up, cat_closure sub
WHERE up.descendant_id = $2 AND sub.ancestor_id = $1;Tree of 299,593 nodes, 8 children each. Subtree: 37,449 nodes. Move: 4,681 nodes. The closure table holds 2,054,353 rows. Postgres also ships ltree, a path type with its own operators.
An adjacency list moves a subtree by changing one row. A path or a closure table reads a subtree as one range but rewrites rows on every move. I choose by which operation is frequent.
| ID | bytes | made by | status |
|---|---|---|---|
| bigint identity | 8 | The database | One database |
| Snowflake: time, worker, sequence | 8 | Any process with a worker number | Approved |
| UUIDv7 | 16 | Anyone, no coordination | Approved |
| UUIDv4 as the primary key | 16 | Anyone | Low insert rate |
| Sequential IDs in public URLs | 8 | The database | Not approved |
uuid_v4(): 122 random bits // lands anywhere
uuid_v7(): 48-bit Unix milliseconds, then random // sorts by time1
snowflake(): 41 bits ms | 10 bits worker | 12 bits sequence2
IF 4,096 IDs used in this ms: wait for the next ms- 1The time comes first, so a later ID sorts later, byte by byte (RFC 9562).
- 24,096 IDs per millisecond per worker, by construction.
Tested source Go: ID generators
// NewV4 returns a random UUID (version 4): 122 random bits. Consecutive IDs land at random
// places in a B-tree index.
func NewV4(r *rand.Rand) UUID {
var u UUID
binary.BigEndian.PutUint64(u[0:8], r.Uint64())
binary.BigEndian.PutUint64(u[8:16], r.Uint64())
u[6] = u[6]&0x0f | 0x40 // version 4
u[8] = u[8]&0x3f | 0x80 // variant 10
return u
}
// NewV7 returns a time-ordered UUID (version 7, RFC 9562): a 48-bit Unix time in milliseconds,
// then random bits. IDs made later sort later, so inserts go to the right edge of the index.
func NewV7(r *rand.Rand, now time.Time) UUID {
var u UUID
ms := uint64(now.UnixMilli())
binary.BigEndian.PutUint64(u[0:8], ms<<16|r.Uint64()&0xffff)
binary.BigEndian.PutUint64(u[8:16], r.Uint64())
u[6] = u[6]&0x0f | 0x70 // version 7
u[8] = u[8]&0x3f | 0x80 // variant 10
return u
}
// Snowflake makes 64-bit IDs: 41 bits of milliseconds since Epoch, 10 bits of worker number,
// 12 bits of sequence within one millisecond. Each process needs its own worker number.
type Snowflake struct {
Epoch time.Time
Worker int64 // 0 to 1023
lastMS int64
seq int64
}
// Next returns the next ID at time now. It returns false when the 4,096 IDs of this millisecond
// are used up; the caller waits for the next millisecond.
func (s *Snowflake) Next(now time.Time) (int64, bool) {
ms := now.Sub(s.Epoch).Milliseconds()
if ms == s.lastMS {
s.seq++
if s.seq > 0xfff {
return 0, false
}
} else {
s.lastMS, s.seq = ms, 0
}
return ms<<22 | s.Worker<<12 | s.seq, true
}
Index after 1,000,000 rows: bigint 21.4 MB, UUIDv7 30.1 MB, UUIDv4 37.2 MB. Sequential IDs reveal your volume and invite guessing; expose a UUID instead.
I use a time-ordered ID: UUIDv7 when clients or many services make IDs, a bigint when one database does. Random UUIDv4 keys wrote 18 times the WAL per insert in the lab.
| need | pattern | status |
|---|---|---|
| One-to-many | FK on the many side, with an index | Approved |
| Many-to-many | Link table; PK both ways plus a reverse index | Approved |
| "Belongs to one of several tables" | One nullable FK per parent, CHECK exactly one | Approved |
| "Belongs to one of several tables" | parent_type plus parent_id | Not approved |
| Soft delete | deleted_at, partial unique indexes | Every query filters it |
| Timestamps | created_at, updated_at as timestamptz | Approved |
| An instant in time | timestamp without time zone | Not approved |
| Money | bigint minor units, or numeric | Approved |
| Money | float8, real | Not approved |
| A fixed set of states | enum | Set changes with code |
| A set the business edits | lookup table with an FK | Approved |
| Attributes that vary by row | JSONB with a GIN index | Not joined or constrained |
- float8
- 0.10 + 0.20 = 0.30000000000000004; equal to 0.30: false.
- numeric
- 0.10 + 0.20 = 0.30; equal to 0.30: true.
- timestamptz
- Written as 09:00 in Kolkata, read in Los Angeles: 2026-10-04 20:30:00-07. Same instant.
- timestamp
- Same write, same read: 2026-10-05 09:00:00. A different instant.
Recorded from Postgres. A parent_type column cannot carry a foreign key, so the database cannot check the link.
I index every foreign key and use link tables for many-to-many. Instants are timestamptz, money is an integer or numeric, and JSONB holds only attributes no query joins on.
| event | result | why it stays correct, or the fix | saved by |
|---|---|---|---|
| N+1: load 50 posts one query each | 51 round trips: 8.8 ms against 0.53 ms for one query, on one machine. | Fetch the 50 posts in one query with a join or an IN list. On a network every round trip adds latency. | One query |
| A reader follows 5,000 authors | The timeline grows by every followed post, without bound. | Trim each timeline to the newest few hundred rows. Older pages use the join. | Trim |
| An author with millions of followers posts | Millions of timeline rows for one post: 100,001 took 497 ms. | Do not fan out large authors. Merge their newest posts at read time. | Hybrid |
| A partition key with few values or one hot value | One partition takes most writes: a date, a country, one big tenant. | Lead with a high-cardinality owner (tenant_id, user_id); move a huge tenant to its own shard. | Key choice |
| A query forgets the tenant filter | Without a policy it reads every tenant. | Row-level security filters by the transaction's tenant. No tenant set: 0 rows. | RLS |
| Random UUID primary keys at a high insert rate | 2,674 WAL bytes per row against 151; leaves 73% full. | Use UUIDv7 or a bigint. More WAL also means more replica traffic. | UUIDv7 |
| A price stored as float | Sums drift: 0.30000000000000004. | bigint cents or numeric. | numeric |
| A closure table subtree moves | 18,724 rows rewritten in one transaction: 310 ms. | If moves are frequent, use an adjacency list or a path. | Model choice |
| A post is edited after fan-out | Timelines hold only the post ID. | The read looks the post up by key, so it always shows the current body. | IDs only |
| step | add | it handles | move up when you see |
|---|---|---|---|
| 1 | One Postgres, normalised, one index per access pattern. | Every query, with joins. The feed join served 199 reads a second at 1,000 follows. | A hot read joins many rows, and its latency grows with the data. |
| 2 | A read model in Postgres: the timeline, written by fan-out. | 29,703 feed reads a second in the lab, flat as follows grow. | Tables grow so large that retention and index size become work. |
| 3 | Partitions and caps: audit by month, timelines trimmed. | Retention by dropping a partition; bounded rows per reader. | Feed reads take most of the primary's CPU. |
| 4 | A timeline cache: a capped sorted set per active reader in Redis. | Hot feeds from memory; Postgres keeps the truth and rebuilds a lost set. | Writes or storage pass one primary. |
| 5 | Shards by the owner key: tenant_id or user_id. | Every access pattern already names the key, so each query stays on one shard. | Top of the ladder. |
Demand example: 10 million daily users × 20 feed opens = 200,000,000 / 86,400 ≈ 2,315 a second. Capacity measured in the lab with 32 clients; read it as an order of magnitude.
I start normalised on one Postgres with an index per access pattern. I add a read model only for a hot read that joins many rows. I shard last, by the key that already leads every table.
0 of 10 known
Why list the access patterns before you draw any table?
Why put tenant_id first in every primary key of a SaaS schema?
A user is soft deleted, then signs up again with the same email. What fails, and what is the fix?
The home feed takes 25 ms for a user who follows 1,000 authors. Why, and what do you change?
An author with 10 million followers posts. What does fan-out on write cost?
Why does a random UUID primary key cost more than a time-ordered one?
When do you choose a closure table over a materialised path?
How do you store a price?
Enum or lookup table for an order status?
When does a wide-column store fit chat messages better than Postgres?
- feed join
- 25 ms at 1,000 follows; 0.82 ms at 10. Grows with follows.
- timeline
- 0.20 ms, 100 rows read, at any follow count.
- reads/s
- 199 joins against 29,703 timeline reads, 32 clients.
- fan-out
- 100,001 timeline rows in about 497 ms.
- N+1
- 51 queries 8.8 ms; one query 0.53 ms.
- UUIDv4
- 2,674 WAL bytes per insert against 151 for UUIDv7; index 1.7× a bigint's.
- closure
- 6.9 rows per node at depth 6.
- snowflake
- 4,096 IDs per ms per worker (12-bit sequence).
Postgres 16 on an 8-core laptop shared with other runs; latency is the median of 200 runs from one client. A server that flushes each commit to durable storage writes slower.