Unique ID generator
Hand out unique 64-bit IDs that sort roughly by time, from many processes in many data centres, without a round trip for each ID.
- 1Make IDs in each process: 41 bits of time, 10 of worker, 12 of sequence. No network call per ID.
- 2Uniqueness rests on the worker ID. Lease it from Postgres with an epoch, never pick it at random.
- 3Fence by time: a holder issues IDs only up to its ceiling, and the next holder starts above it.
- 4IDs sort by time only within the clock skew. Time-first IDs keep inserts at one end of the index.
Design a service that hands out unique, roughly sortable 64-bit IDs across many data centres.
- functional
- next() returns a new ID; a batch call returns many.
- unique
- No ID twice, ever, across every data centre.
- size
- 64 bits: fits a bigint column.
- order
- Later IDs mostly sort later; exact order is not required.
- latency
- Under 1 ms; no cross-data-centre call per ID.
- available
- A data centre keeps issuing when the others are cut off.
I will generate IDs in each process and coordinate only the worker ID, with a lease.
| question | answer assumed | it decides |
|---|---|---|
| Must the ID fit in 64 bits? | Yes: existing bigint keys | Snowflake, not UUIDv7 |
| How strict is the order? | Roughly by time; ms ties may go either way | Time-first layout, no global counter |
| How many IDs? | 1 billion a day, 10× peaks | Sequence bits per ms |
| How many generators at once? | Up to a few hundred processes; they come and go | Worker bits, leases |
| Can IDs be guessable? | Internal keys only; public IDs are separate | No random part needed |
| Can a data centre lose its link? | Yes, and it must keep issuing | No cross-data-centre coordination per ID |
I ask whether 64 bits is a hard limit and how strict the order must be. Those two answers choose between UUIDv7 and Snowflake.
- average
- 10⁹ / 86,400 s ≈ 11,574 IDs a second.
- peak
- × 10 ≈ 115,740 a second.
- one worker
- 4,096 per ms × 1,000 = 4,096,000 a second, at most.
- lifetime
- 2⁴¹ ms / (1,000 × 86,400 × 365.25) ≈ 69.7 years.
- workers
- 2¹⁰ = 1,024 live worker IDs, shared by every data centre.
- key bytes
- 10⁹ × 365 × 8 B ≈ 2.9 TB a year as bigint; 5.8 TB as a 16-byte UUID, per index that holds it.
The worker count, not throughput, is the limit to plan for.
A billion IDs a day is about 11,574 a second. One worker can issue 4 million a second, so worker bits are for the number of processes, not for throughput.
| call | success | errors |
|---|---|---|
| ids.next() | int64 | clock behind, lease lost |
| GET /v1/ids?count=100 | 200 ids[] | 400 count, 503 |
| GET /v1/ids/{id}/parts | 200 time, worker, seq | 400 |
- A retry gets new IDs. An ID that no one used is a gap, which is harmless.
- Batches amortize the round trip for remote callers.
- 503 means this process cannot issue now: no lease, or the clock is behind.
The ID comes from a library call in the process. A network service is a fallback for clients that cannot run the library.
- rows
- 1,024: one per worker ID, made once.
- writes
- One renew per process every ttl / 3. 300 processes at a 10 s lease make 90 renews a second.
- per DC
- Run one lease table per data centre, each with its own range of worker IDs, so a cut link stops nothing.
CREATE TABLE IF NOT EXISTS worker_leases (
worker_id int PRIMARY KEY CHECK (worker_id BETWEEN 0 AND 1023)1,
holder text,
epoch bigint2 NOT NULL DEFAULT 0,
expires_at timestamptz3 NOT NULL DEFAULT '-infinity',
ceiling_ms bigint4 NOT NULL DEFAULT 0
);- 110 bits of worker ID: 1,024 rows, no more.
- 2The fencing token. Every acquire adds 1, and a renew must name the current epoch.
- 3Expiry uses the database clock, now(). Process clocks never decide who holds a lease.
- 4The largest ID time the holder may use. The next holder issues only later times.
One row per worker ID. The epoch fences old holders, and the ceiling tells the next holder where to start.
Step 1: Start
- Take the first worker ID whose lease has expired; the epoch goes up by 1.
- Read the last holder's ceiling and wait until the clock passes it.
- Renew once to set this process's own ceiling.
If it fails
No worker ID is free: retry, or alert. Postgres is down: a running process keeps issuing up to its ceiling, and a new process cannot start.
IDs come from memory. Postgres is on the path only at start and every renew, so a database outage stops new processes but not running ones.
| tool | capability | what it gives this design | also used for |
|---|---|---|---|
| Service | A generator in memory, behind a mutex | About 2,900,000 IDs a second from one generator in the lab. | Trace IDs, event IDs |
| Service | A monotonic clock | A wall-clock step after start cannot move the generator back. | Timeouts, latency measurement |
| Postgres | UPDATE with FOR UPDATE SKIP LOCKED | Two starting processes never take the same free worker ID, and neither waits. | Job queues |
| Postgres | A counter column as a fencing token | A renew from an old holder matches no row. | Leader leases, distributed locks |
| Postgres | now(): one clock for every lease | Process clocks never decide expiry. | Holds, sessions |
| Postgres | GREATEST in an UPDATE | The ceiling only goes up, even from a process with a slow clock. | High-water marks |
| Postgres | Sequences (nextval) | Step 1 of the ladder: about 97,000 IDs a second, one round trip each. | Surrogate keys |
| Redis | INCR | Limit Atomic, but a failover to a replica that lagged can hand out a number again. | Counters, rate limits |
The process gives speed. Postgres gives one place that decides who holds each worker ID, with a fencing epoch.
| scheme | bits | coordination | pairs out of order | always in order beyond | status |
|---|---|---|---|---|---|
| Database auto-increment | 64 | every insert, one database | 0.0% | 0 ms | One database |
| Auto-increment, step per server | 64 | none; fixed odd and even | 12.4% | 166 ms, growing | Not approved |
| Ticket server pair | 64 | a round trip per ID or batch | 0.0% | 0 ms, even load | Moderate rate |
| Postgres sequence, CACHE 20 | 64 | one round trip per block | 2.8% | 19 ms | Order not needed |
| UUIDv4 | 128 | none | 50.5% | never | Public IDs only |
| UUIDv7 | 128 | none | 1.0% | 7 ms | 128 bits fine |
| ULID | 128 | none | 1.0% | 7 ms | 128 bits fine |
| Snowflake, leased worker ID | 64 | a lease renew every ttl / 3 | 1.1% | 7 ms | Approved |
Lab: 4 requests a ms for 250 ms over 4 servers whose clocks are off by 0, 3, -2, 5 ms; 498,000 pairs per scheme. Step per server: one server takes 3 of every 4 writes. Every scheme issued 16,000 IDs from 8 clients at once with 0 duplicates.
For one database I use its sequence. For many writers I use UUIDv7 if 128 bits are fine, and Snowflake with leased worker IDs if the key must be 64 bits.
| limit | value | when you hit it |
|---|---|---|
| Time bits run out | 69.7 years | Pick a recent epoch; plan the change of layout years ahead. |
| Worker IDs | 1,024 live | Lease them; move bits from sequence to worker if needed. |
| Sequence in one ms | 4,096 | Wait for the next ms. The lab waited 1 ms. |
| Data centre bits | 0 to 10 | Fixed DC bits waste IDs in small sites; give each DC a range instead. |
| clock steps back 3 ms | lab result | status |
|---|---|---|
| Ignore it | 2 IDs repeat, 2 more sort below issued IDs | Not approved |
| Sleep until the clock passes | 0 repeats; slept 2 ms | Small steps |
| Refuse until it passes | 0 repeats; 3 requests refused | Large steps |
Clock-step run recorded for X4, the same generator.
next(): // one generator per worker ID1
now = clock in ms
IF now < last: // the clock stepped back
IF last - now <= 5 ms: sleep until now > last2
ELSE: RETURN error
IF now == last:
seq = seq + 1
IF seq == 4096: wait for the next ms3; seq = 0
ELSE: seq = 0
last = now
RETURN (now - epoch) << 22 | worker << 12 | seq4- 1All callers in a process share one generator behind a mutex. Two generators with one worker ID would repeat IDs.
- 2Small NTP steps cost a short wait. A large step fails fast instead of blocking.
- 3The sequence never wraps inside one ms. Wrapping would repeat an ID.
- 4Time in the top bits, so IDs sort by time first.
Tested source Go: next
// Next issues the next ID. Under Wait and Fail it never issues an ID at or below the last one.
func (g *Generator) Next() (int64, error) {
g.mu.Lock()
defer g.mu.Unlock()
now := g.clock.NowMS()
if now < g.last && g.policy != Naive {
behind := g.last - now
if g.policy == Fail || behind > g.maxWait {
return 0, fmt.Errorf("%w: %d ms behind the last ID", ErrClockBackwards, behind)
}
for now < g.last { // Wait: sleep until the clock catches up
g.clock.Sleep(g.last - now)
g.slept += g.last - now
now = g.clock.NowMS()
}
}
if now == g.last {
g.seq++
if g.seq > MaxSeq { // 4,096 IDs in this ms: wait for the next ms
for now <= g.last {
g.clock.Sleep(1)
g.slept++
now = g.clock.NowMS()
}
g.seq = 0
}
} else {
g.seq = 0
}
g.last = now
return (now-g.epoch)<<(NodeBits+SeqBits) | g.node<<SeqBits | g.seq, nil
}
Each field sets a limit: 69 years of time, 1,024 workers, 4,096 IDs per ms. When the clock steps back, I wait up to 5 ms and refuse beyond that.
| method | status | why |
|---|---|---|
| Random worker ID | Not approved | 40 processes share an ID with 54% chance (birthday rule). |
| From the host's IP or MAC | Not approved | 10 bits cannot hold an address; truncation clashes. |
| Static, in configuration | Fixed hosts | Breaks with autoscaling and copied configs. |
| Lease, no fencing | Not approved | A paused holder repeats IDs after takeover. |
| Lease with epoch and ceiling | Approved | Holders use disjoint time ranges. |
| Coordinator sequential node | Already run one | Session expiry frees the ID; same pause problem. |
start():
row = take the first expired worker ID1 // SKIP LOCKED
epoch + 1, expires = now() + ttl
floor = row.ceiling // the last holder's promise2
wait until clock > floor3
renew()
renew(): // every ttl / 3
ceiling = clock + ttl
UPDATE ... SET ceiling = GREATEST(ceiling, $c)
WHERE worker_id = me AND epoch = my_epoch4
IF no row: STOP issuing IDs // someone else holds it
next():
id = snowflake.next()
IF time(id) > ceiling5 OR time(id) <= floor:
RETURN error, renew first
RETURN id- 1SKIP LOCKED: two processes that start together take two different rows, without waiting.
- 2The last holder issues no ID with a later time, even if it is paused now and wakes up later.
- 3A process with a slow clock waits; the lab process waited 141 ms. A wait longer than a bound fails instead.
- 4The fencing check. After a takeover the epoch is higher, so this matches no row.
- 5An ID past the promise is never returned. The process renews first, and the renew fails if it lost the lease.
Tested source SQL: acquire · SQL: renew · Go: acquire and renew · Go: next
UPDATE worker_leases w
SET holder = $1, epoch = w.epoch + 1, expires_at = now() + make_interval(secs => $2::float8 / 1000)
FROM (
SELECT worker_id FROM worker_leases
WHERE expires_at < now()
ORDER BY worker_id
LIMIT 1
FOR UPDATE SKIP LOCKED
) free
WHERE w.worker_id = free.worker_id
RETURNING w.worker_id, w.epoch, w.ceiling_ms;UPDATE worker_leases
SET expires_at = now() + make_interval(secs => $3::float8 / 1000),
ceiling_ms = GREATEST(ceiling_ms, $4)
WHERE worker_id = $1 AND epoch = $2
RETURNING ceiling_ms;// Acquire takes the first free worker ID: one whose lease has expired. The update adds 1 to the
// epoch and returns the previous holder's ceiling, which becomes this holder's floor. Under the
// fenced mode it waits until its clock passes the floor, then renews to set its own ceiling.
func Acquire(ctx context.Context, db *pgxpool.Pool, c clocks.Clock, holder string, mode Mode, ttlMS, maxWait, epochMS int64) (*Leased, error) {
l := &Leased{db: db, clock: c, mode: mode, holder: holder, ttlMS: ttlMS, maxWait: maxWait, epochMS: epochMS}
err := db.QueryRow(ctx, leaseQ["acquire"], holder, ttlMS).Scan(&l.Worker, &l.Epoch, &l.Floor)
if errors.Is(err, pgx.ErrNoRows) {
return nil, ErrNoFreeWorker
}
if err != nil {
return nil, fmt.Errorf("acquire a worker ID for %s: %w", holder, err)
}
if mode == Fenced {
if behind := l.Floor - c.NowMS(); behind > maxWait {
return nil, fmt.Errorf("worker %d: %w: %d ms", l.Worker, ErrFloorTooFar, behind)
}
for c.NowMS() <= l.Floor {
c.Sleep(l.Floor - c.NowMS() + 1)
}
}
if l.gen, err = clocks.NewGenerator(c, l.Worker, clocks.Wait, maxWait, epochMS); err != nil {
return nil, fmt.Errorf("generator for worker %d: %w", l.Worker, err)
}
if err := l.Renew(ctx); err != nil {
return nil, err
}
return l, nil
}
// Renew extends the lease and raises the ceiling to the clock plus the lease time. It matches no
// row once another holder has acquired the worker ID, because that holder raised the epoch.
func (l *Leased) Renew(ctx context.Context) error {
want := l.clock.NowMS() + l.ttlMS
var stored int64
err := l.db.QueryRow(ctx, leaseQ["renew"], l.Worker, l.Epoch, l.ttlMS, want).Scan(&stored)
if errors.Is(err, pgx.ErrNoRows) {
return fmt.Errorf("renew worker %d epoch %d: %w", l.Worker, l.Epoch, ErrLeaseLost)
}
if err != nil {
return fmt.Errorf("renew worker %d: %w", l.Worker, err)
}
l.mu.Lock()
l.Ceiling = max(l.Ceiling, want)
l.mu.Unlock()
return nil
}
// Next issues an ID. Under the fenced mode an ID whose time is past the ceiling is never
// returned: the holder must renew first, and a renew fails once the worker ID has a new holder.
func (l *Leased) Next() (int64, error) {
l.mu.Lock()
defer l.mu.Unlock()
id, err := l.gen.Next()
if err != nil {
return 0, fmt.Errorf("worker %d: %w", l.Worker, err)
}
if l.mode == Fenced {
ms, _, _ := clocks.Decode(id)
if at := ms + l.epochMS; at > l.Ceiling || at <= l.Floor {
return 0, fmt.Errorf("worker %d at %d ms, ceiling %d: %w", l.Worker, at, l.Ceiling, ErrPastCeiling)
}
}
return id, nil
}
| true ms | process | action | naive lease | fenced lease |
|---|---|---|---|---|
| 1000 | P1 | acquire | worker 0, epoch 1, ceiling 1200 | worker 0, epoch 1, ceiling 1200 |
| 1000 | P1 | next | ID 1000 · 0 · 0 | ID 1000 · 0 · 0 |
| 1001 | P1 | next | ID 1001 · 0 · 0 | ID 1001 · 0 · 0 |
| 1001 | P1 | pause 300 ms, past the 200 ms lease | paused | paused |
| 1210 | P2 | acquire | worker 0, epoch 2, floor 1200, ceiling 1260 | worker 0, epoch 2, floor 1200, ceiling 1401; waited 141 ms |
| 1220 | P2 | next | ID 1070 · 0 · 0 | ID 1201 · 0 · 0 |
| 1250 | P1 | next | ID 1250 · 0 · 0 | refused: 1250 > ceiling 1200 |
| 1251 | P1 | renew | lost: the epoch moved | lost: the epoch moved |
| 1400 | P2 | next | ID 1250 · 0 · 0: duplicate | ID 1250 · 0 · 0 |
P2's clock runs 150 ms behind P1's. ID shown as time · worker · sequence. A churn run with 6 processes on 2 worker IDs and pauses past the lease issued about 580,000 IDs. Fenced: 0 duplicates. Naive: 5 and 8 in two runs.
I lease the worker ID from Postgres with an epoch. The holder promises a ceiling on its ID times, and the next holder starts above it, so a paused process cannot repeat an ID.
| primary key | WAL bytes per insert | index, 1M rows | leaves full | status |
|---|---|---|---|---|
| bigint identity | 136 | 21.4 MB | 90.1% | Approved |
| Snowflake | 133 | 21.4 MB | 90.1% | Approved |
| UUIDv7 | 151 | 30.1 MB | 90% | Approved |
| UUIDv4 | 2,674 | 37.2 MB | 73% | Not approved |
From the D3 lab: one batch of 10,000 inserts right after a checkpoint, on a table of 1,000,000 rows.
- IDs from one generator always increase.
- IDs from two machines can invert within their clock skew.
- No scheme without a single counter gives exact order across machines.
next():
now = clock in ms
IF now > last_ms:
id = 48 bits of now, then 80 random bits1
ELSE:
id = last id + 12 // same ms or the clock stepped back
IF the 80 bits overflow: RETURN error3
last_ms = max(last_ms, now)
RETURN id- 1Time first: inserts go to the right edge of the index, as with Snowflake and UUIDv7.
- 2Inside one process, IDs keep increasing even in the same ms or after a clock step.
- 3Unreachable in practice: a random start leaves about 2^79 steps on average before the 80 bits overflow.
Tested source Go: ULID next
// Next returns the next ULID. A new millisecond draws 80 fresh random bits; the same or an
// earlier millisecond adds 1 to the last ID, so IDs from one generator always increase.
func (g *ULIDGen) Next() (ULID, error) {
g.mu.Lock()
defer g.mu.Unlock()
now := g.clock.NowMS()
if now > g.ms {
var u ULID
binary.BigEndian.PutUint16(u[0:2], uint16(now>>32))
binary.BigEndian.PutUint32(u[2:6], uint32(now))
binary.BigEndian.PutUint16(u[6:8], uint16(g.rng.Uint32()))
binary.BigEndian.PutUint64(u[8:16], g.rng.Uint64())
g.last, g.ms = u, now
return u, nil
}
for i := 15; i >= 6; i-- { // add 1 to the 80 random bits
g.last[i]++
if g.last[i] != 0 {
return g.last, nil
}
}
return ULID{}, ErrULIDOverflow
}
Time-first IDs append to the right edge of the index. A random UUID touches a new page on almost every insert, which costs WAL and cache.
- size
- 64 bits (8 bytes)
- made
- in each process, with a leased worker ID
- unique
- 0 duplicates in 16,000 IDs from 8 clients at once
- order
- 1.1% of pairs sort in the wrong order. Two IDs made more than 7 ms apart always sort in order.
- index
- 133 WAL bytes per insert after a checkpoint; index 21.4 MB for 1 million rows; leaves 90.1% full.
- first, last
41943040005259669504
limits
- 41 bits of ms: about 69.7 years from the chosen epoch.
- 1,024 live worker IDs.
- 4,096 IDs per ms per worker.
- A clock step back: wait or refuse.
Order: 4 requests a ms for 250 ms over 4 servers, clocks off by 0, 3, -2, 5 ms. Index: 1 million rows, the data modeling lab.
I pick by three questions: must it be 64 bits, must it sort by time, and who may coordinate. Snowflake with a lease answers yes, yes, and only at start.
| event | result | why it is safe | saved by |
|---|---|---|---|
| NTP steps the clock back a few ms | The clock reads earlier than the last ID. | The generator sleeps until it passes the last ID; it never resets the sequence. | Wait policy |
| The clock steps back by seconds | The generator refuses IDs. | Fail fast and alert. A monotonic clock source avoids most steps. | Fail policy |
| A process pauses past its lease | Another process takes its worker ID. | The old holder stops at its ceiling; the new one starts above it. | Ceiling |
| The paused process tries to renew | The renew matches no row. | The epoch moved on takeover. The process stops and starts again. | Epoch |
| Postgres is down | No new leases, no renews. | Running processes issue up to their ceilings, then stop. No ID repeats. | Postgres |
| A process crashes | Its worker ID stays leased until expiry. | After the lease time, another process takes it, above the old ceiling. | Lease expiry |
| A burst above 4,096 IDs per ms | The sequence is full. | The generator waits for the next ms. | Sequence |
| A data centre loses its link | It cannot reach the others. | Its lease table and worker range are local. It keeps issuing. | Local leases |
| step | add | it handles | move up when you see |
|---|---|---|---|
| 1 | A Postgres sequence or identity column. | About 97,000 IDs a second in the lab; exact order. | Several databases or services need IDs, or the ID is needed before the insert. |
| 2 | UUIDv7, made in each service. | Millions a second per process, no coordination. | The key must be 64 bits. |
| 3 | Snowflake, worker ID from configuration. | About 2,900,000 IDs a second per generator. | Processes start and stop on their own: autoscaling, deploys. |
| 4 | Leased worker IDs with an epoch and a ceiling. | Up to 1,024 live processes; the lease costs nothing per ID. | More than 1,024 live processes. |
| 5 | A small ID service per data centre; callers take batches. | A few workers serve every caller. | Top of the ladder. |
Lab rates: 32 clients for 3 seconds on an 8-core laptop. UUIDv7 used one random source per client, so it shares no lock. Demand: 1 billion IDs a day.
I start with the database sequence. I move IDs into the process only when many databases or services must make IDs, and I lease worker IDs once processes come and go.
| question | answer | go deeper |
|---|---|---|
| Why not a hybrid logical clock for the time part? | An HLC never goes backwards and follows the largest time seen. It fixes clock steps, but the worker ID must still be unique. | X4 |
| Can the IDs give exact global order? | Only with one counter or a consensus log, which every ID must reach. That costs a round trip and availability. | X5 |
| Why fence, if the lease already expired? | Expiry alone does not stop a paused process. Fencing makes the old holder's actions harmless after takeover. | T6 |
| Which ID do you show to users? | A separate random public ID, or an encrypted form. Sequential IDs reveal volume and invite guessing. | D3 |
| How do these IDs shard? | Hash the ID: the time bits make range splits hot at the newest end. | X2 |
| Why do random keys cost more in a B-tree? | Each insert lands on a different page, so the cache and the WAL see many more pages. | D1 |
0 of 9 known
Why does a Snowflake ID have 41 bits of time and not 42?
What happens when one worker needs a 4,097th ID in one millisecond?
NTP steps the clock back 3 ms. What does each policy do?
Why not pick the worker ID at random?
A process pauses longer than its lease, and another process takes its worker ID. Why can they not issue the same ID?
What does the epoch column protect?
Are Snowflake IDs from two machines in creation order?
Why is UUIDv4 a poor primary key at a high insert rate?
Why does a sequence with CACHE 20 lose order?
- layout
- 1 + 41 + 10 + 12 bits. 41 bits of ms is about 69.7 years.
- per worker
- 4,096 IDs per ms; 1,024 workers.
- generator
- About 2,900,000 IDs a second, one generator, 32 callers.
- database
- Postgres nextval about 97,000/s; Redis INCR about 43,000/s.
- random ID
- 40 processes on 1,024 IDs: 54% chance of a clash. 10¹² UUIDv4s: about 1 in 11 trillion.
- order
- Clocks 7 ms apart: IDs more than 7 ms apart always in order.
- index
- UUIDv4 wrote 2,674 WAL bytes per insert; Snowflake 133.
Lab: 8-core laptop, Postgres 16 and Redis 8, 32 clients. The layout follows the Snowflake README. Collision chances use the birthday approximation.