Sql on EXPLAIN ANALYZE

Joins That Lie: The Cardinality Problem

Sat, 18 Apr 2026 00:00:00 +0000

TL;DR

Most silently wrong SQL comes from the same root cause: a join that multiplies rows in a way the author didn’t expect. Join a 1:N relationship and every parent row appears N times. Add a second 1:N join to the same parent and rows multiply again. Aggregations built on top of those rows — SUM, COUNT, AVG — inflate without producing any error. The query looks correct, the results look plausible, and the numbers are wrong. The fix isn’t more careful SQL; it’s understanding the cardinality of every join before you write the aggregation.

There’s a category of SQL bug that never throws an error, never fails code review, and never shows up in tests. The query runs. The results look reasonable. Someone ships a dashboard, and a month later finance asks why revenue is 40% higher than what the billing system reports. That 40% isn’t a bug in the data — it’s a join that multiplied rows, and a SUM that dutifully added them all up.

The tricky part is that structurally the query is fine. The joins are valid. The filters are valid. The aggregation is valid. Every individual piece is correct. But the cardinality of the relationships — how many child rows exist per parent, and how that changes when multiple child tables are joined at once — is doing damage the query never surfaces.

Cardinality, briefly

Cardinality describes the number of rows on each side of a relationship:

One-to-one (1:1) — each row in table A matches at most one row in table B. Less common than 1:N, but legitimately used for optional extensions (splitting off rarely-accessed or sensitive columns into a side table), inheritance patterns (a base table with specialized sub-tables), or separating hot and cold data for caching and storage reasons. A 1:1 join preserves row count.
One-to-many (1:N) — each row in A matches zero or more rows in B. The common case: one order has many order items, one user has many sessions, one post has many comments. Joining A to B duplicates the parent row once per matching child. If a parent has zero children, an inner join drops it entirely; a left join keeps it with NULLs on the child side. This difference matters — and it’s the source of another whole class of silent bugs.
Many-to-many (N:M) — rows in A match many rows in B and vice versa. Always implemented through a bridge table (junction table) that sits between them. A bridge is just two 1:N relationships back-to-back: the bridge table holds a foreign key to A and a foreign key to B, with each row pairing one A with one B. A has many bridge rows, and B has many bridge rows. Joining through it multiplies by the cardinality on both sides.

The shape of the relationship determines what a join does to row counts. This is where aggregations start to lie.

Schema cardinality vs. data cardinality

There’s a distinction worth naming: what the schema allows vs. what the data actually contains. A foreign key from user_profiles.user_id to users.id with a unique constraint is 1:1 at the schema level. A column typed as 1:N by constraint can be 1:1 in practice — if every order in your system happens to have exactly one line item, the relationship is legally 1:N but effectively 1:1. This matters for query planning (the optimizer uses constraints, not observed data), index choice, and reasoning about whether a join can actually multiply rows. A query that’s safe against the current data can break as soon as the data starts exercising the cardinality the schema permits.

The row multiplication problem

The examples in this article use a deliberately simple customers / orders / order_items schema so the mechanics are easy to follow. In real systems the shape changes constantly — invoices and payments, subscriptions and usage events, tickets and messages, events and dimensions in a warehouse. The permutations are endless, but the underlying failure is the same: a join that multiplies rows in a way the author didn’t expect, feeding an aggregation that now lies. Once the pattern is visible in one schema, it’s visible everywhere.

Consider a schema everyone has seen some version of:

CREATE TABLE customers (
    id BIGINT PRIMARY KEY,
    name VARCHAR(255) NOT NULL
);

CREATE TABLE orders (
    id BIGINT PRIMARY KEY,
    customer_id BIGINT NOT NULL REFERENCES customers(id),
    total_cents BIGINT NOT NULL
);

CREATE TABLE order_items (
    id BIGINT PRIMARY KEY,
    order_id BIGINT NOT NULL REFERENCES orders(id),
    price_cents BIGINT NOT NULL,
    quantity INT NOT NULL
);

A simple question: what’s the total revenue per customer? The obvious query:

SELECT c.name, SUM(o.total_cents) AS revenue
FROM customers c
JOIN orders o ON o.customer_id = c.id
GROUP BY c.name;

This is correct. One row per order, total_cents summed per customer. Now someone asks: “can we also see how many items they bought?” The change looks trivial — add a join and a count:

SELECT
    c.name,
    SUM(o.total_cents) AS revenue,
    COUNT(oi.id) AS items_purchased
FROM customers c
JOIN orders o ON o.customer_id = c.id
JOIN order_items oi ON oi.order_id = o.id
GROUP BY c.name;

The items_purchased count is correct. The revenue is wrong.

Here’s what happened. orders to order_items is 1:N. Joining them multiplies each order row by the number of items it contains. An order with 5 items now appears 5 times in the result set — once per item. And total_cents — which lives on the orders row — is duplicated in each of those 5 copies.

SUM(o.total_cents) now sums the same order total once per item. A $100 order with 5 items contributes $500. Revenue is inflated by the average number of items per order.

The query runs. The numbers look like revenue. Nothing is flagged. The dashboard ships.

Why it's easy to miss

The inflation is proportional to the cardinality of the join, so it affects every row by roughly the same factor. Totals grow uniformly, relative rankings stay intact, and top-10 lists still look “right.” There’s nothing that stands out as obviously wrong — except the grand total doesn’t match the source system.

The bridge table trap

Many-to-many relationships make this problem worse because the multiplication happens in both directions. Take a schema with products, orders, and promotions:

CREATE TABLE order_items (
    id BIGINT PRIMARY KEY,
    order_id BIGINT NOT NULL,
    product_id BIGINT NOT NULL,
    price_cents BIGINT NOT NULL
);

CREATE TABLE order_item_promotions (
    order_item_id BIGINT NOT NULL,
    promotion_id BIGINT NOT NULL,
    PRIMARY KEY (order_item_id, promotion_id)
);

CREATE TABLE promotions (
    id BIGINT PRIMARY KEY,
    name VARCHAR(255) NOT NULL
);

An order item can have multiple promotions applied to it (a percentage discount stacked with a free shipping promo). Query: total revenue, broken down by promotion:

SELECT p.name, SUM(oi.price_cents) AS revenue
FROM order_items oi
JOIN order_item_promotions oip ON oip.order_item_id = oi.id
JOIN promotions p ON p.id = oip.promotion_id
GROUP BY p.name;

If an order item had two promotions, its price_cents shows up twice — once under each promotion. Sum those up and total revenue exceeds actual revenue. Worse, if you then compare “sum across all promotions” to “total revenue from order_items,” the numbers don’t tie out, and there’s no obvious reason why.

The bridge table is doing exactly what it’s supposed to do. The query is doing exactly what the SQL says. But the meaning of the aggregation drifts as soon as you cross a many-to-many boundary.

A variation of the grain problem shows up in schemas where related tables each carry their own independently-moving date column — orders vs. shipments, subscriptions vs. invoices, tickets vs. updates, orders vs. returns. When a question is time-bounded (“Q1 revenue from items shipped in Q1”), the date filter has to land on the column that matches the question — filtering on both tables “to be safe” silently excludes rows whose dates diverge. An order placed in December with items shipping in January is a Q1 shipment; a filter on orders.created_at throws it out.

The rule is the same as for row multiplication: pick the grain that matches the question, once. If the question is about shipments, filter on shipped_at. If it’s about orders, filter on created_at. Combining both feels more rigorous and quietly returns the wrong set.

How to diagnose it

The symptom is always the same: a number that doesn’t match what another system says it should be. Revenue doesn’t match billing. User counts don’t match the auth service. Item totals don’t match inventory. When that happens, the first thing to check isn’t the aggregation — it’s the row count at each stage of the query.

Take the aggregation off and see what you’re actually summing:

-- Original (wrong) query
SELECT c.name, SUM(o.total_cents) AS revenue
FROM customers c
JOIN orders o ON o.customer_id = c.id
JOIN order_items oi ON oi.order_id = o.id
GROUP BY c.name;

-- Diagnostic: see the raw rows for one customer
SELECT c.name, o.id AS order_id, o.total_cents, oi.id AS item_id
FROM customers c
JOIN orders o ON o.customer_id = c.id
JOIN order_items oi ON oi.order_id = o.id
WHERE c.id = 42
ORDER BY o.id;

If the same order_id and total_cents appear on multiple rows, the sum is going to double-count. Seeing the raw rows makes the multiplication obvious in a way the aggregated output never does.

Another useful check: compare counts at each level independently.

-- Count orders directly
SELECT COUNT(*) FROM orders WHERE customer_id = 42;
-- Returns: 3

-- Count orders through the joined query
SELECT COUNT(*) FROM orders o
JOIN order_items oi ON oi.order_id = o.id
WHERE o.customer_id = 42;
-- Returns: 12

-- The 4x multiplication is the join's cardinality

When the two numbers don’t match, the join is multiplying rows. Every aggregation downstream of that join is suspect.

How to solve it

There’s no single fix — the right technique depends on whether the aggregation lives on the parent or the child, and how many cardinality boundaries you’re crossing.

Aggregate at the correct grain, then join

The cleanest approach is usually to do each aggregation at the table where the data actually lives, then join the pre-aggregated results together. This keeps row counts under control and makes the query’s intent obvious.

WITH order_stats AS (
    SELECT customer_id, SUM(total_cents) AS revenue
    FROM orders
    GROUP BY customer_id
),
item_stats AS (
    SELECT o.customer_id, COUNT(oi.id) AS items_purchased
    FROM orders o
    JOIN order_items oi ON oi.order_id = o.id
    GROUP BY o.customer_id
)
SELECT
    c.name,
    order_stats.revenue,
    item_stats.items_purchased
FROM customers c
LEFT JOIN order_stats ON order_stats.customer_id = c.id
LEFT JOIN item_stats  ON item_stats.customer_id  = c.id;

Revenue is summed from orders where it lives, once per order. Items are counted through the orders→order_items join separately. Then both are joined back to customers. Each aggregation happens at its correct grain, and the final join is 1:1:1 — no multiplication.

It looks more verbose. It is. That’s the point. The verbosity is making the cardinality explicit instead of hiding it behind a single flat join.

Use `DISTINCT` inside the aggregate — with caution

When the multiplication is already there, SUM(DISTINCT ...) can sometimes paper over it:

SELECT
    c.name,
    SUM(DISTINCT o.total_cents) AS revenue  -- suspicious
FROM customers c
JOIN orders o ON o.customer_id = c.id
JOIN order_items oi ON oi.order_id = o.id
GROUP BY c.name;

This only works if total_cents is guaranteed to be unique across the duplicated rows. If two different orders happen to have the same total, DISTINCT collapses them into one and revenue drops. It’s fragile — correct for the query but wrong for the data.

COUNT(DISTINCT o.id) is safer because id is always unique by definition. Use DISTINCT on natural keys, not on aggregated values.

Window functions for “per parent” aggregates

When you need a running or per-group aggregate without collapsing rows, window functions keep the row count intact and do the math within a partition:

SELECT
    o.id AS order_id,
    o.customer_id,
    oi.id AS item_id,
    oi.price_cents,
    SUM(oi.price_cents) OVER (PARTITION BY o.id) AS order_total,
    SUM(oi.price_cents) OVER (PARTITION BY o.customer_id) AS customer_total
FROM orders o
JOIN order_items oi ON oi.order_id = o.id;

No group-by, no row collapsing, totals computed at the right grain. The cost is a result set the size of order_items, so use this pattern when the row-level detail is actually needed — not as a default replacement for GROUP BY.

LATERAL joins and correlated subqueries

When you need a per-row aggregate — the total for each order, or the most recent child row — a lateral join keeps the parent’s grain and evaluates the child aggregation row by row.

-- PostgreSQL: LATERAL join
SELECT o.id, o.customer_id, items.total, items.item_count
FROM orders o,
LATERAL (
    SELECT SUM(price_cents) AS total, COUNT(*) AS item_count
    FROM order_items
    WHERE order_id = o.id
) items;

One row per order, aggregation computed inside the lateral subquery, no multiplication. This is often faster than joining and then grouping, especially when orders is heavily filtered and order_items is large.

Schema-level defenses

Query-level fixes only work if the person writing the query knows to apply them. Schema-level guarantees work for every query, forever.

Foreign keys tell the query planner about cardinality. PostgreSQL in particular uses FK metadata to make join-order decisions and to eliminate redundant joins during planning. Beyond the integrity benefits, FKs make the shape of the data visible to both humans and the planner. (Foreign Keys Are Not Optional goes deeper on why skipping them compounds into silent corruption over time.)

Unique constraints on bridge tables prevent accidental many-to-many explosions. A bridge table with PRIMARY KEY (a_id, b_id) can’t contain duplicates — so joining through it can’t multiply rows because of duplicate bridge entries (only because of legitimate N:M relationships).

CREATE TABLE order_item_promotions (
    order_item_id BIGINT NOT NULL REFERENCES order_items(id),
    promotion_id BIGINT NOT NULL REFERENCES promotions(id),
    PRIMARY KEY (order_item_id, promotion_id)  -- prevents duplicates
);

Without that composite primary key, a bug in the application layer that inserts the same (order_item_id, promotion_id) pair twice would silently double revenue for that item in any query joining through the bridge. With it, the database rejects the duplicate at write time.

Schema comments on tables and columns document the cardinality and semantics that aren’t visible from the DDL. A line like COMMENT ON TABLE order_item_promotions IS 'N:M bridge. One row per (item, promotion). Joining this multiplies order_item rows by avg promotions-per-item.' tells every future engineer exactly what the table does to row counts. (Comment Your Schema covers the mechanics across MySQL and PostgreSQL and why this metadata layer is almost always empty.)

Denormalized totals, when the trade-off is worth it. For heavily queried aggregates (order totals, user balance, post comment counts), storing the aggregate on the parent table eliminates the join entirely. The write-path cost is keeping the denormalized value consistent — either through application code, triggers, or scheduled reconciliation. For high-read, low-write aggregates, the read simplicity often wins. For everything else, computing on demand is cleaner.

Denormalization has its own failure mode

A stored orders.total_cents that’s out of sync with SUM(order_items.price_cents) is its own form of silent corruption — just moved from the query layer to the write layer. Either invest in keeping it consistent (triggers, reconciliation jobs) or don’t denormalize it at all. A half-maintained denormalized aggregate is worse than no denormalization.

The mental model

The shortcut that prevents most of these bugs: before writing an aggregation, picture the row count at every stage of the query.

Start with the leftmost table. How many rows?
Each join: does this multiply, preserve, or filter the row count?
At the point where the aggregate runs: what is the grain of each row? What does “one row” represent?
Does the aggregate make sense at that grain?

When the answer is “one row represents an order item, but I’m summing an order-level field,” the bug is already obvious. When the answer is “one row represents an order, and I’m summing order totals,” the query is correct.

This isn’t a skill that scales with query complexity — it’s a habit that kicks in before the query gets written. The senior engineers who never seem to hit these bugs aren’t writing smarter SQL. They’re pausing before the SUM and asking what row they’re actually summing over.

Putting it together

Cardinality bugs are a specific kind of wrong: syntactically valid, semantically broken, and invisible to every automated check. Tests pass. Code reviews approve. Reports render. The numbers just happen to be wrong.

The defense is structural, not tactical. Understand the cardinality of each relationship before writing the join. Aggregate at the grain where the data lives. Use the schema to make cardinality explicit — foreign keys, composite primary keys on bridges, comments that document the shape. When diagnosing a wrong number, strip the aggregation and look at the raw rows; the multiplication is almost always visible as soon as the SUM is out of the way.

The worst thing about silent bugs is that they stay silent. A crash gets fixed; wrong numbers persist for quarters. Which means the habit of thinking about cardinality first — before writing the aggregation, not after someone flags the total — is one of the highest-leverage habits in working with relational data.

NULL in SQL: Three-Valued Logic and the Silent Bug Factory

Sat, 18 Apr 2026 00:00:00 +0000

TL;DR

NULL isn’t a value — it’s the absence of one, and SQL uses three-valued logic (TRUE / FALSE / UNKNOWN) to evaluate expressions involving it. Most operators return UNKNOWN when one of their operands is NULL, which means rows with NULLs silently drop out of !=, IN, and NOT IN filters and behave inconsistently across JOIN, GROUP BY, DISTINCT, and aggregate functions. The rules are consistent if you know them, and almost every one of them is a source of silently wrong query results when you don’t.

There’s a category of SQL bug that shows up in almost every mature codebase. Someone writes a filter like WHERE status != 'closed', expecting it to return every row that isn’t closed. Instead it returns fewer rows than the raw table contains — the rows where status is NULL silently dropped out. No error. No warning. The query is doing exactly what the SQL standard says it should, and the result is still wrong for what the author meant.

NULL handling is the single most common source of silently wrong query results in relational databases. The behavior is consistent if you know the rules, but the rules don’t match the intuition most programming languages build. In Java or Python, null != "closed" is true. In SQL, it’s UNKNOWN, and UNKNOWN rows get filtered out. That one difference produces most of the bugs.

NULL is not a value

Every introduction to SQL NULL starts here because it has to. NULL is the absence of a value — a marker that says “this column has no data.” It’s not zero, not empty string, not false. It doesn’t equal itself. It doesn’t not-equal itself either. Any comparison involving NULL returns a third logical state: UNKNOWN.

This is called three-valued logic (3VL), and SQL uses it consistently throughout the language. The three values are TRUE, FALSE, and UNKNOWN. Most operators propagate UNKNOWN: any arithmetic, string, or comparison operation with a NULL operand returns NULL (or UNKNOWN, in a boolean context).

SELECT NULL = NULL;         -- NULL (not TRUE)
SELECT NULL = 5;            -- NULL
SELECT NULL != 5;           -- NULL
SELECT NULL + 1;            -- NULL
SELECT 'hello' || NULL;       -- NULL in PostgreSQL (ANSI standard behavior)
SELECT CONCAT('hello', NULL); -- NULL in MySQL, 'hello' in PostgreSQL

The CONCAT difference is a good example of how engines diverge even within well-defined territory. MySQL’s CONCAT propagates NULL — any NULL argument makes the whole result NULL. PostgreSQL’s CONCAT function does the opposite: it silently skips NULL arguments and returns the concatenation of the non-NULL parts. (PostgreSQL’s || operator still propagates NULL, matching ANSI.) Two queries that look identical can return different results on different engines — and the difference only shows up when a NULL appears.

For NULL-skipping concatenation that behaves the same on both engines, use CONCAT_WS (concat with separator). Both MySQL and PostgreSQL skip NULL arguments with it:

1
2

SELECT CONCAT_WS(' ', first_name, middle_name, last_name);
-- "Alice Smith" even if middle_name IS NULL, on both engines.

One MySQL-specific gotcha: if the separator itself is NULL, the whole result is NULL. The separator is the one argument CONCAT_WS still propagates NULL from. As long as the separator is a literal string, the function is a reliable NULL-safe concat across engines.

IS NULL, not = NULL

The only way to test for NULL is with IS NULL or IS NOT NULL. WHERE col = NULL always returns zero rows, because col = NULL evaluates to NULL — which is not TRUE, so the row is filtered out. This is one of those mistakes that every SQL engineer makes exactly once.

WHERE clauses filter out UNKNOWN

The rule that drives most NULL bugs: WHERE only keeps rows where the condition evaluates to TRUE. UNKNOWN rows are filtered out, same as FALSE rows.

1
2

-- "Users not on the sales team"
SELECT * FROM users WHERE team_id != 3;

If team_id is NULL for unassigned users — a completely normal state — those rows are silently dropped. The expression NULL != 3 evaluates to UNKNOWN, and UNKNOWN is not TRUE, so the row doesn’t survive the filter.

The mental model most developers carry from application code — “anything that isn’t team 3 is included” — is wrong in SQL. To get that behavior, you have to spell it out:

`1`	`SELECT * FROM users WHERE team_id != 3 OR team_id IS NULL;`

This is one of the most common sources of “the numbers don’t match” bugs. A report that’s supposed to count “everyone outside the sales team” quietly excludes every unassigned user, and the total looks plausible because unassigned users aren’t visible in the team-level breakdown either. The discrepancy only surfaces when someone reconciles against a direct row count.

NOT IN is a trap

NOT IN with a nullable subquery is the classic silent-failure NULL bug. The trap is specifically that the subquery has to return a column that can contain NULL — which rules out primary keys but is extremely common for foreign keys, self-references, and any column that’s optional by design.

1
2
3

-- "Find users who aren't anybody's manager."
SELECT * FROM users
WHERE id NOT IN (SELECT manager_id FROM users);

The subquery returns every manager_id in the table — including NULL for users who don’t have a manager (the CEO, top-level roles, anyone unassigned). The moment the subquery contains a single NULL, the outer query returns zero rows.

The reason is how NOT IN expands. x NOT IN (a, b, c) is equivalent to x != a AND x != b AND x != c. If any of a, b, c is NULL, that comparison returns UNKNOWN, and AND with UNKNOWN can only ever be FALSE or UNKNOWN. The row never passes the filter.

Safer alternatives:

-- Use NOT EXISTS — handles NULLs correctly
SELECT * FROM users u
WHERE NOT EXISTS (
  SELECT 1 FROM users m WHERE m.manager_id = u.id
);

-- Or filter NULLs out of the subquery explicitly
SELECT * FROM users
WHERE id NOT IN (
  SELECT manager_id FROM users WHERE manager_id IS NOT NULL
);

NOT EXISTS is the better habit — it’s correct regardless of NULL presence, and the query planner handles it at least as well as NOT IN on any modern engine. Treating NOT IN as “suspicious until proven NULL-free” saves a category of bug that’s almost impossible to catch in review.

COUNT and NULL: skipped, not zero

The single most important thing to know about aggregates and NULL: NULL is not treated as zero. It’s skipped entirely. Nothing about NULL gets coerced or counted — it’s as if the row weren’t there for the purposes of the aggregate.

COUNT makes this visible because it has two forms that behave differently:

COUNT(*) counts rows, regardless of their contents. NULLs in the row don’t matter.
COUNT(col) counts non-NULL values of col. A row where col IS NULL is skipped.
COUNT(DISTINCT col) counts distinct non-NULL values. NULL is not treated as a distinct value; it’s excluded.

SELECT
  COUNT(*) AS total_rows,
  COUNT(email) AS rows_with_email,
  COUNT(DISTINCT email) AS distinct_emails
FROM users;

On a table of 1,000 users where 200 have NULL emails:

COUNT(*) returns 1000 (all rows)
COUNT(email) returns 800 (NULLs skipped)
COUNT(DISTINCT email) returns ≤ 800 (distinct non-NULL emails only)

This shows up in reports all the time. “How many users signed up this month?” gets answered with COUNT(signup_source) and comes up short because the column was added later and older rows have NULL. The row is there. COUNT(*) would see it. COUNT(signup_source) doesn’t.

The rule: use COUNT(*) when you want rows, COUNT(col) when you specifically want “rows with that column populated.”

SUM, AVG, MIN, MAX: also skip NULL

The same rule holds for every aggregate. NULL is not contributed to the sum, not counted in the denominator for the average, not considered for min or max.

`1`	`SELECT SUM(rating), AVG(rating) FROM reviews;`

If half the rows have NULL rating:

SUM(rating) is the sum of the non-NULL half. NULLs don’t contribute 0 — they contribute nothing.
AVG(rating) is the sum of the non-NULL half divided by the count of non-NULL rows, not the total row count.

The AVG behavior is the most common source of surprise. If 10,000 rows have rating = 5 and 10,000 have rating = NULL, AVG(rating) is 5.0, not 2.5. The NULL rows don’t pull the average down toward zero — they’re not in the denominator at all.

If you want NULL-as-zero behavior, you have to opt in:

1
2
3

SELECT AVG(COALESCE(rating, 0)) FROM reviews;
-- Now NULLs become 0 and land in both the sum and the denominator.
-- Returns 2.5 in the example above.

SUM of all NULLs is NULL, not zero

SUM(col) over a set where every value is NULL returns NULL, not 0. A SUM that feeds into arithmetic downstream (total + tax, for example) can propagate NULL through the rest of the expression, often somewhere the query author wasn’t expecting. COALESCE(SUM(col), 0) is the idiomatic fix — make the fallback explicit at the aggregate.

The framing that keeps this straight: NULL is not a value, so aggregates have nothing to aggregate. It’s not “zero,” it’s “absent.” If you want “absent means zero,” that’s a COALESCE decision the query author makes — the engine won’t make it for you.

GROUP BY and DISTINCT treat NULLs as equal

Here’s where the rules get inconsistent in a way that genuinely surprises people: GROUP BY and DISTINCT treat all NULLs as the same group, even though NULL = NULL returns UNKNOWN.

-- All rows where team_id is NULL land in one group, as if they were equal.
SELECT team_id, COUNT(*) FROM users GROUP BY team_id;
-- team_id | count
--   NULL  |  200
--     1   |  500
--     2   |  300

-- DISTINCT collapses all NULLs into one row.
SELECT DISTINCT team_id FROM users;
-- NULL
-- 1
-- 2

This is a deliberate exception carved out by the SQL standard — GROUP BY and DISTINCT use a “NULL-safe” equality for grouping purposes, because the alternative (one group per NULL row) would be useless. But it means the behavior is internally inconsistent: WHERE a = b says NULLs aren’t equal, GROUP BY a says they are.

The practical implication: COUNT(DISTINCT col) excludes NULL entirely (consistent with COUNT(col)), while GROUP BY col produces a single row for all NULLs. Two different “null-handling” behaviors under the same umbrella of “treats NULLs as equal for grouping.” Queries that rely on either for correctness should be written with the awareness that the two operations don’t agree.

NULL-safe comparison operators

Both MySQL and PostgreSQL offer operators that treat NULL as equal to NULL, mirroring the GROUP BY behavior for regular comparisons.

-- MySQL
SELECT * FROM users WHERE email <=> NULL;
-- Matches rows where email IS NULL — <=> is the null-safe equal operator.

-- PostgreSQL (ANSI SQL)
SELECT * FROM users WHERE email IS NOT DISTINCT FROM NULL;
-- Same idea. Treats NULLs as equal to each other.

These are useful when joining or filtering on columns that may contain NULL on both sides and you want NULLs to match:

-- Standard equality misses NULL-to-NULL matches
SELECT * FROM a JOIN b ON a.col = b.col;

-- IS NOT DISTINCT FROM treats NULLs as matching
SELECT * FROM a JOIN b ON a.col IS NOT DISTINCT FROM b.col;

Neither is used often in practice. The habit most teams settle on is “don’t let NULL be meaningful in join columns” — either constrain the columns NOT NULL or filter NULLs out before joining. The operators are there for the cases where those aren’t options.

ORDER BY: NULL placement varies by engine

When sorting, NULL has to go somewhere. The SQL standard leaves the default placement implementation-defined, and engines disagree.

PostgreSQL: NULLs sort last for ASC and first for DESC by default.
MySQL: NULLs sort first for ASC and last for DESC by default.
Oracle and SQL Server: match PostgreSQL’s behavior (NULLs last for ASC).

The fix is to be explicit:

1
2

SELECT * FROM events ORDER BY event_time ASC NULLS LAST;
SELECT * FROM events ORDER BY event_time DESC NULLS LAST;

NULLS FIRST / NULLS LAST is ANSI standard and supported by PostgreSQL, Oracle, and SQL Server. MySQL doesn’t support the NULLS FIRST/LAST syntax directly — you fake it with a computed column:

1
2
3

-- MySQL idiom for "NULLS LAST" on an ASC sort
SELECT * FROM events ORDER BY event_time IS NULL, event_time ASC;
-- event_time IS NULL returns 0 for non-nulls, 1 for nulls; 0 sorts first.

Teams that run the same reports against different engines (especially during a migration or in a polyglot analytics stack) hit this one hard. A top-10 leaderboard quietly reorders when the ORDER BY engine changes underneath it.

JOINs don’t match on NULL

A standard equi-join a.col = b.col doesn’t match rows where either side is NULL. This is consistent with the three-valued logic rule — NULL = NULL is UNKNOWN, so the join predicate fails.

-- Users can have no manager (manager_id IS NULL).
-- This join drops any user with no manager.
SELECT u.name, m.name AS manager_name
FROM users u
JOIN managers m ON u.manager_id = m.id;

If the intent is “every user, with manager info if present,” use a LEFT JOIN. If the intent is “users where manager_id matches some manager row,” the INNER JOIN is correct but it’s worth naming the exclusion: users with NULL manager_id are gone, on purpose.

For joins that should treat NULLs as matching (both sides have NULL, and that means “same”), use the null-safe operator:

1
2

SELECT *
FROM a JOIN b ON a.external_ref IS NOT DISTINCT FROM b.external_ref;

This is rare but legitimate — e.g., matching optional identifiers where “both unspecified” should be treated as a match. Most of the time, the correct answer is to make the column NOT NULL and use a sentinel if needed (and then deal with the sentinel’s own problems, covered below).

Foreign keys are nullable by default

A foreign key column is nullable unless declared NOT NULL. A nullable FK means “the reference is optional” — users may or may not have a manager, orders may or may not be linked to a promotion. This is often the correct intent, but it’s frequently unintentional.

-- manager_id is nullable by default. This is intentional if users can be unmanaged.
CREATE TABLE users (
    id BIGINT PRIMARY KEY,
    name TEXT NOT NULL,
    manager_id BIGINT REFERENCES managers(id)
);

Review migration files with this in mind. A column that should always be populated but was added as nullable will accept NULLs forever. Retrofitting NOT NULL later requires backfilling or cleaning up existing NULL rows — easy when the table is small, painful at scale. (Foreign Keys Are Not Optional covers the broader picture of FK enforcement and why application-level validation is an incomplete substitute.)

What NULL actually means is context-dependent

The SQL rules for NULL are unambiguous. What NULL means in a given column is not. NULL can mean:

“Unknown” — the data exists but we don’t have it. A user’s birthdate where the user declined to share.
“Not applicable” — the field doesn’t make sense for this row. spouse_name on a row for a single person.
“Ongoing” or “Not yet set” — the state isn’t finalized. end_date on an active subscription.
“Data entry error” — the column should have been populated but wasn’t.
“Legacy” — the column was added after the row was created and never backfilled.

The same column may mean different things in different rows, and the schema doesn’t tell you which is which. This is where schema comments earn their keep — documenting the semantics of NULL in each column, at the place the LLM (and the human) reads the DDL.

Sentinel values: the alternative, and its own problems

A common workaround: use a sentinel value instead of NULL. end_date = '9999-12-31' for “ongoing.” status = -1 for “unknown.” deleted_at = '1970-01-01' for “not deleted.”

Sentinels avoid the three-valued-logic rules at the cost of introducing their own bugs. A few to watch for:

Aggregates include sentinels. AVG(rating) over a column where “unknown” is stored as -1 skews the average toward negative. Sentinels break the “aggregates skip missing values” assumption that NULL provides for free.
Range queries break. WHERE end_date < NOW() returns every row with end_date = '9999-12-31' — no, it doesn’t, actually, that’s the point. But WHERE end_date > NOW() returns all the sentinel rows along with real future dates. Every filter has to explicitly exclude the sentinel.
Indexes skew. A column where 80% of the values are the sentinel has a low-selectivity index. The planner may skip the index entirely on queries that filter out the sentinel, because it doesn’t know that’s the intent.
Downstream consumers have to know. Every system that reads the data has to treat 9999-12-31 specially. Miss one consumer and wrong data shows up in a report.

The trade-off is real. NULL forces every query author to think about three-valued logic. Sentinels let queries use normal equality but require every author to know the sentinel. Neither is free; they move the cost around.

The pragmatic middle ground: use NULL for genuinely absent data (ongoing subscriptions, optional fields), use sentinels sparingly and document them, and declare NOT NULL everywhere you can enforce presence. A column that’s NOT NULL is the one case where the rules don’t matter, because NULL can’t get in.

Diagnosing a NULL bug

When a query returns fewer (or more, or none) of the rows it should, the fastest way to narrow it down to a NULL issue:

-- Are there NULLs in the columns referenced by the filter?
SELECT
  COUNT(*) AS total,
  COUNT(team_id) AS with_team,
  COUNT(*) - COUNT(team_id) AS no_team
FROM users;

If no_team is non-zero and the filter is team_id != X or team_id IN (...), the NULL rows are the likely culprit. Rewriting with explicit NULL handling (team_id != X OR team_id IS NULL, or NOT EXISTS, or COALESCE(team_id, -1) != X) will reveal whether NULLs were being silently excluded.

For NOT IN, inspect the subquery:

1
2

-- Does the NOT IN subquery contain NULL?
SELECT COUNT(*) FROM users WHERE manager_id IS NULL;

If the answer is non-zero, NOT IN is returning an empty set regardless of the outer query’s data.

The mental model

NULL handling is consistent once you internalize the rule set, and the rule set is smaller than it looks:

Any comparison involving NULL returns UNKNOWN. WHERE filters out UNKNOWN rows.
Aggregates skip NULLs. COUNT(*) doesn’t. COUNT(col) does.
GROUP BY, DISTINCT, and ORDER BY treat all NULLs as equivalent (with engine-specific sort placement).
NOT IN with a nullable subquery returns empty. Use NOT EXISTS.
Join predicates don’t match NULLs unless you use IS NOT DISTINCT FROM.

Past that, most NULL bugs are prevented by one habit: declare NOT NULL wherever the column should actually be populated. Every NOT NULL column is a column where none of these rules matter, because there’s nothing for them to misbehave on. The fewer nullable columns the schema has, the less of this there is to think about.

The columns where NULL genuinely carries meaning — optional references, ongoing states, data that may not exist — are the ones worth documenting. A schema comment that says “NULL means the subscription is still active” pulls the NULL semantics into the DDL itself, where it’s visible to every engineer, every tool, and every query author who wasn’t around when the decision was made.

NULL isn’t going away. But the bugs it causes are almost entirely the result of assuming it behaves like null in application code. It doesn’t. It behaves like the SQL standard says it does — quietly, consistently, and not at all the way most programming languages have trained people to expect.

Uniqueness and Selectivity: The Two Numbers That Drive Query Plans

Sat, 18 Apr 2026 00:00:00 +0000

TL;DR

Uniqueness (whether a column or column combination contains unique values) and selectivity (the ratio of distinct values to total rows) are two of the most load-bearing concepts in relational databases. Uniqueness governs correctness — what guarantees the database can make about “at most one row per X” — and selectivity governs performance, because the planner’s entire strategy depends on how many rows it expects each step to return. Both are largely invisible unless declared or queried directly, which is why schemas frequently lack the constraints that would keep queries correct and the statistics that would keep them fast.

A schema that looks clean on paper can hide two categories of problem that only surface under load. The first is uniqueness: a UNIQUE constraint that should exist and doesn’t, letting duplicates quietly accumulate until a query downstream multiplies rows or an UPSERT silently does the wrong thing. The second is selectivity: an index that looked reasonable at design time but the planner never uses, or a join order the planner picks based on stale statistics that no longer reflect the data.

These two ideas are tightly coupled. Uniqueness is the shape of the data — how many distinct rows per key. Selectivity is what the planner knows about that shape. Get them right and queries are correct and fast. Get them wrong and the symptoms show up as slow queries, duplicate rows, and aggregations that don’t tie out.

Uniqueness: what the database guarantees

A UNIQUE constraint is the only mechanism the database has to guarantee that no two rows share the same value (or combination of values) for a given column or column set. Without it, the application layer can promise uniqueness and still fail to deliver it — every direct write path (backfill script, migration, admin UI, new microservice) is a potential source of duplicates.

-- Column-level uniqueness
CREATE TABLE users (
    id BIGINT PRIMARY KEY,
    email VARCHAR(255) NOT NULL UNIQUE,
    created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

-- Composite uniqueness on a bridge table
CREATE TABLE user_roles (
    user_id BIGINT NOT NULL REFERENCES users(id),
    role_id BIGINT NOT NULL REFERENCES roles(id),
    PRIMARY KEY (user_id, role_id)
);

The composite primary key on user_roles is doing real work. Without it, an application bug that inserts the same (user_id, role_id) pair twice would silently create a duplicate — and any query joining through user_roles would multiply rows. This is precisely the row multiplication trap covered in Joins That Lie: the bridge table’s composite uniqueness is what prevents the cardinality explosion on the join.

The write-time cost of a unique constraint is negligible — a lookup against an index the database already maintains for the constraint. The diagnostic cost of a duplicate that shipped to production and propagated downstream is measured in days.

Partial and filtered unique indexes

PostgreSQL supports partial unique indexes, which enforce uniqueness only for rows matching a predicate. This is the right tool for one of the most common real-world cases: “email must be unique among active users.”

-- PostgreSQL: email is unique among non-soft-deleted users.
CREATE UNIQUE INDEX users_active_email_uniq
  ON users (email)
  WHERE deleted_at IS NULL;

Without the partial index, a deleted user would block re-registration with the same email. A plain UNIQUE (email) constraint forces a trade-off: either allow re-registration (and lose referential integrity by reusing emails) or block it (and frustrate users). The partial index lets both constraints coexist — uniqueness where it matters, flexibility where it doesn’t.

MySQL doesn’t support partial unique indexes directly. The workaround is a generated column that’s NULL for rows the uniqueness shouldn’t apply to, exploiting the fact that UNIQUE treats NULLs as distinct (in most engines — a quirk covered in NULL: Three-Valued Logic).

-- MySQL idiom: generated column that's NULL for deleted users.
ALTER TABLE users
  ADD COLUMN email_active VARCHAR(255)
  GENERATED ALWAYS AS (CASE WHEN deleted_at IS NULL THEN email END) VIRTUAL,
  ADD UNIQUE KEY users_active_email_uniq (email_active);

The pattern is awkward, but it works because MySQL’s UNIQUE allows multiple NULLs. The constraint effectively fires only for rows where the generated column is non-NULL, which is exactly the partial-index semantics.

UPSERT correctness depends on uniqueness

INSERT ... ON CONFLICT (PostgreSQL) and INSERT ... ON DUPLICATE KEY UPDATE (MySQL) only work if the conflict target corresponds to an actual unique constraint.

-- PostgreSQL
INSERT INTO users (email, name)
VALUES ('alice@example.com', 'Alice')
ON CONFLICT (email) DO UPDATE SET name = EXCLUDED.name;
-- Requires a UNIQUE constraint or unique index on email.
-- Without one, this errors out.

The engine doesn’t invent uniqueness — it matches the ON CONFLICT target against defined constraints. If the constraint doesn’t exist, the UPSERT fails (PostgreSQL) or does something surprising (MySQL’s ON DUPLICATE KEY UPDATE applies to any unique key, which can be different from the one the query author had in mind).

This is especially fragile with partial unique indexes. In PostgreSQL, ON CONFLICT (email) matches a plain unique index on email but does not match a partial unique index with a WHERE deleted_at IS NULL clause — you need ON CONFLICT (email) WHERE deleted_at IS NULL to target the partial index specifically.

-- Targets the partial unique index
INSERT INTO users (email, name)
VALUES ('alice@example.com', 'Alice')
ON CONFLICT (email) WHERE deleted_at IS NULL
DO UPDATE SET name = EXCLUDED.name;

The broader point: UPSERT semantics are tied to the exact shape of the unique constraints. Changing or removing a constraint can silently change the behavior of queries that reference it, often in ways that only surface under a specific sequence of writes.

Missing unique constraints cause silent duplicates

A constraint missing from the schema is a duplicate waiting to happen. The failure mode is predictable: a column that should be unique (username, external ID, composite bridge key) accumulates duplicates over months or years; some dashboards start showing inflated counts; when someone finally notices, adding the constraint requires a cleanup migration because the duplicates already exist.

-- This ALTER fails if duplicates already exist.
ALTER TABLE users ADD CONSTRAINT users_email_uniq UNIQUE (email);
-- ERROR: could not create unique index "users_email_uniq"
-- DETAIL: Key (email)=(alice@example.com) is duplicated.

The migration becomes a multi-step project: find the duplicates, decide which to keep, merge or delete the rest, update any downstream systems that depend on the duplicates, then add the constraint. It’s also a correctness audit — every system that touched this data while duplicates existed is suspect.

The fix, as always, is to add the constraint on day one and let the database reject duplicates at write time. Retrofitting is a second-class option.

Selectivity: what makes an index useful

Selectivity is the ratio of distinct values to total rows in a column. A column with two distinct values (a boolean flag) has selectivity close to zero. A column with unique values has selectivity of one. The planner uses this number — specifically, its estimate of this number — to decide whether to use an index, which index to use, and in what order to join tables.

The intuition: if a filter narrows the row count down to a small fraction of the table, an index lookup is cheap. If the filter matches most of the table, a sequential scan is cheaper because reading pages in order beats jumping around via the index.

-- High selectivity: index is useful.
SELECT * FROM orders WHERE email = 'alice@example.com';
-- One row out of millions. Index scan is 100x-1000x faster than a seq scan.

-- Low selectivity: index is often useless.
SELECT * FROM orders WHERE is_active = TRUE;
-- 99% of rows match. Seq scan wins.

The break-even point is engine- and hardware-dependent, but the rule of thumb is that an index starts to lose to a sequential scan once the filter matches somewhere between 5% and 20% of the table. Below that, the index wins. Above that, the planner will usually skip the index entirely — even if you wrote it specifically for that query.

The 'why isn't my index being used' question

Most of the time, the answer is: the column isn’t selective enough, or the planner thinks it isn’t. An index on a boolean is_deleted column is rarely useful on its own — the planner will scan. An index on (is_deleted, created_at) can be useful for queries filtering on both, because the composite lets the planner narrow more aggressively.

Composite index column ordering

For composite indexes, the order of columns matters — it’s a selectivity decision that depends on how the index will be used. The general rules:

Put equality filters before range filters. An index on (customer_id, created_at) is efficient for WHERE customer_id = 42 AND created_at > '2026-01-01'. The reverse — (created_at, customer_id) — forces the planner to scan a wide range of created_at values and then filter.
Put the more selective column first for equality-only filters. If both columns are filtered with equality, put the column with more distinct values first to maximize the narrowing on the first lookup.
Match the index order to the query’s filter order when possible. A query that always filters by (a, b, c) benefits from an index in that exact order. Reordering the index means the leading columns aren’t being filtered, which often defeats the index entirely.

These aren’t strict rules — they interact with cardinality estimates, sort order requirements, and covering index considerations. But they’re the right starting point. The wrong starting point is “put all the filter columns in some order” without thinking about how the index will actually be used.

Planner statistics drive everything

The planner doesn’t read the data — it reads statistics about the data and makes decisions from those. PostgreSQL tracks per-column statistics in pg_stats:

SELECT
  schemaname, tablename, attname,
  n_distinct,       -- estimated distinct values
  null_frac,        -- fraction of rows that are NULL
  most_common_vals, -- top values by frequency
  most_common_freqs,
  histogram_bounds  -- distribution of non-common values
FROM pg_stats
WHERE tablename = 'orders' AND attname = 'customer_id';

MySQL surfaces similar information through information_schema.STATISTICS and information_schema.INNODB_TABLESTATS, though less comprehensively than PostgreSQL’s stats.

These statistics are gathered by ANALYZE (PostgreSQL) or maintained automatically by InnoDB (MySQL). They go stale between runs. A table that was 10 million rows when last analyzed and is now 200 million rows has planner statistics that no longer reflect reality — and the planner is making decisions based on the outdated picture.

The usual symptom of stale statistics: a query that was fast yesterday is slow today, even though nothing changed. The planner’s row estimate for some step is off by orders of magnitude, so it picks a nested loop where it should have picked a hash join, or scans a table it should have seeked into.

-- PostgreSQL: refresh statistics on a specific table.
ANALYZE orders;

-- See what the planner thinks vs. what it got
EXPLAIN ANALYZE SELECT * FROM orders WHERE customer_id = 42;
-- Look at "rows=X" estimate vs. "actual rows=Y".
-- A large ratio (10x+) is a sign stats are off or correlation is at play.

EXPLAIN ANALYZE comparing estimated to actual row counts is one of the highest-leverage diagnostic tools in SQL. When the ratio is out of whack, the plan is wrong for the data, and the fix is usually statistical (refresh stats, add extended statistics, increase statistics target) rather than rewriting the query.

Cardinality estimation errors

The single most common cause of bad query plans in production is a bad row-count estimate. Two flavors:

Underestimates — the planner thinks a step will return 10 rows when it actually returns 10 million. It picks a nested loop (good for small outer side), which then runs 10 million times. What should have been a 50ms query takes 50 minutes.
Overestimates — the planner thinks a step will return 10 million rows when it actually returns 10. It picks a hash join (good for large sets) that allocates buckets for 10 million entries and spills to disk. What should have been a 5ms query takes 5 seconds.

Both errors are failures of the statistics, not the query. And both are especially hard to diagnose because the query text looks identical in the fast and slow cases — only the underlying data (or the statistics about it) has changed.

Correlated columns break the independence assumption

The planner assumes columns are statistically independent when estimating the selectivity of a compound predicate: WHERE country = 'US' AND state = 'CA' is estimated as (fraction where country=US) × (fraction where state=CA). In reality, state = CA implies country = US — the columns are perfectly correlated, and the compound filter matches many more rows than the independence assumption predicts.

-- PostgreSQL: tell the planner about the correlation.
CREATE STATISTICS country_state_corr (dependencies, ndistinct)
  ON country, state FROM addresses;
ANALYZE addresses;

Extended statistics (PostgreSQL 10+) let you declare that certain column combinations are correlated, giving the planner better estimates for compound filters. They’re not free — statistics take space and have to be maintained — but for a few specific high-traffic queries, they can turn a disastrous plan into a good one.

MySQL doesn’t have an equivalent facility. Correlated-column estimation errors are harder to fix at the planner level; the workaround is typically to restructure the query or add indexes that capture the correlated access pattern.

UNIQUE constraints help the planner

A UNIQUE constraint isn’t just a correctness tool — it’s also a signal to the planner. Knowing a column is unique lets the optimizer prove that a join on that column produces at most one match per input row, which enables specific plan optimizations:

Deduplication elimination. If joining A to B on a unique B column, the planner knows the result set can’t have duplicates and can skip any deduplication step.
Join order reordering. Unique columns are guaranteed 1:1 or 1:0, which constrains how joins can be reordered without changing semantics. More options usually means better plans.
Index-only scan eligibility. Unique constraints are backed by unique indexes, which are often the right target for index-only scans.

The practical implication: schemas that declare their unique constraints let the planner do more. Schemas that rely on application-level uniqueness get correct results but may get worse plans, because the planner can’t trust assumptions it can’t see.

Diagnosing uniqueness and selectivity issues

Both problems show up in predictable ways. A short playbook:

“Why is this query slow?”

First, run EXPLAIN ANALYZE and compare estimated to actual row counts.

`1`	`EXPLAIN ANALYZE SELECT * FROM orders WHERE customer_id = 42 AND status = 'shipped';`

If estimated and actual are within a factor of 2 or 3, the plan is reasonable and the slowness is elsewhere (missing index, wrong index, large result set). If they differ by 10x or more, the statistics are off or the columns are correlated. Refresh stats (ANALYZE), check for correlation, and re-examine the plan.

“Why isn’t my index being used?”

Check the column’s selectivity:

-- PostgreSQL
SELECT
  COUNT(*) AS total,
  COUNT(DISTINCT status) AS distinct_status,
  COUNT(DISTINCT status)::float / COUNT(*) AS selectivity
FROM orders;

If selectivity is below ~5%, the planner may be skipping the index in favor of a sequential scan — and that’s often the right choice. If selectivity is high and the index still isn’t being used, check for functions in the WHERE clause (non-SARGable predicates), type mismatches (implicit casts that prevent index use), or stale statistics.

“Why do I have duplicates?”

Find them:

SELECT email, COUNT(*) FROM users
GROUP BY email
HAVING COUNT(*) > 1
ORDER BY COUNT(*) DESC;

If duplicates exist, the schema is missing a unique constraint that should have been there. Adding the constraint requires cleaning up first, which is a separate migration. Going forward, every write path has to be considered — an application change that prevents new duplicates doesn’t help the duplicates that already slipped through other paths.

“Why is my UPSERT doing the wrong thing?”

Check which unique constraint the ON CONFLICT (or ON DUPLICATE KEY) target is matching against. In MySQL, ON DUPLICATE KEY UPDATE matches any unique key that conflicts — if a row has a unique constraint on both email and external_id, an insert that conflicts on either triggers the update. In PostgreSQL, ON CONFLICT (col) requires a specific named column or constraint and fails if one doesn’t match — which is annoying when writing the query but means the behavior is explicit.

The mental model

Uniqueness and selectivity map to two questions the planner and the engineer both need answered for every table and every query:

How many rows per key? (Uniqueness: 1:1, 1:N, N:M.) This determines whether joins multiply, whether UPSERTs do the right thing, and whether aggregations can be trusted.
How many distinct values relative to total? (Selectivity.) This determines whether indexes are useful and which join order the planner will pick.

Both questions are answerable from the schema if the constraints are declared and the statistics are current. Both questions become hand-waving if they’re not. Schemas that leave uniqueness implicit — enforced in application code, documented in a wiki, “known” by the senior engineers — are schemas where the database can’t help with correctness or performance. The constraints are the contract between the schema and everything that touches it.

The habit that pays off: declare every unique constraint that should exist, including the composite ones on bridge tables; keep planner statistics fresh on busy tables; use EXPLAIN ANALYZE to catch cardinality estimation errors early; and add extended statistics for the handful of correlated-column cases that show up in hot paths. None of this is glamorous. All of it compounds.

The cost of getting it wrong isn’t usually a crash. It’s a query that’s 100x slower than it should be, a report that’s silently inflated, or an UPSERT that corrupts data one weekend when traffic patterns shift. These are the failure modes that make databases feel unpredictable — and they’re almost all traceable back to the same two numbers.

Sql on EXPLAIN ANALYZE

Joins That Lie: The Cardinality Problem

Cardinality, briefly

The row multiplication problem

The bridge table trap

A related trap: date filters across joined tables

How to diagnose it

How to solve it

Aggregate at the correct grain, then join

Use DISTINCT inside the aggregate — with caution

Window functions for “per parent” aggregates

LATERAL joins and correlated subqueries

Schema-level defenses

The mental model

Putting it together

NULL in SQL: Three-Valued Logic and the Silent Bug Factory

NULL is not a value

WHERE clauses filter out UNKNOWN

NOT IN is a trap

COUNT and NULL: skipped, not zero

SUM, AVG, MIN, MAX: also skip NULL

GROUP BY and DISTINCT treat NULLs as equal

NULL-safe comparison operators

ORDER BY: NULL placement varies by engine

JOINs don’t match on NULL

Foreign keys are nullable by default

What NULL actually means is context-dependent

Sentinel values: the alternative, and its own problems

Diagnosing a NULL bug

The mental model

Uniqueness and Selectivity: The Two Numbers That Drive Query Plans

Uniqueness: what the database guarantees

Partial and filtered unique indexes

UPSERT correctness depends on uniqueness

Missing unique constraints cause silent duplicates

Selectivity: what makes an index useful

Composite index column ordering

Planner statistics drive everything

Cardinality estimation errors

Correlated columns break the independence assumption

UNIQUE constraints help the planner

Diagnosing uniqueness and selectivity issues

“Why is this query slow?”

“Why isn’t my index being used?”

“Why do I have duplicates?”

“Why is my UPSERT doing the wrong thing?”

The mental model

Use `DISTINCT` inside the aggregate — with caution