Postgres and MySQL DR on EC2: AWS Automates the Snapshot, Not the Restore

Data Lifecycle Manager snapshots every volume on a database host at the same instant, runs a freeze script first, copies the set to another region, and expires it on schedule. The MySQL freeze script AWS publishes takes its global read lock in a client that exits as soon as the statement returns. The snapshots restore anyway. What does the restore need that the service never touches?

Postgres, Kafka, Elasticsearch on EC2: Keep the Data on EBS, Not the Instance

Zalando made Kafka cluster upgrades 42 times faster by moving disks instead of data. Elasticsearch documents that a copy of its data directories is not a backup and cannot be restored. Every EBS volume attached to an r7i.8xlarge shares one 40,000 IOPS budget. What does a storage system gain when its data stops living on the instance, and what does it pay?

Talk Is Cheap: Why Nobody Argued With Your Migration Plan

A talented engineer proposes moving fifty MySQL clusters to TiDB. The plan is well written, the destination is defensible, and the motivation is real, because patching the fleet genuinely is miserable. In the meeting nobody objects. Nobody asks a hard question either, and nothing gets funded. The plan was not refuted, it was discounted. What would it have needed to survive that room?

A Billion Row Versions From One Uncommitted Read Stalled a MySQL Replica

A read replica is three days behind and falling further. The source is serving 15,000 queries a second, exactly as it has every month for a year. Restarting the replica clears the lag, and within a day it is behind again. The replica's data directory has grown to four times the size of the source's. One connection is open against it, doing nothing, started by a server that no longer exists. What is that connection still holding?

A Query on a 3-Billion-Row Table Took 100 Minutes: One Added Line Made It 40ms

A report against a 3.2-billion-row table runs for 1 hour 43 minutes and returns 1,823 rows for one retail chain's 603 stores. Every join hits an index. The chain_id column that would make the whole thing trivial is sitting right there on the table. One added line takes it to 40 milliseconds with no schema change. Which line, and why couldn't the optimizer find it on its own?

Collation Drift: The Silent Index Killer

Two customer_ref VARCHARs, same name, same length, one on utf8mb4_general_ci and one on utf8mb4_0900_ai_ci because they were born three years apart on either side of a version upgrade. The join still runs. It just quietly stopped using the index.

Self-Healing Databases with a Health Check and a Watch Script

A disk fills with binlogs while the on-call sleeps. The fix is a single PURGE statement, and picking the wrong target file leaves a lagging replica unable to ever reconnect. A health check wired to a handler can pick the safe target in the moment, with every replica's position in hand, and file the ticket that keeps the burst from becoming invisible.