Atharva Pandey Gopherine

There is a class of production problems where exact answers are either impossible or not worth the cost. You want to know approximately how many unique visitors hit your site today. You want to sample 1% of requests for tracing without reading every request into memory first. You want to check if a username is already taken without querying the database on every keystroke.

Randomized algorithms provide exact answers with known probability bounds, or approximate answers with bounded error, using a fraction of the memory or time that exact computation would require. I was skeptical of “probabilistic” algorithms for a long time — it seemed like trading correctness for efficiency. Then I learned what the error bounds actually are. A HyperLogLog cardinality estimate with 1.5% error using 12KB of memory is a better engineering choice than a perfect count using 100MB, for the vast majority of use cases.

I once inherited a codebase where every HTTP handler had raw SQL queries inline — sqlx::query! calls scattered through 80+ route handlers. Changing a table name meant grep-and-replace across the entire project. Adding a cache layer meant touching every handler. Testing a handler meant spinning up a real database. It worked, technically, but nobody wanted to touch it.

The repository pattern fixes this. It puts a wall between your business logic and your database, and that wall pays for itself fast.

Most CLI tools work perfectly on the happy path and fail silently on the unhappy one. The user presses Ctrl-C, the OS sends SIGINT, and the process dies immediately — leaving a temp file half-written, a database connection open, or a progress bar frozen mid-operation. The problem is invisible because most of the time nobody looks at what gets left behind. Until a deployment script depends on that temp file being complete, or a database hits its connection limit, or a batch job loses six hours of progress because the container was terminated between checkpoints.

A payment service I worked on had a subtle bug: it deducted money from the user’s wallet, then tried to create an order record. If the order insert failed — constraint violation, timeout, anything — the money was already gone. The user’s balance was decremented but they had no order. We called these “ghost charges” internally, and customers called them something less polite.

The fix was embarrassingly simple: wrap both operations in a transaction.

Shipping to production is where the real education begins. Your local dev environment is a controlled fantasy — one instance, no load balancer, fast database on localhost, unlimited memory. Production is a hostile environment where your service gets killed mid-request, runs out of memory at 3am, and needs to tell you what went wrong without you SSH-ing into a container. This lesson is about surviving out there.

Dockerfile: The Multi-Stage Build

Rust binaries are statically linked (or nearly so). A compiled Rust service can run in a scratch or distroless container with no runtime dependencies. This means tiny images — often under 20MB.

I have shipped offset-based pagination more times than I would like to admit. It looks clean, it is trivially easy to implement, and it works perfectly — until the table reaches about 100,000 rows. Then it starts to slow down, and by a million rows it is actively harmful. I had a support dashboard that froze every time someone clicked to page 47. That was the moment I stopped using OFFSET.

A teammate once ran ALTER TABLE orders DROP COLUMN status on the production database because he’d tested it locally and “it worked fine.” What he didn’t realize was that three other services depended on that column, and they all started throwing errors simultaneously. We spent the evening restoring from a backup.

Schema migrations exist to prevent exactly this — they’re version control for your database.

The Problem

Your database schema isn’t static. Features get added, requirements change, data models evolve. You need a way to:

The advice “start concrete, extract when proven” is easy to say and surprisingly hard to follow when you’re in the middle of writing the third version of the same function. The temptation to generalize early is real. I’ve felt it. Most engineers who care about clean code have felt it.

But the cost of premature abstraction is higher than the cost of temporary duplication. A duplicated function can be removed with a search-and-replace. A bad abstraction gets built around, extended in wrong directions, and defended by sunk-cost thinking. This lesson is about how to do the refactoring correctly — waiting until the pattern is proven, then extracting it cleanly.

I worked on a codebase that had 600 unit tests with mocked HTTP clients, mocked databases, mocked everything. All 600 passed. The application didn’t work. The mocks were wrong — they returned data in a format the real database never produced. Those 600 tests gave the team confidence to ship broken code. Integration tests that hit real infrastructure are harder to write but they tell you whether your application actually works.

I once watched a Rust service fall over under modest load — maybe 200 concurrent requests — because every request opened a new Postgres connection, used it for one query, and dropped it. The database was spending more time on TLS handshakes and connection setup than on actual queries. CPU was fine, memory was fine, but pg_stat_activity showed 200+ connections churning constantly. The fix took ten lines of code: add a connection pool.