Practical System Design: The Quiet Elegance of Boring Architecture
Junior engineers often assume that impressive system design requires a complex web of microservices, event streaming through Apache Kafka, CQRS patterns, and distributed consensus mechanisms running across every cluster.
Read More
How I Keep Myself Alive Using Golang: Automated Incident Management for Type 1 Diabetes
How many grams of carbohydrates are in a pint of beer or a seemingly healthy salad? For most people, the answer is “who cares,” unless they are following a strict diet. But for approximately 8 million people living with Type 1 Diabetes (T1D) worldwide, this is a continuous, daily life-or-death question.
Read More
Lies We Tell Ourselves to Keep Using Golang
In the backend engineering community, Golang (Go) is frequently praised for its radical simplicity. From microservices at Google, Uber, and Grab to core infrastructure projects like Docker, Kubernetes, and Terraform, Go seems to be everywhere. Engineers routinely swap familiar praises: “Go is dead simple to learn,” “Concurrency in Go is practically free thanks to Goroutines,” and “The Go toolchain compiles instantly into a single self-contained binary.”
Read More
Golang disables Nagle's Algorithm by default: A performance trick on unstable networks
Hello everyone, today I want to share with you a funny and sad story that I and many of my fellow system engineers have experienced.
Read More
Graceful Shutdown: Benefits and Reasons to Have It
There was a time when our team deployed a new version at 11 pm. After running kubectl rollout restart, 30 seconds later, PagerDuty alerted: 200 502 errors in 5 seconds. Customers were placing orders when they encountered a timeout. We quickly rolled back, then checked the logs - it turned out that the old pod was terminated with SIGTERM, the HTTP server shut down immediately, and 50 requests being processed were cut off entirely.
Read More
Goroutine management: errgroup, leak-proof, backpressure
Last month, one of our team’s services suddenly slowed down like a turtle. The p50 latency jumped from 50ms to 3 seconds. Upon checking Grafana, there were 120,000 goroutines running - normally, there are only 200. Someone had just pushed code and forgotten to call defer cancel() in a batch processing loop.
Read More
When to use cache and when not to
There is a production bug that I still remember vividly: user A cancels an order, but the app still displays “delivering” for the next 10 minutes. Support receives 30 tickets in one morning. The reason: cache TTL is 10 minutes, but no one invalidates it when the order status changes.
Read More
API Filtering: retrieving data like a coffee connoisseur
In my first year of working, I once wrote an endpoint GET /api/menus that returned… the entire menu. 200 items every time it was called. The JSON was 1.2MB heavy. The frontend only needed the name and price of 10 active dishes. I remember the first thing my lead said: “You’re sending the entire warehouse to someone who just needs to view the menu, aren’t you?”
Read More
Context in Go: passing data, deadline, cancel - use correctly or die of performance
When I first coded in Go, I encountered a strange production bug: after running for 3-4 days, the service suddenly consumed 8GB of RAM and caused an OOM error. I spent the entire Friday afternoon debugging and finally discovered the issue: a goroutine never ended because I forgot to pass the context to the gRPC call. Each request leaked a goroutine, and after 100K requests, the server was overwhelmed.
Read More