Goroutine management: errgroup, leak-proof, backpressure
Last month, one of our team’s services suddenly slowed down like a turtle. The p50 latency jumped from 50ms to 3 seconds. Upon checking Grafana, there were 120,000 goroutines running - normally, there are only 200. Someone had just pushed code and forgotten to call defer cancel() in a batch processing loop.
Read More
Context in Go: passing data, deadline, cancel - use correctly or die of performance
When I first coded in Go, I encountered a strange production bug: after running for 3-4 days, the service suddenly consumed 8GB of RAM and caused an OOM error. I spent the entire Friday afternoon debugging and finally discovered the issue: a goroutine never ended because I forgot to pass the context to the gRPC call. Each request leaked a goroutine, and after 100K requests, the server was overwhelmed.
Read More