Khalil's Website

About

I’m a software engineer and entrepreneur focused on building AI-powered products, scalable systems, and practical technology solutions. I enjoy computer science for its breadth, and for the ability to turn complex ideas into reliable products in the real world.

I also enjoy teaching — during my time at UC Berkeley I was involved in the instruction of the introductory computer science, linear algebra, discrete math, and probability courses.

Email: khalil.sarwari [at] berkeley [dot] edu

Bug Tales

Short stories about long debugging sessions.

Who Watches the Watchman?

I was using set_detect_anomaly(True) to hunt a memory leak in my GPU training pipeline. I fixed most of it, but one small leak wouldn’t go away no matter what I did. It was detect_anomaly itself leaking memory. The tool I was using to find the leak was the leak.

Lost in Transmission

For my grad project I built a custom data capture system with camera+lidar calibration. Indoors it worked perfectly; the moment I mounted it on my car, it stopped. The culprit was the USB cable running from the computer in the back of the car to the camera up front, just long enough for transmission errors to creep in.

The Leaning Tower of PCIe

My home machine for deep learning experiments ran fine for months, then began throwing errors with increasing frequency — the GPU kept “falling off the bus.” The error was more literal than advertised: the card had slowly sagged under its own weight until the GPU fan scraped the bottom of the case, and the whole thing stopped working.

Works on My Instance

I built a platform on top of Next.js, deployed through a custom AWS pipeline rather than Vercel. Staging was flawless; production failed intermittently, but only after the instances moved to arm64. On arm64, Next.js (+ Sentry) produces inconsistent build hashes, which end up in the filepaths for the generated page chunks. Staging ran a single instance, so it could never disagree with itself; production ran several, each serving chunks from a slightly different build. The fix: build once, copy everywhere.

Around the World in 100ms

I was serving video from an on-prem machine to users around the globe over a public static IP, and latency was high even though the NIC, CPU, memory, IO (even the ISP uplink itself) were nowhere near saturated. The bottleneck wasn’t my machine; it was my ISP’s peering: packets were riding busy, rate-limited public routes around the world. A CDN in front fixed it even though the packets traveled the same distance; the bulk of the route was now private to the CDN’s backbone, a far faster path.