Khalil's Website

About

I’m a software engineer and entrepreneur focused on building AI-powered products, scalable systems, and practical technology solutions. I enjoy turning complex ideas into reliable products that create real-world value.

Email: khalil.sarwari [at] berkeley [dot] edu

Bug Tales

Short stories about long debugging sessions.

Who Watches the Watchman?

I was using set_detect_anomaly(True) to debug a memory leak in my GPU training pipeline. I was able to fix most of it, but there was a small persistent leak that was not going away. Turns out the detect_anomaly itself was causing the memory leak.

Lost in Transmission

I set up a custom data capture system with camera+lidar calibration for my grad project. It was working fine indoors, but as soon as I deployed it on my car, it stopped working. Turns out using a long USB cable between the computer in the back of the car and the camera in the front was causing transmission errors.

The Leaning Tower of PCIe

I was using a custom machine at home with a GPU for deep learning experiments. It worked fine for months, but then after some time started giving errors with increasing frequency (GPU fell off the bus, etc.). Upon closer inspection, turned out over time the card started sagging, the GPU fan scraped the bottom of the case, and it would stop working.

Works on My Instance

Built a platform on top of Next.js, deployed with a custom AWS pipeline instead of Vercel. It would work fine on staging. On production, it would also work fine sometimes; other times the page navigation would fail — only started happening after switching production instances to arm64. Turns out, on arm64, Next.js (+ Sentry) produces inconsistent hashes, used in the filepaths for the generated page chunks. This only was a problem in production because production ran on more than one instance, whereas staging ran on a single instance. Solved by generating the build on one instance and copying it over, instead of building independently per instance for each deployment.

Around the World in 100ms

I was serving video content from an on-prem machine to users around the globe using a public static IP, and noticed high latency even though my NIC and CPU/memory/IO were not saturated. Turns out the problem was my ISP’s peering with other networks — essentially it was using public routes around the world which were busy/rate-limited. By putting a CDN in front, request times improved even though the packets were traveling the same distance — this time the bulk of the route was private/internal to the CDN, which was a higher speed route.

Teaching

I enjoy teaching — during my time at UC Berkeley I was involved in the instruction of the introductory computer science, linear algebra, discrete math, and probability courses.