Performance

After setting up CUDA on my other laptop, I moved to a different(older) machine that doesn’t have an NVIDIA GPU. This one is an everyday laptop with integrated Intel graphics, but that doesn’t mean we have to settle for slow CPU-only performance. On this machine, I switched to the Vulkan backend for llama.cpp and the results were even more dramatic than I expected. Machine Hardware Info This laptop is running Debian 13 (Trixie/Sid) with the following specs: ...

Setting up NVIDIA CUDA on Debian 13 (Trixie/Sid) to run Large Language Models (LLMs) can be a bit of a journey, especially if you’re transitioning from the default open-source drivers to the proprietary stack required for GPGPU workloads. Over the last few days, I’ve been working on getting llama.cpp to run with CUDA on my laptop to see how much of a difference it makes compared to pure CPU execution. ...

Note: These are suggested optimizations I have not personally tried yet. I’m blogging them as I’m planning to test them throughout this week. OrbStack is highly optimized for macOS, using a proprietary, high-performance networking stack and a custom VirtioFS implementation with aggressive caching. Podman, while being open-source and standard-compliant, can be tuned to significantly bridge the performance gap. The following plan outlines key areas where Podman’s performance can be improved on macOS: ...

Have you ever wondered exactly which files are being hammered by your Linux system in real-time? While tools like iotop or lsof are great, sometimes you want something more visual, custom, and lightweight. In this post, I’ll walk you through how I built a Real-time File I/O Heatmap using the power of eBPF for data collection and Java 25 for a modern Terminal UI (TUI). What is eBPF and Why Use It? eBPF (Extended Berkeley Packet Filter) is a revolutionary technology that allows you to run sandboxed programs in the Linux kernel without changing kernel source code or loading kernel modules. ...

The Spark The other day I came across a fascinating post on Substack by Skilled Coder about Java data structure performance. The post showed some eye-opening numbers for 10M operations: Get operations: HashMap.get() → ~140 ms TreeMap.get() → ~420 ms ArrayList.get(i) → ~40 ms LinkedList.get(i) → ~2.5 s Insertion (10M elements): ArrayList.add() → ~180 ms HashMap.put() → ~300 ms LinkedList.add() → ~900 ms This got me thinking: how do these numbers compare to Eclipse Collections? And more importantly, how can we calculate these numbers ourselves using open source tools? ...