Posts

NVIDIA offers free access to their powerful Nemotron models through their NVIDIA Build platform. In this post, I’ll walk you through how to get a free API key and configure it in VS Code to use models like nvidia/nemotron-3-ultra-550b-a55b directly in VS Code. Also you can choose any other model from here. Prerequisites VScode needs to be above 1.27 it needs to have customendpoint supporting Getting Your Free NVIDIA API Key Go to build.nvidia.com Sign in or create an NVIDIA account Navigate to the model you want to use (e.g., Nemotron 3 Ultra) Click “Get API Key” button Note: In my experience, the “Get API Key” button in the UI didn’t work properly (it appeared to do nothing). If this happens to you, here’s how to get your API key: ...

In the world of software engineering, we often obsess over the “Testing Pyramid.” We pour resources into unit tests, integration tests, and E2E suites. These are critical, they tell us that our features work as designed. But there’s a shadowy category of bugs that traditional tests often miss: the architectural “anti-patterns” and “API misuses” that don’t break functionality today but lead to system failures, memory leaks, or portability issues tomorrow. ...

After setting up CUDA on my other laptop, I moved to a different(older) machine that doesn’t have an NVIDIA GPU. This one is an everyday laptop with integrated Intel graphics, but that doesn’t mean we have to settle for slow CPU-only performance. On this machine, I switched to the Vulkan backend for llama.cpp and the results were even more dramatic than I expected. Machine Hardware Info This laptop is running Debian 13 (Trixie/Sid) with the following specs: ...

Setting up NVIDIA CUDA on Debian 13 (Trixie/Sid) to run Large Language Models (LLMs) can be a bit of a journey, especially if you’re transitioning from the default open-source drivers to the proprietary stack required for GPGPU workloads. Over the last few days, I’ve been working on getting llama.cpp to run with CUDA on my laptop to see how much of a difference it makes compared to pure CPU execution. ...

Note: These are suggested optimizations I have not personally tried yet. I’m blogging them as I’m planning to test them throughout this week. OrbStack is highly optimized for macOS, using a proprietary, high-performance networking stack and a custom VirtioFS implementation with aggressive caching. Podman, while being open-source and standard-compliant, can be tuned to significantly bridge the performance gap. The following plan outlines key areas where Podman’s performance can be improved on macOS: ...