Accelerating LLMs on Debian 13: Setting up CUDA for llama.cpp
Setting up NVIDIA CUDA on Debian 13 (Trixie/Sid) to run Large Language Models (LLMs) can be a bit of a journey, especially if you’re transitioning from the default open-source drivers to the proprietary stack required for GPGPU workloads. Over the last few days, I’ve been working on getting llama.cpp to run with CUDA on my laptop to see how much of a difference it makes compared to pure CPU execution. ...