⚠️ This post is archived from my phlog in Gopherspace. Please read my post on the Gopher Protocol to get started!
This is both a server announcement and a guide!
I quadrupled my server memory and installed a GPU.
Quadro P620
Lenovo ST50 “Unqualified DIMM”
Making the most out of my Nvidia p620 (?)
Getting the GPU working
Time to install the appropriate drivers.
sudo apt update
sudo apt install nvidia-driver
I got some warnings about some driver bugs with certain Linux kernel, but after
confirming I wasn’t affected by checking my kernel version uname -r I just
accepted/moved on. My display stopped displaying during the install process.
After reboot, check if it’s working:
nvidia-smi
I got:
baudrillard@simulacra ~ % nvidia-smi
Wed Dec 4 16:18:35 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2 |
|-----------------------------------------+----------------------+----------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+======================+======================|
| 0 Quadro P620 On | 00000000:02:00.0 On | N/A |
| 34% 44C P8 N/A / N/A | 188MiB / 2048MiB | 0% Default |
| | | N/A |
+-----------------------------------------+----------------------+----------------------+
+---------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=======================================================================================|
| 0 N/A N/A 10606 G /usr/lib/xorg/Xorg 185MiB |
+---------------------------------------------------------------------------------------+
Can check OpenGL, if you’re using an actual display (maybe works in xrdp, too):
sudo apt update
sudo apt install mesa-utils
glxinfo | grep OpenGL
Caveats
xrdp
After I installed the GPU I noticed xrdp stopped working!
IDK why but I rebooted or something and now it’s working, so just try connecting a few times, restarting, etc.
Testing ollama
I checked these pages to make sure my GPU is supported:
- https://developer.nvidia.com/cuda-gpus
- https://github.com/ollama/ollama/blob/main/docs/gpu.md
I saw the P620 is supported on both pages, so pretty sure!
I installed the nvidia-cuda-toolkit:
sudo apt install nvidia-cuda-toolkit
Verify CUDA installation:
nvcc --version
Although this prompted me about how gcc-11 has a serious bug that should not be shipped in trixie. I just accepted the risk.
I’m going to install nvtop to see if my GPU is being used by ollama (I ran
ollama run llama3. While nvtop was running I asked ollama to “write a
story that’s mathematically complex” and I saw my GPU usage shoot up. I was also able to see by running
nvidia-smi, you could also do something like watch -n 0.5 nvidia-smi.
All looks pretty good to me!
Final notes
Darn, that fan is sure annoying on that little low-profile P620.
Original content in gopherspace: gopher://gopher.someodd.zip:70/0/phlog/upgrading-server-gpu.gopher.txt