Series 1 — Large Language Models

What actually fits in 120 GB of unified memory, how fast it really runs, and how to turn the box into an OpenAI-compatible API and a web UI on your own network.

This series treats the Ascent GX10 as what it is: a high-capacity, moderate-bandwidth inference server. The lead article is the map — what the hardware can hold, what it can hold usefully, and which serving engine to reach for. The three that follow are the build.

1. What Fits, and How Fast It Really Goes lead The memory arithmetic, measured throughput on GB10 silicon, the prefill-versus-decode split that decides whether the box feels fast, and an engine selection matrix. Read this before downloading 60 GB of weights.
2. vLLM as an Always-On API build The serving path for everything programmatic: a containerised vLLM on sm_121a, memory flags that behave on a unified pool, a systemd unit so it survives reboots, and a model switcher for the one-model-at-a-time discipline.
3. llama.cpp and Open WebUI: the Interactive Path build Building llama.cpp for compute capability 12.1, the mmap question that has opposite answers on this hardware than on a discrete card, on-demand model swapping, and a browser UI on the LAN.
4. Making It Fast: Quantization, Speculative Decoding, Measurement build NVFP4 against MXFP4 against GGUF, KV-cache quantization, why speculative decoding triples single-stream speed and then evaporates under load, and a benchmark harness so you are tuning against numbers rather than impressions.