Articles
Field notes from building and teaching this. Measured on our own hardware, with the numbers shown.
Memory bandwidth, not memory size, decides which model runs well on a DGX Spark
A DGX Spark holds 128 GB, so a 72B model fits. It still generates about five tokens per second. Here is the arithmetic that explains why, and what to run instead.