which gpu is worth renting?

rough guide: elementwise ops ~0.2, softmax ~1, conv ~10 to 100, big matmul ~300+

at your intensity

rooflines

token/s ceilings

these are roofline ceilings, not predictions: decode reads every active weight once per step, prefill is ~2 flops per active parameter per token. well-tuned stacks land at 50-85% of the ceiling, none land above. quant sizes are gguf-realistic effective bytes per parameter. geforce/titan prefill uses the fp32-accumulate tensor rate (half the fp16 peak). multi-gpu assumes an ideal tensor-parallel split; interconnect costs extra.