AI Economics v1.01Self-Hosting Cost Calculator
Pick an open-weight model, find which Azure GPU VM sizes can actually run it, and see what it would cost to self-host instead of calling a hosted API.
Infrastructure cost
Pick a model and quantization with a compatible VM.
This is the real Azure billing rate — you pay it for every hour the VM runs, regardless of how many requests it actually serves.
Throughput & per-token cost — estimate
Pick a model and compatible VM to see a throughput estimate.
Single-stream, memory-bandwidth-bound estimate — see the methodology note below. Real throughput with request batching can be substantially higher; this is a conservative floor, not a benchmark.
How these estimates work
VRAM requirement = total parameters × bytes per weight (FP16/BF16 = 2, INT8 = 1, INT4/FP4 = 0.5), plus a flat 20% for KV cache and activation memory. Mixture-of-Experts models are sized on total parameters, not active ones — every expert has to fit in memory even though only a fraction runs per token, so MoE gets a compute/throughput advantage but no memory discount.
Throughput uses the standard memory-bandwidth-bound "roofline" estimate for autoregressive decoding: generating each token requires streaming every active parameter through the GPU once, so tokens/second ≈ (aggregate GPU memory bandwidth) ÷ (active parameters × bytes per weight). GPU bandwidth figures come from NVIDIA/AMD published datasheets. This estimate assumes a single request at a time — real deployments serve many requests concurrently and amortize the same weight-streaming cost across all of them, so production throughput (and therefore $/token) is typically better than shown here, not worse.
Prices are Azure pay-as-you-go, Linux, East US 2 — see the Azure GPU VMs page for sourcing. None of this accounts for storage, networking egress, or the engineering time to actually operate a self-hosted deployment.