Scenario · Last verified 2026-08-25
Self-hosted LLM API
A private CPU-side API in front of a local model: request volume versus a small resident instance.
| Metric | Value | Unit | Source type | Derivation | Link |
|---|---|---|---|---|---|
| Weights and scratch | 40 | GB | Author assumption assumption | Tokenizer, adapters, and scratch on object storage. A 70B weight file usually lives on GPU disk, not in this snapshot's object classes. | — |
| API response bytes | 12 | GB | Author assumption assumption | 2 million completions at 6 KB average JSON payload including tokens. 2e6 * 6e3 / 1e9 = 12 GB. | — |
| Completions | 2000000 | request | Author assumption assumption | Internal tool traffic on the order of two million calls a month, not a public chatbot. | — |
Adjust the assumptions
| Vendor | Monthly total at the hub usage |
|---|---|
| Cloudflare | $5.00 |
| AWS | $15.46 |
| DigitalOcean | $18.00 |
Assumptions and limits
This scenario is an API gateway and CPU glue, not a 70B inference bill. GPU instance list prices and token-hour GPU clouds are absent from the snapshot, so anyone costing vLLM or Ollama must price the GPU elsewhere and use this page only for the HTTP front door. Two million completions at twelve gigabytes out will not move hyperscaler internet ladders. The interesting comparison is Lambda wall-clock GB-seconds versus a resident small VM versus Workers CPU-ms plus the Paid platform fee. Long waits on the model are billed on Lambda and mostly not on Workers. Keep cold-start retries out of the request count. Sources for the compute meters: https://aws.amazon.com/lambda/pricing/ https://developers.cloudflare.com/workers/platform/pricing/ https://www.digitalocean.com/pricing/droplets