GGUF Cloud

Deploy any open LLM as your own API

Name: GGUF Cloud
Brand: ModelsLab

Paste a HuggingFace GGUF link, pick a GPU, and get a dedicated OpenAI- and Anthropic-compatible endpoint. Flat monthly price. No DevOps.

Deploy a model API reference View pricing

Build your endpoint

Configure your deployment

Pick a trending model — or paste any HuggingFace GGUF link — then choose a GPU and deploy. We pre-select the cheapest card that fits and re-check it before you pay.

Pick a trending model

Top GGUF models on HuggingFace, updated daily.

Paste any HuggingFace GGUF link

Pick a GPU

One flat monthly price — no per-token or idle charges.

You're deploying

Xing4.0 29B A4B · 256K ctx on RTX 5090 · $749/mo

Bring your own GGUF Deploy Xing4.0 29B A4B on RTX 5090

From link to live endpoint in three steps

No Dockerfiles, no vLLM configs, no GPU shopping. Paste, pick, deploy.

Paste a GGUF link

Drop in a HuggingFace GGUF repo or a direct .gguf URL — or pick one of the trending models. Choose your quant: Q4_K_M, Q5_K_M or Q8_0.

We recommend a GPU

GGUF Cloud reads the model size and pre-selects the cheapest dedicated GPU that fits, showing the flat monthly price before you commit.

Get your endpoint + key

One click spins up a dedicated llama.cpp container. You get an OpenAI- and Anthropic-compatible endpoint and your ModelsLab API key.

Deploy a model

Why GGUF Cloud

Built for the models everyone else turns away

GGUF & llama.cpp native

Bring your own quantization — Q4, Q6 or Q8. The managed LLM hosts force vLLM and safetensors and reject GGUF outright. We run it as a first-class primitive.

Flat monthly per GPU

One predictable price per dedicated GPU. No per-token billing, no idle meter ticking at zero requests — the single most-complained-about thing in the category.

Dedicated single-tenant

Your model runs on its own GPU. No noisy neighbours, no shared-pool throttling, no preemptible boxes vanishing mid-run. Guaranteed throughput.

OpenAI and Anthropic

Every endpoint speaks both protocols natively. Point the OpenAI SDK, the Anthropic SDK, or Claude Code straight at your own model — no translation layer.

Automatic GPU sizing

We compute the VRAM your model needs from its parameters, quant and context, then recommend the cheapest card that fits. Nobody else does this.

Transparent pricing

Every GPU tier and its monthly price is on this page. No "contact sales" wall to find out what a dedicated endpoint actually costs.

Point any SDK at your own model

Your endpoint speaks both the OpenAI and Anthropic protocols natively — no shim, no rewrites. Swap the base URL and keep your existing code.

OpenAI SDK (Python)

Python

1from openai import OpenAI
2
3client = OpenAI(
4    base_url="https://modelslab.com/api/gguf/{id}/v1",
5    api_key="$MODELSLAB_API_KEY",
6)
7
8resp = client.chat.completions.create(
9    model="local",
10    messages=[{"role": "user", "content": "Explain GGUF in one line."}],
11    stream=True,
12)
13for chunk in resp:
14    print(chunk.choices[0].delta.content or "", end="")

Anthropic SDK (Python)

Python

1from anthropic import Anthropic
2
3# Point Claude Code or the Anthropic SDK at your own model.
4client = Anthropic(
5    base_url="https://modelslab.com/api/gguf/{id}",
6    api_key="$MODELSLAB_API_KEY",
7)
8
9msg = client.messages.create(
10    model="local",
11    max_tokens=512,
12    messages=[{"role": "user", "content": "Explain GGUF in one line."}],
13)
14print(msg.content[0].text)

cURL

bash

1curl https://modelslab.com/api/gguf/{id}/v1/chat/completions \
2  -H "Authorization: Bearer $MODELSLAB_API_KEY" \
3  -H "Content-Type: application/json" \
4  -d '{
5    "model": "local",
6    "messages": [{"role": "user", "content": "Hello!"}]
7  }'

Read the full API docs

Deploy a trending model

Llama, Qwen, Mistral, Mixtral, Phi, DeepSeek and more — pre-verified GGUF builds, or paste any HuggingFace repo of your own.

Xing4.0 29B A4B

TrendingNew

31.22B · Q4_K_M · 18.95 GB · 256K ctx

Chat

Fits RTX 5090 · $749/mo

Humanizer

TrendingNew

11.91B · Q4_K_M · 7.63 GB · 256K ctx

Chat

Fits RTX 4090 · $499/mo

Deepseek V4 Flash 0731

TrendingNew

19.85B · BF16 · 11.31 GB · 1024K ctx

Chat

Fits RTX A6000 · $899/mo

Ternary Bonsai 2 27B

TrendingNew

26.9B · BF16 · 0.93 GB · 256K ctx

Chat

Fits A100 80GB · $1,499/mo

Qwen3.8 27B

Trending

27.32B · Q4_K_M · 16.46 GB · 256K ctx

Chat

Fits RTX 3090 · $249/mo

Qwen3.8 27B Gsq Rco

TrendingNew

26.9B · BF16 · 0.93 GB · 256K ctx

Chat

Fits A100 80GB · $1,499/mo

Swift 1.5 Qwen3.8 27B Gsq Rco

TrendingNew

26.9B · IQ2_S · 18.87 GB · 256K ctx

Chat

Fits RTX 3090 · $249/mo

Qwen3.8 9B

TrendingNew

9.2B · Q4_K_M · 5.78 GB · 256K ctx

Chat

Fits RTX 4090 · $499/mo

Qwen3.8 27B

TrendingNew

27.32B · Q4_K_M · 17.12 GB · 256K ctx

Chat

Fits RTX 3090 · $249/mo

Qwen3.8 27B Humanlike Chat

TrendingNew

26.9B · Q4_K_M · 16.56 GB · 256K ctx

Chat

Fits RTX 3090 · $249/mo

Deepseek V4

TrendingMoE

19.85B · IQ2XXS · 1189.42 GB

Chat

Fits RTX 5090 · $749/mo

Xing4.0 29B A4B

TrendingNew

31.22B · IQ4_NL · 20.1 GB · 256K ctx

Chat

Fits RTX 3090 · $249/mo

Twil Lm3

TrendingNew

3.08B · Q4_K_M · 1.92 GB · 64K ctx

Chat

Fits RTX 4090 · $499/mo

Redcell 26B A4B Osint Cyber Apex

TrendingMoE

25.23B · BF16 · 1.19 GB · 256K ctx

Chat

Fits A100 80GB · $1,499/mo

Qwen Image 2.1 Text Encoder Heretic

TrendingNew

8.19B · Q4_K_M · 5.03 GB · 256K ctx

Coding

Fits RTX 4090 · $499/mo

Qwen3 Coder 30B A3B Instruct

TrendingMoE

30.53B · Q4_K_M · 18.56 GB · 256K ctx

Coding

Fits RTX 5090 · $749/mo

Dolphin3 Cyber 8B

Trending

8.03B · Q4_K_M · 4.92 GB · 128K ctx

Custom

Fits RTX 4090 · $499/mo

Qwopus3.6 27B Coder Mtp

TrendingNew

0.46B · Q4_K_M · 16.81 GB

Coding

Fits RTX 4090 · $499/mo

Gemma 4 12B Coder Fable5 Composer2.5 V1

Trending

11.91B · Q4_K_M · 7.38 GB · 256K ctx

Coding

Fits RTX 4090 · $499/mo

Kwaipilot Kat Coder V2.5 Dev

TrendingNewMoE

34.66B · Q4_K_M · 21.39 GB · 256K ctx

Coding

Fits RTX 5090 · $749/mo

Huihui Qwythos 9B Claude Mythos 5 1M Abliterated

TrendingNew

9.2B · BF16 · 19.33 GB · 1024K ctx

Custom

Fits RTX 4090 · $499/mo

Qwen27B Abliterated Fable Mtp

NewMoE

27.32B · Q4_K_M · 16.81 GB · 256K ctx

Custom

Fits RTX 3090 · $249/mo

One flat price per dedicated GPU

Pick the card your model fits on. Billed monthly, cancel anytime — no per-token charges, no idle meter, no surprises.

RTX 3090

Entry

Standard

$249/mo

VRAM: 24 GB
Usable for weights: ~21 GB

Deploy on RTX 3090

RTX 4090

Fast

$499/mo

VRAM: 24 GB
Usable for weights: ~21 GB

Deploy on RTX 4090

RTX 5090

Fastest (consumer)

$749/mo

VRAM: 32 GB
Usable for weights: ~28 GB

Deploy on RTX 5090

RTX A6000

Big VRAM

$899/mo

VRAM: 48 GB
Usable for weights: ~43 GB

Deploy on RTX A6000

A100 80GB

Datacenter

$1,499/mo

VRAM: 80 GB
Usable for weights: ~72 GB

Deploy on A100 80GB

RTX 6000 PRO

Workstation (Blackwell)

$1,999/mo

VRAM: 96 GB
Usable for weights: ~86 GB

Deploy on RTX 6000 PRO

H100 80GB

Datacenter (max)

$2,499/mo

VRAM: 80 GB
Usable for weights: ~72 GB

Deploy on H100 80GB

Flat monthly billing
Cancel anytime
No per-token charges
Pause to stop compute billing

Why teams pick GGUF Cloud

The managed LLM hosts reject GGUF. The dedicated GPU hosts bill hourly and punish idle time. GGUF Cloud is the synthesis.

Capability	GGUF Cloud	HF Endpoints	Together / Fireworks	Featherless
Runs GGUF / llama.cpp	Yes	Yes	Rejects GGUF	FP8 only
Bring your own quant	Q4 / Q6 / Q8	Limited	No	No
Dedicated single-tenant	Yes	Yes	Shared	Shared
Flat monthly price	Per GPU	Hourly	Per token	Yes
No idle / per-token charges	Yes	Idle billed	Per token	Yes
OpenAI + Anthropic API	Both	OpenAI only	Varies	OpenAI only
Automatic GPU recommendation	Yes	No	No	No

Comparison reflects each provider's standard offering as of June 2026. Based on publicly documented pricing and engine support.

GGUF Cloud FAQ

GGUF Cloud is a managed hosting product that turns any open large language model into your own dedicated API. You paste a HuggingFace GGUF link or pick a trending model, GGUF Cloud recommends a GPU that fits, and one click spins up a dedicated llama.cpp (llama-server) container. You get an OpenAI- and Anthropic-compatible endpoint plus your API key, for a flat monthly price per GPU.

Sign up for ModelsLab, open GGUF Cloud, and either paste a HuggingFace GGUF repo (for example bartowski/Meta-Llama-3.1-8B-Instruct-GGUF) or select a trending model. Choose a quantization, accept the recommended GPU, and subscribe. Within a few minutes you receive a dedicated endpoint at https://modelslab.com/api/gguf/{id} that you authenticate with your ModelsLab API key.

Yes. Every GGUF Cloud endpoint exposes both the OpenAI API (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/models) and the Anthropic Messages API (/v1/messages) natively through llama-server. Point the OpenAI SDK at .../api/gguf/{id}/v1, point the Anthropic SDK or Claude Code at .../api/gguf/{id}, and use your ModelsLab API key as the key.

You pay one flat monthly price per dedicated GPU — there are no per-token charges and no idle meter. Tiers run from an RTX 3090 (24 GB) for smaller models up to an H100 (80 GB) for 70B-class models. The exact monthly price for every GPU is listed on the pricing section of this page.

Any GGUF model that fits in a single GPU works — Llama, Qwen, Mistral, Mixtral, Gemma, Phi, DeepSeek distills and more, in Q4_K_M, Q5_K_M or Q8_0. GGUF Cloud reads the model size and recommends the cheapest GPU that fits, from RTX 3090 / 4090 / 5090 and RTX A6000 up to A100 and H100 80 GB.

Shared pools throttle under load, cap quality at FP8, and can drop requests mid-run. A dedicated single-tenant GPU gives you guaranteed throughput, your choice of GGUF quant, and no noisy neighbours — which matters for agents, coding assistants, roleplay and any latency-sensitive workload.

Yes. You can pause an endpoint to stop compute billing while keeping the weights saved (it resumes warm in a minute or two), swap to a different GGUF or quant on the same GPU, resize the GPU tier, or delete the deployment and cancel the subscription at any time.

Deploy your first model in minutes

Bring your own GGUF or pick a trending one. Get a dedicated, single-tenant endpoint that speaks OpenAI and Anthropic — for a price you can predict.

Deploy a model See GPU pricing

Explore Our Other Solutions

Unlock your creative potential and scale your business with ModelsLab's comprehensive suite of AI-powered solutions.

Imagen

AI Image Generation & Tools

Generate, edit, upscale, and transform images with state-of-the-art AI models.

Explore Imagen

Audio Gen

AI Audio Generation

Text-to-speech, voice cloning, music generation, and audio processing APIs.

Explore Audio Gen

Video Fusion

AI Video Generation & Tools

Create, edit, and enhance videos with AI-powered generation and transformation tools.

Explore Video Fusion

3D Verse

Create Stunning 3D Models

Transform images and text into 3D models with advanced AI-powered generation.

Explore 3D Verse

Deploy any open LLM as your own API

Configure your deployment

Pick a trending model

Pick a GPU

From link to live endpoint in three steps

Paste a GGUF link

We recommend a GPU

Get your endpoint + key

Built for the models everyone else turns away

GGUF & llama.cpp native

Flat monthly per GPU

Dedicated single-tenant

OpenAI and Anthropic

Automatic GPU sizing

Transparent pricing

Point any SDK at your own model

OpenAI SDK (Python)

Anthropic SDK (Python)

cURL

Deploy a trending model

Xing4.0 29B A4B

Humanizer

Deepseek V4 Flash 0731

Ternary Bonsai 2 27B

Qwen3.8 27B

Qwen3.8 27B Gsq Rco

Swift 1.5 Qwen3.8 27B Gsq Rco

Qwen3.8 9B

Qwen3.8 27B

Qwen3.8 27B Humanlike Chat

Deepseek V4

Xing4.0 29B A4B

Twil Lm3

Redcell 26B A4B Osint Cyber Apex

Qwen Image 2.1 Text Encoder Heretic

Qwen3 Coder 30B A3B Instruct

Dolphin3 Cyber 8B

Qwopus3.6 27B Coder Mtp

Gemma 4 12B Coder Fable5 Composer2.5 V1

Kwaipilot Kat Coder V2.5 Dev

Huihui Qwythos 9B Claude Mythos 5 1M Abliterated

Qwen27B Abliterated Fable Mtp

One flat price per dedicated GPU

RTX 3090

RTX 4090

RTX 5090

RTX A6000

A100 80GB

RTX 6000 PRO

H100 80GB

Why teams pick GGUF Cloud

GGUF Cloud FAQ

What is GGUF Cloud?

How do I deploy a GGUF model as an API?

Is the endpoint OpenAI and Anthropic compatible?

How much does it cost to host an LLM on GGUF Cloud?

Which models and GPUs can I use?

Why dedicated instead of a shared LLM pool?

Can I pause or swap a deployment?

Deploy your first model in minutes

Explore Our Other Solutions

AI Image Generation & Tools

AI Audio Generation

AI Video Generation & Tools

Create Stunning 3D Models