How Much VRAM Do You Need to Fine-Tune an LLM? (Full vs LoRA vs QLoRA)

This post contains affiliate links. We may earn a small commission at no extra cost to you. Recommendations are based on verified specifications and price context reviewed on 2026-07-11.

Most people asking “how much VRAM do I need to fine-tune” have quietly assumed it is close to the inference number. It is not. Inference holds the weights and a little context. Training holds the weights, the gradients, the optimizer states, and the activations for the backward pass — and the optimizer states are the expensive part. The gap between “I can run this model” and “I can train this model” is roughly a factor of eight, and it is the single fact that decides whether your card is enough. The good news is that you almost never need a full fine-tune, and the method that fits a 7B model on a 24GB card is the same method the pros reach for first.

Why training needs so much more VRAM than inference

For inference, the rule of thumb is about 2GB of VRAM per 1B parameters at FP16, or roughly 0.5GB per 1B at 4-bit. A 7B model at 4-bit fits inside a mid-range card because all you are storing is the weights plus a small overhead for context.

Training is a different budget. According to Modal’s fine-tuning guidance, a full fine-tune runs about 16GB per 1B parameters — roughly 8x the inference cost. That multiplier comes from everything training has to keep resident that inference does not:

• Gradients — one value per trainable parameter, same precision as the weights.
• Optimizer states — AdamW keeps two running estimates per parameter, and in a standard full fine-tune those live in FP32. This is usually the largest single line item.
• A master copy of the weights in FP32 for mixed-precision training.
• Activations — the intermediate values saved on the forward pass so gradients can be computed on the backward pass.

Spheron’s breakdown for full fine-tuning adds these up to roughly 18 bytes per parameter before activations: FP16 weights, FP16 gradients, FP32 AdamW optimizer states, and the FP32 master weights. At 18 bytes, a 7B full fine-tune needs about 126GB of base memory — and that is before the activations, which scale with batch size and sequence length. No single consumer GPU is in that conversation. Using an 8-bit or paged optimizer trims this by roughly 30%, which helps, but it does not turn 126GB into anything a 24GB card can hold.

The last term, activations, is the one you can actually control. Activation memory scales with batch size × sequence length, and gradient checkpointing — standard practice for consumer fine-tuning — trades a bit of recomputation for a large drop in activation memory. If a run is spilling, cutting batch size, shortening sequences, and enabling checkpointing are the first levers, in that order.

Full vs LoRA vs QLoRA

There are three ways to fine-tune, and they differ almost entirely in how much of the model you train. That choice, not the GPU, is what sets your VRAM bill.

Full fine-tuning updates every parameter. You pay the full ~18-bytes-per-parameter cost above — the FP16 weights, matching gradients, and the FP32 optimizer states and master weights. It gives you the most control, and for the overwhelming majority of real tasks it is not worth the hardware it demands.

LoRA freezes the base weights and trains a small set of low-rank adapter matrices bolted onto the model. The frozen base still sits in memory in FP16 at about 2 bytes per parameter, but the adapters are tiny, so their gradients and optimizer states are negligible. That is the whole trick: you skip the most expensive line item — full optimizer states across billions of parameters — and only pay it on a rounding-error’s worth of adapter weights. Per Spheron’s and the community QLoRA/LoRA tables on r/LocalLLaMA, a 7B LoRA fine-tune fits comfortably on a 24GB card.

QLoRA is LoRA with one more move: quantize the frozen base weights to 4-bit (~0.5 bytes per parameter) before attaching the adapters. The base model now takes a quarter of the memory it did under LoRA, and the small adapters still train normally. A 7B QLoRA run uses roughly 6-10GB — it fits a 24GB card easily and even fits well below it. Unsloth’s guide shows a 3B QLoRA fine-tune in about 8GB, which is inside the reach of much smaller cards.

Method	What trains	Base weights	7B VRAM (approx)	Fits a 24GB card?
Full fine-tune	Every parameter	FP16 (~2 B/param)	~126GB base, before activations	No
LoRA	Small adapters	FP16, frozen (~2 B/param)	Fits comfortably on 24GB	Yes
QLoRA	Small adapters	4-bit, frozen (~0.5 B/param)	~6-10GB	Yes, easily

The decision rule: full fine-tuning is almost never worth it on consumer hardware. Start with QLoRA. It fits roughly 10x bigger models on the same card for a small quality cost, and for most tasks that quality cost is not one you will notice in the output. Reach for LoRA when you have memory to spare and want the frozen base kept at full precision; reach for full fine-tuning only when you have data and a task that genuinely justify a multi-GPU node.

What fits on your GPU (24GB, 48GB, and up)

Mapping the methods above onto real cards, per RunPod’s budget fine-tuning guidance:

• 24GB (RTX 3090 / RTX 4090): a single 24GB GPU handles 7B and 13B QLoRA fine-tunes. This is the sweet spot for local fine-tuning, and it is the same class of card most people already own for inference. LoRA on a 7B also fits here.
• 32GB (RTX 5090): more headroom for longer sequences and larger batch sizes on the same 7B-13B QLoRA workloads, and a little more room before you have to fight for it.
• 48GB and up: the door to larger targets. 30B and above needs more VRAM or multiple GPUs — a single 24GB card will not carry a 30B-class fine-tune, even under QLoRA, without offloading that hurts.

If you are picking a card specifically to fine-tune, memory capacity is the spec that matters, and memory-per-dollar is how you should rank options. The GPU Value Finder sorts cards on exactly that. To plan a specific model, quantization, and context window, use the fine-tuning VRAM calculator above rather than guessing from the tables.

GPUs worth considering for local fine-tuning

All three of these are 24GB or 32GB cards that will run 7B-13B QLoRA fine-tunes locally. Prices are approximate ranges as of 2026-07-11.

GPU	VRAM	Approx price	Notes
RTX 3090 (used)	24GB	~$1,500-1,700	The value pick. Enough memory for 7B-13B QLoRA, and the cheapest route to 24GB if you buy used.
RTX 4090	24GB	~$3,400-3,600	Same 24GB capacity as the 3090 with more compute — faster runs, same fit limits.
RTX 5090	32GB	~$4,100-4,300	32GB gives extra headroom for longer context and larger batches on the same workloads.

Note that all three sit at 24-32GB. More money here buys you speed, not a bigger model you can fit — the fit ceiling is set by memory, and 24GB and 32GB land in the same tier for QLoRA purposes. If the goal is to fine-tune larger models, the honest answer is that you are looking at multi-GPU or rented data-center cards, not a bigger single consumer GPU. For the full buying decision on inference-plus-fine-tuning builds, see the AI workstation guide.

Rent or buy the GPU?

Fine-tuning is bursty. You run a job, wait, evaluate, adjust, run again — and then often do not touch it for weeks. That pattern favors renting, at least until you know you will do it regularly.

The cloud numbers, per io.net’s fine-tuning budget guide, are small:

• A LoRA 7B fine-tune costs under ~$10 of cloud GPU time.
• A 70B QLoRA run costs roughly ~$15-30.
• A full 70B fine-tune on 8 GPUs for a day runs roughly ~$200-300.

Put that against a used RTX 3090 at ~$1,500-1,700. At under $10 a LoRA run, you would need well over a hundred runs before the card pays for itself on fine-tuning alone. If fine-tuning is the only thing you do, rent — the arithmetic is not close.

The math flips when the card is also doing daily inference. A GPU you already run locally every day for chat, coding, or image work has effectively already paid for itself on that workload, and fine-tuning on it is free compute you already own. That is the real case for buying: not fine-tuning in isolation, but fine-tuning as a bonus on hardware you were going to buy anyway. To run that comparison against your own usage, the local vs cloud break-even calculator turns hours and rates into a payback date.

Common mistakes

• Budgeting fine-tuning like inference. The most expensive mistake in this whole topic. If your model just barely fits for inference, it will not fit for training. Plan for roughly 8x, then pick a method that claws it back.
• Reaching for a full fine-tune first. Full fine-tuning of a 7B model wants ~126GB. Unless you have measured a real quality gap that QLoRA cannot close, you are paying an enormous hardware premium for a difference you may never see.
• Ignoring activation memory. The per-parameter math is the base cost. Batch size and sequence length pile activations on top, and a run that fits at batch size 1 can OOM at batch size 8. Turn on gradient checkpointing before you blame the card.
• Buying a bigger card to fit a bigger model when the capacity is the same. A 4090 does not fit a larger fine-tune than a 3090 — both are 24GB. You are paying for speed, not capacity. Decide which one you actually need.
• Not sanity-checking the target first. Before you buy anything, run your intended model and quantization through the fine-tuning VRAM calculator above. It is faster than reading three forum threads, and it will tell you whether QLoRA on a 24GB card clears your workload or whether you should be renting.
• Confusing training memory with model quality. A QLoRA adapter on a 4-bit base is not a “worse model” for most tasks — it is a smaller training footprint. The quality cost is small and often invisible in the output. Do not overspend on hardware to avoid a cost you have not confirmed matters to you.

If you are still sizing up inference before you get to training, the VRAM requirements guide covers the memory math for running models, which is the floor you build the training budget on top of.

Frequently Asked Questions

Can I fine-tune a 7B model on a 24GB GPU? With QLoRA, yes, comfortably. A 7B QLoRA run uses roughly 6-10GB according to Spheron’s VRAM breakdown, so a single 24GB card like an RTX 3090 or 4090 handles it with headroom. A LoRA fine-tune of a 7B model also fits on 24GB. A full fine-tune of a 7B model does not — that needs roughly 126GB of base memory before activations.

Why does fine-tuning need so much more VRAM than inference? Inference only holds the weights. Training also holds gradients, optimizer states, and activations for the backward pass. Modal’s guidance puts a full fine-tune at roughly 16GB per 1B parameters, versus about 2GB per 1B for FP16 inference — roughly 8x more.

What is the difference between LoRA and QLoRA for VRAM? LoRA freezes the base weights in FP16 (~2 bytes per parameter) and trains small adapters. QLoRA quantizes the frozen base weights to 4-bit (~0.5 bytes per parameter) first, then trains the same adapters. QLoRA uses less memory, which is why it fits bigger models on the same card, at a small quality cost.

Should I ever do a full fine-tune on a consumer GPU? Almost never. Full fine-tuning of a 7B model needs roughly 126GB of base memory, which no single consumer card comes close to. Start with QLoRA — it fits roughly 10x bigger models on the same card and is good enough for the large majority of real tasks.

Is it cheaper to rent a GPU or buy one for fine-tuning? For occasional runs, rent. Per io.net’s budget figures, a LoRA 7B fine-tune costs under ~$10 of cloud GPU time and a 70B QLoRA run is ~$15-30. You need a lot of runs before buying a 24GB card pays back. If you also run inference daily, owning the card changes the math.