Fine-Tuning VRAM Calculator for Local LLMs
Use this for a first-pass training-memory estimate, not a fit guarantee. Mixed-precision AdamW full tuning is modeled at roughly 18 bytes per parameter before activations; LoRA and QLoRA use frozen-base assumptions. Existing calculators already serve this family, so this route remains noindex until direct demand and a human SERP check justify competing.
Fine-Tuning VRAM Calculator
Training needs far more memory than inference. Pick a model, method, and GPU for a conservative planning estimate and a lighter-method fallback when the selected setup is too large.
Mixed-precision AdamW full tuning is modeled at about 18 bytes per parameter before activations. LoRA and QLoRA freeze the base model, but rank, optimizer, kernels, and architecture still matter.
QLoRA (4-bit base) is estimated to fit: about 9.3GB against a 21.6GB planning limit on your 24GB GPU. Real peak memory varies by architecture, adapter rank, optimizer, kernels, and framework.
| Base (8B, QLoRA (4-bit base)) | 4.4GB |
| Activations (batch 1, 2,048 tokens) | 3.4GB |
| Overhead | 1.5GB |
| Total VRAM needed | 9.3GB |
| Your GPU | 24GB |
| Planning limit (10% reserve) | 21.6GB |
The frozen NF4 base is approximately 4-bit plus quantization metadata. Adapter rank, optimizer, kernels, and memory spikes still affect the real peak.
Planning estimate, not a fit guarantee. It assumes gradient checkpointing, a small adapter rank, and the method defaults described above; it does not model dataset packing, attention implementation, distributed optimizer state, or transient allocation peaks. Baselines: Hugging Face memory anatomy and the QLoRA paper. Once you know the GPU class you need, rank cards by memory per dollar in the GPU Value Finder, or check what that card can run afterwards with the VRAM calculator.
What you enter
- Model size (or custom parameters)
- Method: full, LoRA, or QLoRA
- Your GPU VRAM
- Batch size
- Sequence length
What it returns
- Conservative estimated-fit status
- Estimated training VRAM
- A breakdown of base, activations, and overhead
- The first lighter method estimated to fit
Unique value
This implementation exposes its base-memory and activation assumptions and reserves 10% of GPU memory for unmodeled peaks. ApXML, GigaGPU, tune.fyi, and other tools already calculate this family, so uniqueness is not claimed.