---
title: "QLoRA runbook: from dataset to adapter served in multi-LoRA (operational procedure)"
url: https://blog.lo0.es/en/posts/qlora-runbook-dataset-to-served-adapter/
markdown: https://blog.lo0.es/en/posts/qlora-runbook-dataset-to-served-adapter/index.md
lang: en
date: 2026-06-09
lastmod: 2026-09-13
author: "José Manuel Román"
description: "Step-by-step operational procedure to fine-tune an SLM with QLoRA on a single RTX 4090 (environment, dataset, TRL/PEFT script, monitoring, validation, adapter versioning) and serve it in multi-LoRA with vLLM (flags, dynamic loading, LoRAResolver from S3). With real commands."
tags: ["qlora","lora","multi-lora","vllm","peft","trl","bitsandbytes","runbook","operational","fine-tuning","slm","small-models"]
translations:
  es: https://blog.lo0.es/posts/qlora-runbook-fine-tuning-serving/index.md
---

# QLoRA runbook: from dataset to adapter served in multi-LoRA (operational procedure)


> This is the **operational companion** to [QLoRA and multi-LoRA at the limit on small models](https://blog.lo0.es/en/posts/qlora-multi-lora-limit-small-models/index.md). That post takes apart the *why*, NF4, double quantisation, paged optimizers, the adapter maths; this is the *how*, with commands you copy and paste. If you have not read the fundamentals one, read it first: here we assume you know what an adapter is, why the base lives in 4-bit and why the gradient only touches the adapter.

## TL;DR

A reproducible procedure in five phases: **(1)** pin the environment with fixed versions; **(2)** prepare the dataset in chat format; **(3)** train the QLoRA adapter with TRL + PEFT on an **RTX 4090 (24 GB, Ada Lovelace)** using gradient checkpointing, gradient accumulation and `paged_adamw_8bit`; **(4)** validate and version the adapter as an artefact of **megabytes**; **(5)** serve it in **vLLM** with `--enable-lora`, loading it hot without restarting the server and resolving it from object storage. All on-premise, on consumer hardware, without a single data item leaving the perimeter. What follows are the exact commands and the memory budget that separates "it fits" from "OOM".

## The end-to-end flow

> **Diagram:** QLoRA operational pipeline: dataset, training, adapter artefact, registry, vLLM serving — [see the HTML version](https://blog.lo0.es/en/posts/qlora-runbook-dataset-to-served-adapter/)

## Phase 0 — Environment and versions

QLoRA is sensitive to the versions of `bitsandbytes`, `transformers`, `peft` and `trl`: misaligned combinations give dequant errors or adapters that do not load in vLLM. Pin the environment and do not touch it mid-campaign. Reference versions as of June 2026 (check the specific ones in your index; the exact pin matters less than the coherence between them):

```bash
python -m venv .venv && source .venv/bin/activate
pip install --upgrade pip

# Training (producer)
pip install "torch>=2.4" \
            "transformers>=4.50" \
            "peft>=0.14" \
            "trl>=0.15" \
            "bitsandbytes>=0.45" \
            "accelerate>=1.2" \
            "datasets>=3.2"

# Serving (consumer) — in its own environment/image
pip install "vllm>=0.8"
```

What each piece does and why it is pinned:

| Package | Role in the flow | Why the version matters |
|---|---|---|
| `torch` | tensor runtime and CUDA kernels | the CUDA ABI has to match the driver and `bitsandbytes`; a major jump breaks the 4-bit kernels. |
| `transformers` | loads the base, the tokenizer and the `chat_template` | it has to **know the architecture** of the SLM you use; a new model needs a version that supports it. |
| `peft` | implements LoRA/QLoRA: injects the `A,B` matrices and writes the `adapter_config.json` | that `adapter_config.json` is what **vLLM reads** when serving; old versions write fields the serving side does not understand. |
| `trl` | the `SFTTrainer`: the supervised training loop | it integrates `peft` natively; its API (`SFTConfig`) changes between versions, hence the pin. |
| `bitsandbytes` | the NF4 quantisation and `paged_adamw_8bit` | **the most sensitive piece**: a badly compiled binary gives corrupt dequant or hangs on the first step. |
| `accelerate` | orchestrates device, mixed precision and `device_map` | the silent backend of almost everything; misaligning it with `transformers` gives cryptic errors. |
| `datasets` | loads the JSONL (and allows streaming if the corpus is large) | not very sensitive; any recent 3.x will do. |
| `vllm` | the multi-LoRA serving | **separate environment or image**: do not mix its stack with the training `bitsandbytes`. |

The golden rule: **coherence between the four above** (`transformers`, `peft`, `trl`, `bitsandbytes`) matters more than the exact number of each. Pin them when starting a campaign and do not move them until it closes.

Check that the GPU and CUDA are healthy before starting; a badly compiled `bitsandbytes` shows up late:

```bash
python -c "import torch, bitsandbytes; print(torch.cuda.get_device_name(0), torch.cuda.is_available())"
nvidia-smi --query-gpu=name,memory.total,driver_version --format=csv
```

For 100 % sovereignty: download the base once from your internal Hugging Face mirror (or a MinIO with the weights) and export `HF_HOME` to a local volume. Nothing in this flow needs to leave the perimeter.

## Phase 1 — Preparing the dataset

The canonical format for a conversational task is JSONL, one conversation per line, with the model's chat template. Do not invent your own format: use the base tokenizer's `chat_template`, because any mismatch between how you train and how you serve degrades quality silently.

```jsonl
{"messages":[{"role":"system","content":"You are a network support assistant."},{"role":"user","content":"The north wing AP does not come up after the outage."},{"role":"assistant","content":"First confirm the PoE on the port..."}]}
{"messages":[{"role":"user","content":"Generate the VLAN change for customer 42."},{"role":"assistant","content":"interface GigabitEthernet0/3\n switchport access vlan 42..."}]}
```

What each field is and why:

| Field | What it is | Operational note |
|---|---|---|
| `messages` | the complete conversation, a list of turns | one conversation per JSONL line; it is what `apply_chat_template` turns into tokens. |
| `role` | who is speaking: `system`, `user`, `assistant` | the adapter learns to produce the `assistant` turns; the `user`/`system` ones are context, not target. |
| `content` | the text of the turn | the `system` one sets the persona/task; keep it **identical** to the one you will use in production or the adapter drifts out of alignment. |

Operational rules that save grief: watch the **ratio of examples** (a well-curated narrow-task dataset of 2,000-20,000 examples performs better than 200,000 noisy ones), **deduplicate**, and set aside 5-10 % as a validation split that does NOT go into training. Building the corpus from production signal is covered by [Retrain: closing the loop](https://blog.lo0.es/en/posts/retrain-closing-the-loop-feedback-dataset-adapter/index.md).

## Phase 2 — The training script

A minimal, complete script with TRL + PEFT. It trains an r=8 adapter on an 8B SLM quantised to NF4. Each block has its rationale commented.

```python
# train_qlora.py
import torch
from datasets import load_dataset
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import LoraConfig
from trl import SFTConfig, SFTTrainer

BASE = "Qwen/Qwen3-8B"           # or whichever SLM you serve; ALWAYS use the same one in train and serve
OUT  = "adapters/soporte-redes-v1"

# 1) Base frozen and quantised to 4-bit NF4 with double quantisation
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",            # NormalFloat, quantile-optimal for Gaussian weights
    bnb_4bit_use_double_quant=True,       # quantises the scale constants -> ~0.37 bits/param less
    bnb_4bit_compute_dtype=torch.bfloat16 # the matmuls run in BF16 after dequant on the fly
)

tok = AutoTokenizer.from_pretrained(BASE)
model = AutoModelForCausalLM.from_pretrained(
    BASE, quantization_config=bnb, torch_dtype=torch.bfloat16, device_map={"": 0}
)

# 2) The adapter: low rank, attention projections only (aggressive). Raise target_modules if the eval asks for it.
peft_cfg = LoraConfig(
    r=8, lora_alpha=16, lora_dropout=0.05, bias="none", task_type="CAUSAL_LM",
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
)

ds = load_dataset("json", data_files={"train": "data/train.jsonl",
                                      "eval":  "data/eval.jsonl"})

# 3) Training config designed to fit in 24 GB
cfg = SFTConfig(
    output_dir=OUT,
    per_device_train_batch_size=1,        # small real batch
    gradient_accumulation_steps=16,       # EFFECTIVE batch = 1*16 = 16, without paying its VRAM all at once
    gradient_checkpointing=True,          # recomputes activations in backward: trades compute for memory
    optim="paged_adamw_8bit",             # paged optimizer: the airbag against VRAM peaks
    learning_rate=2e-4, lr_scheduler_type="cosine", warmup_ratio=0.03,
    num_train_epochs=3, bf16=True,
    max_length=2048,                      # bounds the sequence: activations scale with it
    logging_steps=10, eval_strategy="steps", eval_steps=100, save_steps=200,
    report_to="none",
)

trainer = SFTTrainer(model=model, args=cfg, peft_config=peft_cfg,
                     train_dataset=ds["train"], eval_dataset=ds["eval"],
                     processing_class=tok)
trainer.train()
trainer.save_model(OUT)                   # saves ONLY the adapter (MB), not the base
```

### `BitsAndBytesConfig` — how the base is quantised

| Option | What it does | Why this value / when to change it |
|---|---|---|
| `load_in_4bit=True` | loads the base weights in 4-bit | it is the foundation of QLoRA: without it the 8B does not even fit to train. |
| `bnb_4bit_quant_type="nf4"` | uses the NF4 format (quantile-optimal for Gaussian weights) | `"fp4"` exists, but NF4 performs better on transformer weights; leave NF4. |
| `bnb_4bit_use_double_quant=True` | quantises the scale constants themselves | saves ~0.37 bits/param (hundreds of MB on an 8B); the margin separating "it fits" from "OOM". Leave it at `True`. |
| `bnb_4bit_compute_dtype=torch.bfloat16` | matmul precision after undoing the quantisation on the fly | BF16 on Ada/Hopper (4090, H100); use `float16` only on GPUs without BF16. |

### `LoraConfig` — the shape of the adapter

| Option | What it does | Why this value / when to change it |
|---|---|---|
| `r=8` | adapter rank: its correction capacity | 4-8 for a narrow task (aggressive); raise it to 16-64 only if the eval shows underfitting. |
| `lora_alpha=16` | scale factor of the delta (effective `α/r`) | common convention `α=2r`; it modulates how much the adapter "weighs" on the base. |
| `lora_dropout=0.05` | regularisation on the adapter | 0.05-0.1 with small datasets (avoids overfit); 0 if the corpus is large. |
| `bias="none"` | does not train the bias terms | `"none"` is the standard; `"all"`/`"lora_only"` rarely help and cost params. |
| `task_type="CAUSAL_LM"` | objective/head type | fixed for a generative LLM. |
| `target_modules=[q,k,v,o]` | which matrices get an adapter | attention only = cheap and aggressive; add `gate_proj`/`up_proj`/`down_proj` (MLP) if the task demands rewriting more behaviour and the eval asks for it. |

### `SFTConfig` — the memory budget and the loop

| Option | What it does | Why this value / when to change it |
|---|---|---|
| `per_device_train_batch_size=1` | microbatch per GPU | 1 on 24 GB; the real batch is built by `gradient_accumulation_steps`. |
| `gradient_accumulation_steps=16` | accumulates 16 microbatches before updating | **effective** batch = 1×16 = 16 without paying its VRAM all at once; raise it if you shorten the sequence and want a bigger effective batch. |
| `gradient_checkpointing=True` | recomputes activations in the backward pass instead of storing them | essential on a 4090: ~20-30 % slower in exchange for much less VRAM. |
| `optim="paged_adamw_8bit"` | Adam optimizer in 8-bit + states pageable to RAM | less state VRAM **and** the airbag that avoids the OOM at the peaks. |
| `learning_rate=2e-4` | learning rate of the adapter | 1e-4-3e-4 is the typical QLoRA range; adapters tolerate a higher LR than a full fine-tune. |
| `lr_scheduler_type="cosine"` | LR decay curve | `cosine` or `linear`; cosine usually gives a smooth drop at the end. |
| `warmup_ratio=0.03` | warms the LR over the first 3 % of steps | avoids the instability of the first steps. |
| `num_train_epochs=3` | complete passes over the dataset | 1-3; watch the *eval loss* so as not to overfit. |
| `bf16=True` | compute and adapter precision | BF16 on Ada/Hopper; `fp16=True` if your GPU has no BF16. |
| `max_length=2048` | maximum sequence length | **the #1 VRAM lever** for activations: shorten it first if there is an OOM. |
| `eval_strategy`/`eval_steps`/`save_steps` | validation and checkpoint cadence | tune them to the dataset size; evaluating often costs time. |

The four pieces that make it fit on a 4090 are: `per_device_train_batch_size=1` + `gradient_accumulation_steps` (a large effective batch without its memory cost all at once), `gradient_checkpointing=True` (recomputing activations instead of storing them) and `optim="paged_adamw_8bit"` (paging states to RAM at the peaks). Remove any of the three with long sequences and you will see the OOM.

A declarative alternative with **Axolotl** if you prefer YAML over Python (same result):

```yaml
base_model: Qwen/Qwen3-8B
load_in_4bit: true
adapter: qlora
lora_r: 8
lora_alpha: 16
lora_target_modules: [q_proj, k_proj, v_proj, o_proj]
sequence_len: 2048
micro_batch_size: 1
gradient_accumulation_steps: 16
gradient_checkpointing: true
optimizer: paged_adamw_8bit
learning_rate: 0.0002
num_epochs: 3
bf16: true
datasets:
  - path: data/train.jsonl
    type: chat_template
```

## Phase 3 — Launch and monitor

```bash
# Simple launch on one GPU
python train_qlora.py

# In another terminal: watch VRAM. If it gets close to the ceiling, lower max_length or raise grad accumulation.
watch -n 2 nvidia-smi --query-gpu=memory.used,memory.total,utilization.gpu --format=csv
```

Approximate VRAM budget when training the 8B on the 4090, and what to touch when it gets tight:

| Component | Approx. VRAM | Lever if there is an OOM |
|---|---:|---|
| 8B NF4 base (frozen) | ~4.0 GB | — (fixed) |
| Adapter + grad + Adam states | ~0.3-0.7 GB | lower `r` |
| Activations (batch × sequence) | ~6-14 GB | lower `max_length`, `batch_size`; raise `grad_accum` |
| Dequant buffers / workspace | ~1-2 GB | — |

Table of quick OOM remedies, in order of cost: **(1)** lower `max_length`; **(2)** confirm `gradient_checkpointing=True`; **(3)** raise `gradient_accumulation_steps` and lower `per_device_train_batch_size` to 1; **(4)** use `paged_adamw_8bit` (already in the script); **(5)** as a last resort lower `r`. If after all that it still does not fit, the sequence or the model is too large for 24 GB: either you bound it, or you move up the hardware.

## Phase 4 — Validating the adapter

Never promote an adapter on the strength of the *training loss*. Measure against the validation split you set aside and against a handful of real prompts.

```python
# quick_eval.py
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16)
tok = AutoTokenizer.from_pretrained("Qwen/Qwen3-8B")
base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-8B", quantization_config=bnb, device_map={"": 0})
model = PeftModel.from_pretrained(base, "adapters/soporte-redes-v1")  # base + adapter

msgs = [{"role": "user", "content": "The north wing AP does not come up after the outage."}]
ids = tok.apply_chat_template(msgs, add_generation_prompt=True, return_tensors="pt").to(0)
print(tok.decode(model.generate(ids, max_new_tokens=256)[0], skip_special_tokens=True))
```

For a serious verdict, run the adapter through your eval suite (the layer described by [LLM evals](https://blog.lo0.es/en/posts/evals-llm-layer-after-tracing/index.md)) and compare against the base **without** an adapter and against the previous version of the adapter. Promote only if it wins on the task metric without regressing on safety/format.

## Phase 5 — Versioning the adapter as an artefact

The adapter is a pair of files of a few MB (`adapter_model.safetensors` + `adapter_config.json`). Treat it as a versioned, signed, traceable artefact, not as a loose file.

```bash
# Reproducible checksum + upload to internal object storage (MinIO/S3)
sha256sum adapters/soporte-redes-v1/adapter_model.safetensors > adapters/soporte-redes-v1/SHA256
aws --endpoint-url https://minio.interno s3 cp \
    adapters/soporte-redes-v1/ s3://adapters/soporte-redes/v1/ --recursive
```

A convention that works: `s3://adapters/<task-or-customer>/<version>/`. Immutable per version, with its `SHA256`. Deleting a customer is deleting a prefix of a few MB, not retraining anything. Versioning 500 adapters costs what versioning 500 heavy configuration files costs.

## Phase 6 — Serving in multi-LoRA with vLLM

The consumer loads **one** shared base and applies the adapter delta per request. Startup with static adapters declared:

```bash
VLLM_ALLOW_RUNTIME_LORA_UPDATING=True \
vllm serve Qwen/Qwen3-8B \
  --enable-lora \
  --max-loras 8 \              # max nº of DIFFERENT adapters per batch (not the total loadable)
  --max-lora-rank 8 \          # = the maximum rank of your adapters; do not inflate it (wastes memory)
  --max-cpu-loras 64 \         # adapters cached in RAM for fast swap to VRAM
  --lora-modules soporte-redes=/srv/adapters/soporte-redes/v1
```

Each flag, what it controls and how to size it:

| Flag / variable | What it controls | How to size it |
|---|---|---|
| `--enable-lora` | turns on adapter support | mandatory; without it, vLLM ignores any `model` that is an adapter. |
| `--max-loras 8` | nº of **different adapters in a single batch** | more adapters per batch makes the SGMV kernels more expensive; 8-32 is reasonable. It is not the total loadable. |
| `--max-lora-rank 8` | maximum rank the server reserves | set it **equal to the real rank** of your adapters (8 here); inflating it wastes VRAM and performance. |
| `--max-cpu-loras 64` | adapters cached in RAM ready to page to VRAM | ≥ nº of active adapters; it is the "bench" from which fast swapping happens. |
| `--lora-modules name=path` | declares **static** adapters at startup | useful for the fixed ones; leave it out if everything goes through dynamic loading/Resolver. |
| `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True` | enables the hot load/unload endpoints | essential for `/v1/load_lora_adapter`; without it, the server is static. |

`--max-loras` limits the different adapters **per batch**, not how many you can have loaded; the bulk lives on CPU (`--max-cpu-loras`) and is paged to VRAM on demand. Set `--max-lora-rank` to the real rank (8 here): inflating it wastes memory and performance. Requests pick the adapter through the `model` field:

```bash
curl http://localhost:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
  "model": "soporte-redes",
  "messages": [{"role":"user","content":"The north wing AP does not come up after the outage."}]
}'
# model:"Qwen/Qwen3-8B" (no adapter) uses the bare base on the same server
```

**Hot loading** of a new adapter without restarting (thanks to `VLLM_ALLOW_RUNTIME_LORA_UPDATING=True`):

```bash
curl -X POST http://localhost:8000/v1/load_lora_adapter -H "Content-Type: application/json" -d '{
  "lora_name": "cliente-42",
  "lora_path": "/srv/adapters/cliente-42/v3"
}'
# and to free VRAM/CPU when a customer goes idle:
curl -X POST http://localhost:8000/v1/unload_lora_adapter -H "Content-Type: application/json" -d '{"lora_name":"cliente-42"}'
```

For multi-tenant at scale, avoid declaring hundreds of adapters by hand: the **LoRAResolver** resolves and loads the adapter from local storage or S3 the first time an unknown `model` arrives, so the server stays lean and the adapters are pulled lazily from your MinIO. The internals of *how* thousands of concurrent adapters are batched (SGMV kernels, unified paging, the heterogeneous gather/scatter) are in [Multi-LoRA serving](https://blog.lo0.es/en/posts/multi-lora-serving-shared-base-concurrent-adapters-sgmv-kernel/index.md); this runbook only switches them on. To squeeze the base's decode throughput on a 4090, combine this with [Optimising decode in vLLM](https://blog.lo0.es/en/posts/optimising-decode-vllm-squeezing-every-token-small-hardware/index.md).

## Serving multi-adapter vs merging per task

Two deployment architectures, and the procedure changes:

**Serving multi-LoRA (the above).** A shared base + N hot adapters. It is the default sovereign pattern: minimum footprint, per-customer isolation, hot-swap. Use standard QLoRA and merge nothing.

**Merging per task.** If you want a single quantised-and-adapted artefact per task (no adapter at runtime), do not merge a standard QLoRA adapter into the 4-bit base: the merge reintroduces precision that NF4 does not represent and on requantising you lose part of what was learnt. For that case, train with **QA-LoRA** (quantization-aware), which merges cleanly onto a quantised base. It is an architecture decision, not a quality one; the conceptual detail is in the [fundamentals post](https://blog.lo0.es/en/posts/qlora-multi-lora-limit-small-models/index.md).

## Checklist of operational gotchas

- **Coherent chat template** between training and serving. The most common and most silent mismatch: you train with one `chat_template` and serve with another. Use the base's on both sides.
- **Exactly the same base** (revision included) in `train` and `serve`. An adapter trained on `Qwen3-8B` is not valid on another revision of the model.
- **`--max-lora-rank` ≥ the rank of ALL the adapters** served together, but no more: inflating it wastes VRAM.
- **KV budget vs `--max-loras`.** The bottleneck in serving is not the adapters (MB), it is the KV cache and concurrency; see [Inverted roofline](https://blog.lo0.es/en/posts/roofline-flips-optimising-small-models/index.md) for the SLM regime.
- **`r` too low** = underfitting if the task demands rewriting a lot of behaviour. Raise `r` only if the eval asks for it.
- **Do not promote on training loss.** Validate against the reserved split + real prompts + a safety regression.
- **Version and make immutable** every adapter with its `SHA256`; never overwrite a served version.

## Applied to on-premise infrastructure

On an **RTX 4090 (24 GB)** the same machine is producer and consumer: you train a customer's adapter in hours and serve it on the same server on top of the shared base. It is the canonical case for multi-tenant demos and platform prototypes.

On a **generic 4×H100 SXM cluster (320 GB, NVLink, native FP8)** QLoRA stops being necessary in order to *fit*, but it serves to **parallelise production** (several adapter jobs at once) and to keep the quantised format consistent between training and serious serving of hundreds of concurrent adapters. The base can run in native FP8; the mechanics of the runbook do not change, only the scale.

## See also

- [QLoRA and multi-LoRA at the limit on small models](https://blog.lo0.es/en/posts/qlora-multi-lora-limit-small-models/index.md) — the fundamentals post: the why of NF4, double quantisation, paged optimizers and the adapter maths. This runbook is its executable face.
- [Multi-LoRA serving](https://blog.lo0.es/en/posts/multi-lora-serving-shared-base-concurrent-adapters-sgmv-kernel/index.md) — the consumer internals we only switch on here: SGMV, unified paging, heterogeneous batching of thousands of adapters.
- [Optimising decode in vLLM](https://blog.lo0.es/en/posts/optimising-decode-vllm-squeezing-every-token-small-hardware/index.md) — how to squeeze the decode throughput of the base you serve the adapters on top of, on a 4090.
- [Retrain: closing the feedback → dataset → adapter loop](https://blog.lo0.es/en/posts/retrain-closing-the-loop-feedback-dataset-adapter/index.md) — where the Phase 1 dataset comes from.
- [LLM evals: the layer after tracing](https://blog.lo0.es/en/posts/evals-llm-layer-after-tracing/index.md) — how to validate the Phase 4 adapter with judgement, not with the training loss.
- [Inverted roofline on small models](https://blog.lo0.es/en/posts/roofline-flips-optimising-small-models/index.md) — the performance regime that explains why the serving bottleneck is the KV cache, not the adapters.
- [Aggressive quantisation: from 4-bit to ternary](https://blog.lo0.es/en/posts/aggressive-quantisation-4-bit-to-ternary/index.md) — what happens to the quantised base below NF4 underneath the adapter.

## References

- Dettmers, T., Pagnoni, A., Holtzman, A., Zettlemoyer, L. *QLoRA: Efficient Finetuning of Quantized LLMs*. NeurIPS 2023. <https://arxiv.org/abs/2305.14314>
- Hugging Face TRL — PEFT integration (SFTTrainer + QLoRA): <https://huggingface.co/docs/trl/peft_integration>
- Hugging Face PEFT: <https://github.com/huggingface/peft>
- bitsandbytes: <https://github.com/bitsandbytes-foundation/bitsandbytes>
- vLLM — LoRA Adapters (serving, dynamic loading, LoRAResolver): <https://docs.vllm.ai/en/stable/features/lora/>
- Axolotl: <https://github.com/axolotl-ai-cloud/axolotl>
- Xu, Y. et al. *QA-LoRA: Quantization-Aware Low-Rank Adaptation*. ICLR 2024. <https://arxiv.org/abs/2309.14717>