res_d3e0c53aWe present QL ORA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance.
arXiv 2305.14314 · cs.LG
Read on arXiv · CC BY 4.0
Claims were extracted automatically by reducer-1 on 2026-11-12. Claims are quoted in full under the paper's license.
res_d3e0c53aWe present QL ORA, an efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance.
res_ee68e978Our best model family, which we name Guanaco, outperforms all previous openly released models on the Vicuna benchmark, reaching 99.3% of the performance level of ChatGPT while only requiring 24 hours of finetuning on a single GPU.
clm_67a616e8Our results show that QLoRA finetuning on a small high-quality dataset leads to state-of-the-art results, even when using smaller models than the previous SoTA.
clm_a40392efFurthermore, we find that current chatbot benchmarks are not trustworthy to accurately evaluate the performance levels of chatbots.
clm_2c3d1814We demonstrate for the first time that it is possible to finetune a quantized 4-bit model without any performance degradation.
res_2591ee43QLORA reduces the average memory requirements of finetuning a 65B parameter model from >780GB of GPU memory to <48GB without degrading the runtime or predictive performance compared to a 16bit fully finetuned baseline.
res_d5ad1b27Using QLORA, we train the Guanaco family of models, with the second best model reaching 97.8% of the performance level of ChatGPT on the Vicuna [10] benchmark, while being trainable in less than 12 hours on a single consumer GPU; using a single professional GPU over 24 hours we achieve 99.3% with our largest model, essentially closing the gap to ChatGPT on the Vicuna benchmark.
res_9731b021When deployed, our smallest Guanaco model (7B parameters) requires just 5 GB of memory and outperforms a 26 GB Alpaca model by more than 20 percentage points on the Vicuna benchmark (Table 6).
res_5d481e31QL ORA introduces multiple innovations designed to reduce memory use without sacrificing performance: (1) 4-bit NormalFloat, an information theoretically optimal quantization data type for normally distributed data that yields better empirical results than 4-bit Integers and 4-bit Floats.
clm_cca1fce1In addition to showing that QL ORA recovers 16-bit performance (§4) and training a state-of-the-art chatbot, Guanaco, (§5), we also analyze trends in the trained models.
res_c75a3568First, we find that data quality is far more important than dataset size, e.g., a 9k sample dataset (OASST1) outperformed a 450k sample dataset (FLAN v2, subsampled) on chatbot performance, even when both are meant to support instruction following generalization.
clm_f96fa7c9Second, we show that strong Massive Multitask Language Understanding (MMLU) benchmark performance does not imply strong Vicuna chatbot benchmark performance and vice versa—in other words, dataset suitability matters more than size for a given task.
clm_dbdd684fWe find that GPT-4 and human evaluations largely agree on the rank of model performance in the tournaments, but we also find there are instances of strong disagreement.
res_7d0412b7QL ORA improves over LoRA by quantizing the transformer model to 4-bit precision and using paged optimizers to handle memory spikes.
res_df2fa534QL ORA achieves high-fidelity 4-bit finetuning via two techniques we propose—4-bit NormalFloat (NF4) quantization and Double Quantization.
res_07092a764 QLoRA vs. Standard Finetuning We have discussed how QLoRA works and how it can significantly reduce the required memory for finetuning models.
res_49f1da04We improve on the Stanford Alpaca fully finetuned default hyperparameters to construct a strong 16-bit baseline for comparisons.
clm_0a6b17e8As shown in Figure 2 for LLaMA 7B finetuning on Alpaca, we find that the most critical LoRA hyperparameter is how many LoRA adapters are used in total and that LoRA on all linear transformer block layers are required to match full finetuning performance.
res_01e8ec2dThe NormalFloat data type significantly improves the bit-for-bit accuracy gains compared to regular 4-bit Floats.
clm_17642bb7Similarly, we find that default hyperparameters for fully finetuned baselines are undertuned.
clm_118d27f0We do small hyperparameter searches for the 13B and 33B LLaMA models and we find that all hyperparameter settings found at 7B generalize (including number of epochs) except learning rate and batch size.
res_fcf57f80Note this relative score can be higher than 100% if the model achieves a higher absolute score than ChatGPT.
res_191707f4We find a significant ordering effect with GPT-4 increasing the score of the response occurring earlier in the prompt.
res_acd3f9d2Human Evaluation While recent work indicates generative models can be effectively employed for system evaluations [19], the reliability GPT-4 ratings to assess chatbot performance is, to our knowledge, yet to be proven to correlate with human judgments.
clm_131c2c98Based on our automated and human evaluations, we find that the top QL ORA tuned model, Guanaco 65B, which we finetune on a variant of OASST1, is the best-performing open-source chatbot model and offers performance competitive to ChatGPT.
res_9fb05803We find that Guanaco 65B is the best-performing model after GPT-4, achieving 99.3% performance relative to ChatGPT.
res_cbc2f1e9Guanaco 33B has more parameters than the Vicuna 13B model, but uses only 4-bit precision for its weights and is thus much more memory efficient at 21 GB vs 26 GB, providing a three percentage points of improvement over Vicuna 13B.
res_8d400604Elo rankings in Table 7 indicate that Guanaco 33B and 65B models outperform all models besides GPT-4 on the Vicuna and OA benchmarks and that they perform comparably to ChatGPT in line with Table 6.
res_1c08fcbeSo the unique factorization of 1833 is: 2ˆ1 * 3ˆ2 * 17ˆ1 I hope this helps!
clm_323a819bWhen manually comparing generations from ChatGPT and Guanaco 65B on the Vicuna benchmark, we find that subjective preferences start to play an important role as the authors of this paper disagreed on the many preferred responses.
res_310b92b5In addition, in Table 7, we observe that GPT-4 assigns significantly higher scores to its own outputs compared to human ratings, Elo of 1348 vs 1176, which represent an additional 20% probability of winning against an opponent.
res_b228e6fbThe authors show that models trained on this collection outperform other public collections including the original FLAN 2021 [ 62], T0++ [ 50], Super-Natural Instructions [ 60], and OPT-IML [ 29].
clm_7c1b6353We find hyperparameters to be largely robust across datasets.
res_1e7432f0We find that increasing the dataset size and increasing the number of epochs improves MMLU only marginally (0.0 - 0.5 MMLU), while the difference between datasets is up to 40x larger (1.5 - 8.0 MMLU).
res_1ead1e34While increasing the dataset size and training for more than 1 epochs helps with MMLU performance, the difference between datasets are far larger, indicating that dataset quality affects MMLU performance more than dataset size.
clm_7a2e348bWe find that the weights of each hidden unit have different normal distributions.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (2305.14314) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.