clm_01d4e85fAs an alternative, we propose transfer with adapter modules.
arXiv 1902.00751 · cs.LG cs.CL stat.ML
Read on arXiv · arXiv non-exclusive license
Claims were extracted automatically by reducer-1 on 2026-11-12. The paper's license does not allow republishing it, so long claims are shortened to a quotation. Read the full text on arXiv.
clm_01d4e85fAs an alternative, we propose transfer with adapter modules.
clm_58052438Adapters attain near state-of-the-art performance, whilst adding only a few parameters per task.
res_f1465b77On GLUE, we attain within 0.4% of the performance of full fine-tuning, adding only 3.6% parameters per task.
clm_bd89623bFor this, we propose a transfer learning strategy that yields compact and extensible downstream models.
clm_38ad86adWe demonstrate on a large and diverse set of text classification tasks that adapters yield parameter-efficient tuning for NLP.
clm_f6dfdb66We propose a simple yet effective, bottleneck architecture.
res_02ecb8dfOn the GLUE benchmark, our strategy almost matches the performance of the fully fine-tuned BERT, but uses only 3% task-specific parameters, while fine-tuning uses 100% task-specific parameters.
clm_6f5ea334In summary, adapter-based tuning yields a single, extensible, model that attains near state-of-the-art performance in text classification.
clm_1c0f6d2bTo achieve these properties, we propose a new bottleneck adapter module.
clm_31a9f285To limit the number of parameters, we propose a bottleneck architecture.
clm_7d7a967eWe show that adapters achieve parameter efficient transfer for text tasks.
res_f65d66bcAdapters achieve a mean GLUE score of 80.0, compared to 80.4 achieved by full fine-tuning.
clm_0b3c1d55Unlike the GLUE tasks, there is no comprehensive set of state-of-the-art numbers for this suite of tasks.
res_bb0a30baAdapter tuning achieves comparable overall score (80.0) to full fine-tuning (80.4) using 1.3× parameters in total, compared to 9×.
res_a1beedd2Training adapters with sizes 0.5− 5% of the original model, performance is within 1% of the competitive published results on BERTLARGE.
res_c0f58109This relates to our observation that for some tasks, fine-tuning only the top layers outperforms full fine-tuning, see Table 2.
Record your result in the Mearc app on your own machine, then use Publish this result with this paper's id (1902.00751) and the claim id shown next to the claim. Mearc creates a file that holds only what you choose to publish. Add it to the public claim data by pull request; a maintainer other than you reviews it before it counts.