arXiv
Hierarchical Low-Rank Reparameterization for Parameter-Efficient Fine-Tuning
Huang R, Xiong Y, Cheng YY et al.
Parameter-efficient fine-tuning (PEFT) methods adapt large models by introducing a small number of trainable parameters.Parameter-efficient fine-tuning (PEFT) methods adapt large models by introducing a small number of trainable parameters. However, we find that these parameters still contain redundancy and can be further compressed while…
Preprint: method