Comment fine tune un modèle en limitant les coûts computationnels ?

Screenshot 2023-07-18 at 12.16.51.png

Screenshot 2023-07-18 at 12.17.37.png

Screenshot 2023-07-18 at 12.18.48.png

Ajout d’une matrice au niveau des poids avant la couche d’attention

Screenshot 2023-07-18 at 12.20.58.png

Screenshot 2023-07-18 at 12.23.24.png

Screenshot 2023-07-18 at 14.49.22.png

Screenshot 2023-07-18 at 12.34.20.png

Screenshot 2023-07-18 at 12.36.24.png

Prompt tuning perform as much as Multi task of full fine tuning when dealing with large models