wassname/peft
1
0
Fork 0
mirror of https://github.com/wassname/peft.git synced 2026-09-09 11:28:32 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update other.py

Browse Source 
typo
This commit is contained in:
tpoisonooo
2023-04-03 14:02:13 +08:00
committed by GitHub
parent e536616888
commit 7ef47be5f5
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/peft/utils/other.py
+1 -1
View File
@@ -34,7 +34,7 @@ def prepare_model_for_int8_training(
model, output_embedding_layer_name="lm_head", use_gradient_checkpointing=True, layer_norm_names=["layer_norm"]
):
r"""
This method wrapps the entire protocol for preparing a model before running a training. This includes:
This method wraps the entire protocol for preparing a model before running a training. This includes:
1- Cast the layernorm in fp32 2- making output embedding layer require grads 3- Add the upcasting of the lm
head to fp32