wassname/lora-lite
1
0
Fork 0
mirror of https://github.com/wassname/lora-lite.git synced 2026-06-27 15:15:55 +08:00
Code Issues Packages Projects Releases Wiki Activity

justfile: lr=5e-3 for all antipasto_* cores in bench-variant

Browse Source 
The small-param antipasto family (gain/block/ablate/corda) all need the higher
lr to clear the bf16 round-to-nearest floor, not just antipasto. Glob the case.

Co-Authored-By: Claudypoo <noreply@anthropic.com>
This commit is contained in:
wassname
2026-06-14 19:20:35 +08:00
parent 0d40cc9b38
commit d6b242818a
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
justfile
+1 -1
View File
@@ -88,7 +88,7 @@ bench-variant model variant steps="5000":
delora) lr=1e-3 ;;
ia3) lr=5e-3; target='(k_proj|v_proj)$' ;;
ia3_ff) lr=5e-3; target='(down_proj)$' ;;
antipasto) lr=5e-3 ;; # small params need higher lr
antipasto*) lr=5e-3 ;; # small params (gain/block) need higher lr; covers all antipasto_* cores
esac
exec uv run --extra benchmark python scripts/metamath_gsm8k_benchmark.py \
--model '{{model}}' \