wassname/Ranger21
1
0
Fork 0
mirror of https://github.com/wassname/Ranger21.git synced 2026-09-09 11:15:43 +08:00
Code Issues Packages Projects Releases Wiki Activity

Ranger21 now training on ImageNet

Browse Source
This commit is contained in:
Less Wright
2021-04-27 22:01:06 -07:00
committed by GitHub
parent 07b3195440
commit 3c1cc095cd
1 changed files with 2 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2
View File
@@ -18,6 +18,8 @@ Net results: 14.15% greater accuracy with Ranger21 vs Adam, same training epochs
### Ranger21 Status:</br>
<b> April 27 PM - Ranger21 now training on ImageNet!</b> Starting work on benchmarking Ranger21 on ImageNet. Due to cost, will train to 40 epochs on ImageNet and compare with same setup with 40 epochs using Adam to have a basic "gold standard" comparison. Training is underway now, hope to have results end of this week.
<b> April 26 PM - added smarter auto warmup based on Dickson Neoh report (tested with only 5 epochs), and first pip install setup thanks to @BrianPugh! </b></br>
The warmup structure for Ranger21 is based on the paper by Ma/Yarats which uses the beta2 param to compute the default warmup. However, that also assumes we have a longer training run. @DNH on the fastai forums tested with 5 epochs which meant it never got past warmup phase.
Thus have added a check for the % warmup relative to the total training time and will auto fall back to 30% (settable via warmup_pct_default) in order to account for shorter training runs.</br>