wassname/Ranger21
1
0
Fork 0
mirror of https://github.com/wassname/Ranger21.git synced 2026-08-22 11:40:21 +08:00
Code Issues Packages Projects Releases Wiki Activity
86 Commits 1 Branch 0 Tags
main
Commit Graph
86 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
wassname 3013c47545 link to https://github.com/kozistr/pytorch_optimizer 2023-05-08 13:35:24 +08:00
Less Wright 1a96777278 Merge pull request #44 from JackKelly/patch-1 
Use collections.abc.Callable
 2022-05-06 10:24:24 -07:00
Jack Kelly b507df6a4e Use collections.abc.Callable 
Fixes #43
 2022-05-06 17:16:44 +01:00
Less Wright 0a906ef9df Merge pull request #26 from nestordemeure/patch-2 
Update README.md
 2021-08-09 17:24:44 -07:00
Nestor Demeure a53096e434 Update README.md 
added link to the Flax implementation
renamed `gradient standardization` into `gradient normalization`
 2021-08-07 00:57:47 +02:00
Less Wright 57add4c944 Merge pull request #23 from TheZothen/main 
Activate/deactivate softplus for MADGRAD & choosing beta softplus
 2021-07-19 17:06:58 -07:00
Zohten f5422d1af0 Activate/deactivate softplus + beta softplus 2021-07-19 21:26:54 +02:00
Less Wright 60c0725d01 add @TheZothen as contributor in header comments 2021-07-18 14:52:27 -07:00
Less Wright e8e136f0a2 Merge pull request #22 from TheZothen/main 
Some fixes when using MADGRAD (Softplus and Stable weight decay)
 2021-07-18 14:48:57 -07:00
Zohten 72825ba6ca Softplus for madgrad denominator 2021-07-18 18:02:09 +02:00
Zohten cd022ff593 fix stable weight decay for madgrad 2021-07-18 18:00:21 +02:00
Zohten d670dbaab5 replace len(list(x.size())) by x.dim() 2021-07-18 17:58:06 +02:00
Less Wright b575cc4cf3 Update README.md 2021-07-10 16:03:32 -07:00
Less Wright 5dc411c119 Update README.md 2021-07-10 16:01:50 -07:00
Less Wright 455d369240 adds softplus, gradient normalization, updated unit norm for 3D+ dims 2021-07-10 15:48:36 -07:00
Less Wright aab030163c extend agc to any dimension 2021-07-02 17:05:25 -07:00
Less Wright ac9bc7af39 Merge pull request #13 from nestordemeure/patch-1 
added reference to Ranger21 paper
 2021-06-28 11:02:00 -07:00
Nestor Demeure ba89132416 added reference to Ranger21 paper 
added bibtex to cite paper
Also updated introduction
 2021-06-28 04:25:28 +02:00
Less Wright adf9c77ef0 Update README.md 2021-06-25 18:46:24 -07:00
Less Wright 4e2dff8e9e display lr in scientific notation for easier viewing 2021-05-16 13:36:55 -07:00
Less Wright c23a2a0e12 Update README.md 2021-05-16 13:23:07 -07:00
Less Wright 0e8eabe89b Update README.md 2021-05-16 13:20:36 -07:00
Less Wright 6073f8cbd0 fix off by 1 iteration on linear warmup, adds show_schedule() to show lr quickly after training 2021-05-16 13:07:49 -07:00
Less Wright d4c7b7b521 Merge branch 'main' of https://github.com/lessw2020/Ranger21 into main 2021-05-10 22:24:29 -07:00
Less Wright a7edd418b1 fix context mgr for closure and grad 2021-05-10 22:24:18 -07:00
Less Wright 2e82f432cb Update README.md 2021-05-01 17:03:39 -07:00
Less Wright adfdd0e618 Update README.md 2021-05-01 17:00:20 -07:00
Less Wright 39dd1bb6db Update README.md 2021-05-01 16:58:10 -07:00
Less Wright de331ad196 Update ranger21.py 2021-05-01 16:15:28 -07:00
Less Wright 3c1cc095cd Ranger21 now training on ImageNet 2021-04-27 22:01:06 -07:00
Less Wright 07b3195440 Update README.md 2021-04-26 19:22:56 -07:00
Less Wright 414f3bc961 Update ranger21.py 2021-04-26 19:09:41 -07:00
Less Wright 873212cbbf Merge pull request #2 from BrianPugh/package 
Package and repo reorg
 2021-04-26 18:43:52 -07:00
Brian Pugh 33808aa833 package initial commit 2021-04-26 09:02:41 -07:00
Brian Pugh dd6d8dbc4e python ignores 2021-04-26 08:50:09 -07:00
Less Wright 399004484b Update README.md 2021-04-25 20:55:18 -07:00
Less Wright a7e9e867b0 Merge branch 'main' of https://github.com/lessw2020/Ranger21 into main 2021-04-25 20:39:47 -07:00
Less Wright 8767ecc844 fix potential key error for lookahead params 2021-04-25 20:39:30 -07:00
Less Wright bb4b5bed12 Update README.md 2021-04-25 14:07:50 -07:00
Less Wright b6bde02a9c Update README.md 2021-04-25 14:07:07 -07:00
Less Wright 2f4269eef3 Update README.md 2021-04-25 13:48:50 -07:00
Less Wright 5bb150c0da linear warmdown 2021-04-25 13:33:28 -07:00
Less Wright 6dad8d4295 adds clear_cache, option to use la params at validation, move la step processing to it's own function 2021-04-25 11:36:19 -07:00
Less Wright 3e697e17e9 Update README.md 2021-04-25 05:06:56 -07:00
Less Wright 1ab4d2b834 update Readme - adding lookahead set new high 2021-04-24 17:49:20 -07:00
Less Wright b70977a626 Update README.md 2021-04-24 17:40:52 -07:00
Less Wright df35971812 NormLoss, Lookahead, new settings display function 2021-04-24 17:30:08 -07:00
Less Wright 551f0a3140 Update README.md 2021-04-23 20:48:39 -07:00
Less Wright fd0be4abe8 Update README.md 2021-04-23 20:45:50 -07:00
Less Wright 0f85142981 fixes cosine warmdown 2021-04-18 21:21:02 -07:00