From bc78e88fde68c7b19a77981be87ab10996ced95e Mon Sep 17 00:00:00 2001 From: Less Wright Date: Sun, 18 Apr 2021 12:28:56 -0700 Subject: [PATCH] Update README.md --- README.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/README.md b/README.md index a660b0c..2c55def 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,12 @@ Ranger, with Radam + Lookahead core, is now 1.5 years old. In the interim, a nu Thus, Ranger21 (as in 2021) is a rewrite with multiple new additions reflective of some of the most impressive papers this past year. The focus for Ranger21 is that these internals will be parameterized, and where possible, automated, so that you can easily test and leverage some of the newest concepts in AI training, to optimize the optimizer on your respective dataset. ### Ranger21 Status:
+ April 18 AM - chebyshev fractals added, cosine warmdown (cosine decay) added
+Chebyshev performed reasonably well, but still needs more work before recommending so it's defaulting to off atm. +There are two papers providing support for using Chebyshev, one of which is: +https://arxiv.org/abs/2010.13335v1
+Cosine warmdown has been added so that the default lr schedule for Ranger21 is linear warmup, flat run at provided lr, and then cosine decay of lr starting at the X% passed in. (Default is .65). + April 17 - building benchmark dataset(s) As a cost effective way of testing Ranger21 and it's various options, currently taking a subset of ImageNet categories and building out at the high level an "ImageSubNet50" and also a few sub category datasets. These are similar in spirit to ImageNette and ImageWoof, but hope to make a few relative improvements including pre-sizing to 224x224 for speed of training/testing. First sub-dataset in progress in ImageBirds, which includes:
n01614925 bald eagle