From 8d9834017ae8efab5a9edfb4c0735770c90c481b Mon Sep 17 00:00:00 2001 From: Less Wright Date: Sat, 10 Apr 2021 21:06:56 -0700 Subject: [PATCH] Update README.md --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 8724e73..3dc23a1 100644 --- a/README.md +++ b/README.md @@ -12,6 +12,9 @@ Thus, you'll be able to simply toggle which opt engine to use, as well as the va Still testing things and then will update code here... Gradient centralization good for both - first findings are gradient centralization definitely improves MadGrad (just like it does with Adam core) so will have GC on as default for both engines. +![madgrad_added_ranger21](https://user-images.githubusercontent.com/46302957/114292041-aca4d480-9a40-11eb-92b3-4243fd6d4390.JPG) + + ### LR selection is very different between MadGrad and Adam core engine: One item - the starting lr for madgrad is very different (much higher) than with Adam....have done some testing with automated LR scheduling (HyperExplorer and ABEL), but that will be added later if it's successful. But if you simply plug your usual Adam LR's into Madgrad you won't be impressed :)