diff --git a/README.md b/README.md
index 110ca5c..482f01c 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,36 @@
# Ranger21 - integrating the latest deep learning components into a single optimizer
-Ranger deep learning optimizer rewrite to use newest components
+
+A rewrite of the Ranger deep learning optimizer to integrate newer optimization ideas and, in particular:
+
+- uses the AdamW optimizer as its core (or, optionally, MadGrad)
+- Adaptive gradient clipping
+- Gradient centralization
+- Positive-Negative momentum
+- Norm loss
+- Stable weight decay
+- Linear learning rate warm-up
+- Explore-exploit learning rate schedule
+- Lookahead
+
+You can find a full description of our algorithm in the [Ranger21 paper](https://arxiv.org/abs/2106.13731).
+
+## Installation
+
+Until this is up on PyPi, this can either be installed via cloning the package:
+
+```
+git clone https://github.com/lessw2020/Ranger21.git
+cd Ranger21
+python -m pip install -e .
+```
+
+or directly installed from github:
+
+```
+python -m pip install git+https://github.com/lessw2020/Ranger21.git
+```
+
+## History of the project and latest evolutions
Ranger, with Radam + Lookahead core, is now approaching two years old.
*Original publication, Aug 2019: [New deep learning optimizer Ranger](https://lessw.medium.com/new-deep-learning-optimizer-ranger-synergistic-combination-of-radam-lookahead-for-the-best-of-2dc83f79a48d)
@@ -10,8 +41,6 @@ Thus, Ranger21 (as in 2021) is a rewrite with multiple new additions reflective
Full Run on ImageNet in progress - results so far (going to 60 epochs, Ranger21 started later):
-
-
#### Latest Simple Benchmark comparison (Image classification, dog breed subset of ImageNet, ResNet-18):
Ranger 21:
Accuracy: 74.02% Validation Loss: 15.00
@@ -82,7 +111,6 @@ Additional updates include adding a 'clear_cache' to reset the cached lookahead
April 24 - New record on benchmark with NormLoss, Lookahead, PosNeg momo, Stable decay etc. all combined NormLoss and Lookahead integrated into Ranger21 set a new high on our simple benchmark (ResNet 18, subset of ImageWoof).
Best Accuracy = 73.41 Best Val Loss = 15.06
-
For comparison, using plain Adam on this benchmark:
Adam Only Accuracy = 64.84 Best Adam Val Loss = 17.19
@@ -109,7 +137,6 @@ Also did some initial benchmarking to set vanilla Adam as a baseline, and ablati
Here's how the Ranger21 settings output looks atm:
-
April 18 AM - chebyshev fractals added, cosine warmdown (cosine decay) added
Chebyshev performed reasonably well, but still needs more work before recommending so it's defaulting to off atm.
There are two papers providing support for using Chebyshev, one of which is:
@@ -137,7 +164,6 @@ https://arxiv.org/abs/2103.17182
You can toggle between madgrad or not with the use_madgrad = True/False flag:
-
April 10 - madgrad core engine integrated Madgrad has been added in a way that you will be able to select to use MadGrad or Adam as the core 'engine' for the optimizer.
Thus, you'll be able to simply toggle which opt engine to use, as well as the various enhancements (warmup, stable weight decay, gradient_centralization) and thus quickly find the best optimization setup for your specific dataset.
@@ -146,7 +172,6 @@ Gradient centralization good for both - first findings are gradient centralizati
-
### LR selection is very different between MadGrad and Adam core engine:
One item - the starting lr for madgrad is very different (typically higher) than with Adam....have done some testing with automated LR scheduling (HyperExplorer and ABEL), but that will be added later if it's successful. But if you simply plug your usual Adam LR's into Madgrad you won't be impressed :)
@@ -177,19 +202,15 @@ Current feature set planned:
If you have ideas/feedback, feel free to open an issue.
+## Referencing this work
-
-# Installation
-Until this is up on pypi, this can either be installed via cloning the package:
+You can use the following BibTex to cite the [Ranger21 paper](https://arxiv.org/abs/2106.13731) in your research:
```
-git clone https://github.com/lessw2020/Ranger21.git
-cd Ranger21
-python -m pip install -e .
-```
-
-or directly installed from github:
-
-```
-python -m pip install git+https://github.com/lessw2020/Ranger21.git
+@article{wright2021ranger21,
+ title={Ranger21: a synergistic deep learning optimizer},
+ author={Wright, Less and Demeure, Nestor},
+ year={2021},
+ journal={arXiv preprint arXiv:2106.13731},
+}
```