diff --git a/docs/videos/beales.gif b/docs/videos/beales.gif deleted file mode 100644 index 7c3a6f1..0000000 Binary files a/docs/videos/beales.gif and /dev/null differ diff --git a/docs/videos/beales_20171117_00-02-20_2d.gif b/docs/videos/beales_20171117_00-02-20_2d.gif new file mode 100644 index 0000000..08674bd Binary files /dev/null and b/docs/videos/beales_20171117_00-02-20_2d.gif differ diff --git a/docs/videos/beales_CyclicLR_20171117_04-01-14_2d.gif b/docs/videos/beales_CyclicLR_20171117_04-01-14_2d.gif new file mode 100644 index 0000000..7ac96c2 Binary files /dev/null and b/docs/videos/beales_CyclicLR_20171117_04-01-14_2d.gif differ diff --git a/readme.md b/readme.md index 2b52c28..fca0181 100644 --- a/readme.md +++ b/readme.md @@ -4,18 +4,32 @@ This project generates animations of pytorch optimizers solving toy problems. Ex # Examples -![](docs/videos/beales.gif) +Please note that these don't have the same learning rate as simple optimizers perform better on low dimensional problems. So with a constant learning rate SGD gets to the minima too fast to see while Adam crawls along. So I used differen't learning rate per optimizer to show them in the same video. +## With cyclic annealing: + +![](docs/videos/beales_CyclicLR_20171117_04-01-14_2d.gif) + +## Constant learning rate + +### Beales function +![](docs/videos/beales_20171117_00-02-20_2d.gif) +![](docs/videos/beales_20171115_07-18-03_3d.gif) + +### Six humped camel function ![](docs/videos/six_humped_camel_back_20171115_09-38-57.gif) ![](docs/videos/six_humped_camel_back_20171115_09-38-57_3d.gif) ![](docs/videos/six_humped_camel_back_20171115_09-38-57_loss.gif) +### Madsen function + ![](docs/videos/madsen_20171115_10-04-15_3d.gif) ![](docs/videos/madsen_20171115_10-04-15_loss.gif) +### Rosenbrock function + ![](docs/videos/rosenbrock_20171115_09-47-52.gif) -![](docs/videos/beales_20171115_07-18-03_3d.gif) # Usage: