Some nice animations were posted a few years ago by Alex Radford but didn't include Adam or landscapes with noise. Louis Tiao blogged about how to make the visulisations. The pytorch unit tests show how to run the optimizers on a test function. I pulled these together and shared the result at https://github.com/wassname/visualizing_torch_optim. Please make some better animations than I did and share them.

Examples

Usage:

git clone https://github.com/wassname/visualizing_torch_optim
jupyter notebook
open main.ipynb
install any missing dependencies with pip