Previouly, a constant value was used for the clipping parameters. This
meant that it stayed at r=1 and d=0 forever, making it essentially
equivalent to regular batch normalization.
* Instance Normalization layer
* fix Instance Normalization for theano broadcasting rules
* Support rank 1 tensor in Instance Normalization
* address issue with axis parameter in instance normalization and add unit test for per channel correctness
* avoid assert_raises in normalization unit test
* Instance normalization set axis default to None
* Began work on batch renormalization
* Correct the implementation to use broadcasting properly, so it supports th dim ordering
* Correct the implementation to use broadcasting properly, so it supports th dim ordering
* Corrected normalizing term for d
* Moved location of updates
* Fixed the training problems and now auto adjusts the r_max and d_max values
* Removed unused code comments
* Finished implementation of batch renormalization
* Corrected the get_config method
* Added tests
* Added user changeable parameter t_delta to change the rate at which time steps increase
* Updated tests to state renorm everywhere
* Attempt a fix for mode=1 test fail (giving nans)
* Applied fix for nans to all calculations involving running mean and std
* Corrected time delta default value
* Reverts a mistake in pytest settings
* Corrections to calculations of r and d due to wrong variable naming (running_std perserves running_variance)
* Added t_delta to get_config
* Fix commit change mixup
* Corrected the check for uses_learning_flag
* Made a few corrections, added moments to backend, added test to backend
* Probable fix for test
* Corrected backend test
* Added support for mode 2
David Stygstra