wassname/pytorch-lightning
1
0
Fork 0
mirror of https://github.com/wassname/pytorch-lightning.git synced 2026-09-11 12:31:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update learning rate on each backward pass instead of each forward pass. (#1477)

Browse Source 
* change lr scheduler step interval to update every backwards pass instead of every forwards pass

* update CHANGELOG

* fix spacing

* Add TODO to lr schedule update

* remove trailing whitespace

Co-authored-by: William Falcon <waf2107@columbia.edu>
This commit is contained in:
Roshan Rao
2020-04-20 08:03:52 -04:00
committed by GitHub
co-authored by William Falcon
parent 4fca994d0e
commit 0203938af8
2 changed files with 10 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+5 -5
View File
@@ -24,20 +24,21 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Changed the default behaviour to no longer include a NaN check with each training iteration. ([#1475](https://github.com/PyTorchLightning/pytorch-lightning/pull/1475))
- Changed lr schedule step interval behavior to update every backwards pass instead of every forwards pass ([#1476](https://github.com/PyTorchLightning/pytorch-lightning/issues/1476))
- Updated semantic segmentation example with custom u-net and logging ([#1371](https://github.com/PyTorchLightning/pytorch-lightning/pull/1371))
-
### Deprecated
-
-
### Removed
-
-
-
-
### Fixed
@@ -52,7 +53,6 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/).
- Fixed a bug that caused the `callbacks` Trainer argument to reference a global variable ([#1534](https://github.com/PyTorchLightning/pytorch-lightning/pull/1534)).
## [0.7.3] - 2020-04-09
### Added
pytorch_lightning/trainer/training_loop.py
+5 -2
View File
@@ -454,8 +454,11 @@ class TrainerTrainLoopMixin(ABC):
# when returning -1 from train_step, we end epoch early
early_stop_epoch = batch_result == -1
# update lr
self.update_learning_rates(interval='step')
# TODO: consolidate all actions that need to take place only after
# self.accumulate_grad_batches steps (optimizer step, lr update, global step increment)
if (self.batch_idx + 1) % self.accumulate_grad_batches == 0:
# update lr
self.update_learning_rates(interval='step')
# ---------------
# RUN VAL STEP