Back hook (#424)

* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356

* Fixes #356
This commit is contained in:
William Falcon
2019-10-24 07:56:56 -04:00
committed by GitHub
parent a4b43ce095
commit d5ca464cc6
3 changed files with 47 additions and 5 deletions
+22
View File
@@ -115,6 +115,28 @@ def on_before_zero_grad(self, optimizer):
# do something with the optimizer or inspect it.
```
---
#### backward
Called to perform backward step.
Feel free to override as needed.
The loss passed in has already been scaled for accumulated gradients if requested.
```python
def backward(self, use_amp, loss, optimizer):
"""
Override backward with your own implementation if you need to
:param use_amp: Whether amp was requested or not
:param loss: Loss is already scaled by accumulated grads
:param optimizer: Current optimizer being used
:return:
"""
if use_amp:
with amp.scale_loss(loss, optimizer) as scaled_loss:
scaled_loss.backward()
else:
loss.backward()
```
---
#### on_after_backward
Called in the training loop after model.backward()